.NET Mutterings

Developmentor has moved

2012-01-17T21:24:00.001Z

In addition to working hard at RSK myself and Richard are also instructors for Developmtor. Both of us author and teach a range of courses for them. This year has seen Developmentor move from Hammersmith to central London, for all of you who are based in and around London they are offering an evening of tech and nibbles to show case the new location. Details below, hopefully see some of you there.

Come and see us! On Monday 6th February 2012, 5-7pm to cement the move to our new London City location in Moorgate, we invite you to our house warming, entertainment provided by Richard Blewett and Andy Clymerwho will be revealing the magic behind the new and exciting C#5 Async and AWait keywords. To register your interest for this free event, please email us salesuk@develop.com or call 01242 525108 .

Code smarter with Design Patterns

2012-01-12T22:47:00.001Z

Just finished updating the Code Smarter with Design Patterns for Developmentor.

The course traditionally has focused on GoF design patterns, in this release I have added MVC and Repository pattern, to round off the offering.

Both these patterns build upon the GoF patterns discussed earlier in the course, and show how the basic GoF patterns are often compounded to produce elegant solutions.

Other enhancements include addition of Prototype and Builder patterns, and a general upgrade to the latest features offered by the .NET framework.

To learn more click here

Devweek 2012

2011-11-24T10:01:00.001Z

RSK guys will be speaking at DevWeek this year, and as usual we will be talking about a large range of cutting edge topics, including

Our fluffy cuddly colouring in guy Dave Wheeler will be doing a huge number of sessions on Windows 8 Metro including the Keynote to get you all fired up, but don’t think they will be all soft, one session includes a COM primer for all those youngsters who didn’t take their COM apprentiship.

Rich will be covering topics from Whats New in WCF 4.5, and the ever cool Reactive Framework, and a deep dive into the GC, something all developers should have intimate knowledge of.

Andy will be covering the new async functionality of C#5, highlighting how you now write both ui and server side code to consume and enable the use of these new wonderful keywords, and doing a whole day of threading as a pre con on Monday

Kevin will be presenting a day of building MVC applications, along with a session on unit testing.

During the breaks Dr Rock Man will also be there to try and solve your specific technical problems or just happy to hear you rant about the code base you have just inherited.

In total we will be presenting a total 19 sessions, plus pre and post conference sessions. Hopefully see you there with what should be a fun pact week..

Customer Access Terminal V2

2011-08-12T14:44:00.001+01:00

Rich and myself are just about to install the next version of our Customer Access Terminal solution at one of our existing customers. So it seems a convenient time to mention we now have a web site that provides a detailed overview of all the features in V2.

Face 2 Face Asssit

CAT Terminal on film

2011-03-31T16:10:00.001+01:00

The council that recently purchased our CAT kiosk made a short movie for their local tv station, if you are interested in seeing the CAT in action

Click here

The CAT piece is 1 minute 29 seconds into the movie.

CAT Terminal

2011-03-22T16:59:00.001Z

Following up from my recent post on our CAT kiosk deployment.

Here is a link to the councils own article

Customer Access Terminal (CAT)

2011-03-12T14:07:00.001Z

Yesterday was a big day for Rich and myself we successfully deployed our CAT software to our first customer in conjunction with our video conferencing partner First Connections. The software allows members of the public to interact with their local council almost as if they are with them in person. Allowing members of the public to access services from rural locations.

The kiosk utilises two screens, one being a touch screen. The touch screen is used to visit specific council web sites, or to start a video call with a member from one of the council help desk team. Once in a video call the council representative can help complete specific council forms including taking passport style photos with the built in HD camera and customers signatures using a high resolution LCD signature pad.

All built using .NET 4 and Windows Embedded.

Concurrent Data Structures screen cast

2011-03-10T16:26:00.001Z

Must be screen cast Thursday, another screen cast posted to RSK on Concurrent Data Structures in .NET 4

RSK screen casts

async and await screencast

2011-03-10T12:52:00.001Z

Just uploaded a new screencast on async and await keywords due to be introduced in C# 5.

Rock Solid Knowledge Screencasts

TPL Dataflow

2011-02-11T06:59:00.001Z

One parallel pattern that required some work with base .NET 4 was parallel pipe lines. The CTP release of the Parallel Dataflow extensions, makes this trivial now. I’m intending to do a screencast really soon to show this action, but in the mean time try it out for yourself you can download it from here

Coding Challenge,an alternative to regular class room training

2011-02-09T11:35:00.001Z

What do most engineers want to do, “Build Things”. Regular training classes are good at introducing new technology, but what if you want to take it further; to stretch and motivate your senior developers. This was the requirement that our major investment bank client presented to Rock Solid Knowledge the co-developed solution was to produce a Coding Challenge…

The Training Day One

The first day was spent with a deep dive into Silverlight; all teams had prior knowledge of WPF. The focus of the day was to effectively demonstrate the differences and similarities of the two technologies, giving the teams the ability to deliver a high quality Silverlight solution to match the code challenge requirements document.

The Challenge Day Two

The three person teams, of senior developers, spent the next 24 hours straight building a working product to the supplied specification. Not only did this test their ability to apply the new technology but demonstrated who ships and who doesn’t. Failing to organise is clearly highlighted when you only have a short period of time to deliver.

The Assessment Day Three

After working nearly flat out for 24 hours the teams submitted their solutions to be assessed by Rock Solid Knowledge. Initial marks were given, with each team then given a technical grilling on their solution to provide feedback on what was good and bad.

The Winner

The winning team all received iPads, a satisfactory reward for all their hard efforts and our client introduced a new technology in a vibrant and compelling way.

If this has wetted your appetite a more detailed write up can be downloaded from here or if you would just simply like to know how we could bring this experience to your company contact us by clicking here

No more free Reflector

2011-02-03T08:22:00.001Z

Red Gate have announced that from the next release in March no more free versions of Reflector.

http://www.red-gate.com/products/dotnet-development/reflector/announcement

Devweek 2011

2011-01-24T09:59:00.001Z

Just in case you are not aware Dev Week 2011 is fast approaching, this is without doubt the best independent .NET technical conference in the UK. I’m delighted to say that Rock Solid Knowledge will be attending again this year delivering a variety of sessions from Parallel programming to Workflow to fluffy UI. A full list of our sessions can be found here

In addition to the traditional lecture sessions we will be running our drop in clinic where our consultants will help you solve your design and development issues. The whole team will be there so whether its WPF, Silverlight, .NET 4.0, WCF , Design Patterns, Multithreading, Web Development or anything else, we’ll be on hand to lend a sympathetic ear and hopefully solve your problems.

Software Architecture 2010

2010-06-04T12:53:00.001+01:00

Myself and other Rock Solid Knowledge guys have had various talks accepted for Software Architecture conference this October in London.

Rich and myself will be spending a day talking about how to use various bits of .NET 4 technology to build a MVC, WF and Entity Framework based application. The aim to build the app live, so you really get to see how this tech actually ticks.

Through out the week I’ll be doing stuff on Patterns, and Parallel programming.

Finishing the week with another day with Rich talking about parallel programming, all using the latest bits of .NET 4.

Hopefully see you there…

Devweek 2011

2010-04-24T10:18:00.001+01:00

Just heard that Dev Week in London is going to run 14th to 18th March 2011. Rock Solid Knowledge will be submitting a variety of talks…If you have some ideas let us know and we will do our best to accommodate.

And yep I’m sure Dr Rock Man will be back for the drop in consultations.

Demos from DevWeek 2010

2010-03-25T10:12:00.003Z

Thanks to everyone who attended any of the Rock Solid Knowledge sessions at DevWeek we now have all our demos available for download

DevWeek 2010

2010-03-19T16:51:00.001Z

Had a great time again at DevWeek 2010.

All my demos are uploaded to our conference page.

It was the first year we ran our “Ask the Doctor” booth for all those that dropped in I hope you found the advice useful…we certainly had a great time digging into interesting issues, from unit testing, Entity Framework, WCF and general architecture issues..

EF4 And Repository Pattern

2010-03-11T16:06:00.001Z

Just released a screencast on one possible way to implement the repository pattern using EF4. I’ve been very down on EF over the past year or so, but its nice at last to see it actually starting to be usable.

You can listen to this screencast or other RSK screencasts here

Ask the doctor ( Devweek 2010)

2010-02-04T11:09:00.001Z

Dr Rockman is In!

At DevWeek 2010 Rock Solid Knowledge will be running a drop in clinic where our consultants will help you solve your design and development issues. The whole team will be there so whether its WPF, Silverlight, .NET 4.0, WCF , Design Patterns, Multithreading, Web Development or anything else, we’ll be on hand to lend a sympathetic ear and hopefully solve your problems. In addition the RSK crew will be delivering pre and post conference days and a whole bunch of sessions which you can see on our conference page.

Visual Studio Ctrl+Tab Sticking

2010-02-03T10:56:00.001Z

I use Ctrl+Tab inside VS to move between different source files…one day I noticed that when I Ctrl+Tab to the file I wish to view and simply let go of the Ctrl and Tab keys it just sat there, I needed to hit enter to confirm the selection. This started to get annoying other people had mentioned similar problem. It seemed to happen when I was training, and finally the penny dropped…..On Windows 7 I use the magnifier via Windows Key and Plus, to highlight bits of code, and then use Windows Key and minus to revert back to normal resolution. The Magnifier app continues running, it appears that with the magnifier app enabled I have to use Enter to select my file without Ctrl+Tab just works fine…

So now I close the magnifier app so that Ctrl+Tab continues to work.

Devweek 2010

2009-12-01T10:03:00.001Z

Rock Solid Knowledge are pleased to announce that we have had a host of talks accepted for Devweek 2010, including the latest advances in Silverlight, Designing for testing, Enterprise and OO Design Patterns, Workflow and a full day of whats new in .NET 4. Hope to see you there…

For a full list of talks click here

VS2010 Beta 2 Download

2009-10-19T16:43:00.001+01:00

VS2010 Beta 2 is now available for download for MSN subscribers.

Geometric Decomposition Screencast

2009-10-05T14:24:00.001+01:00

I’ve posted a new screen cast on the Rock Solid Knowledge screen casts side that provides an example of how to partition a set of data for parallel processing.

The screen cast covers a simple way using Parallel.ForEach, and a more efficient version combining the use of Barrier.

You can checkout the screen cast and others here

Parallel utilities

2009-10-05T14:21:00.001+01:00

Whilst working with .NET 4 parallel extensions I often find the need to extend the framework to assist me in various day to day tasks. I’ve finally got around to compiling a library of such extensions.

The extensions include the following

Set Process Affinity, so I can see how my algorithm scales on a different number of cores. Will select non hyperthreaded cores first.
Determine the number of Real cores, not including hyperthreading ones.
A Range type that supports the partitioning of a range so that I can farm sub ranges out to different tasks
A Sequence class for building non integer parallel loops or loops with steps other than 1
Finally a simplified way to handle aggregated exceptions.

You can download the library from here

Its a Visual Studio 2010 project, containing the library and a unit test project which should provide enough insight to how the library works.

Im very keen to know what people think of the aggregate exception handling.

Are Singletons Evil ?

2009-10-02T16:54:00.001+01:00

Finally got around to delivering a conference talk on this subject this week at Software Architecture Week, its a topic myself and Kevin Jones are constantly being asked. Of course a quick google reveals the answer there are numerous rants about this evil pattern. But like most things in life its not as simple as yes or no.

Our goal for this talk was to expose the areas were this pattern causes a developer a whole load of pain. In fact one member of the audience was experiencing such pain in his attempt to take legacy code base heavily utilising singletons and start to write unit tests.

So whilst the majority of the time was spent examining the consequences of using the singleton pattern we also took time to highlight that one or two singletons correctly positioned in your application could in fact enable unit testing, and coding to interface without having to refactor large areas of a legacy code base.

This talk took the format of a short geeky play, featuring two developers trying to wrestle with getting the job done and unit testing. You can download the script and accompanying code

So like all good consultants our answer to this question is “It Depends”

Fluent Parallel While

2009-08-14T07:42:00.001+01:00

During devweek 2009 Oliver introduced me to Fluent Api's, I personally love programs that naturally read, after all programs are read far more often than written. This week a posting on the Parallel Extensions blog demonstrated how to achieve parallel while, since we only have Parallel.For and Parallel.ForEach

static void Main(string[] args)
        {
            ConcurrentQueue<int> queue = new ConcurrentQueue<int>();

            for (int i = 0; i < 10; i++)
            {
                queue.Enqueue(i);
            }

           
            Action<ParallelLoopState> processQueue = (lps) =>
            {
                int item;
                if (queue.TryDequeue(out item))
                {
                    Console.WriteLine(" Thread Id = {0} Task Id = {1} : {2}", Thread.CurrentThread.ManagedThreadId, Task.Current.Id, item);
                }
            };

            Func<bool> queueContainsItems = () => queue.IsEmpty == false;

            ParallelUtil.While(new ParallelOptions(), queueContainsItems, processQueue);
        }

I thought I’d combine both these ideas and produce what I would consider a more Fluent version

public static class ParallelUtil
    {
        private static IEnumerable<bool> Infinite()
        {
            while (true) yield return true;
        }

        private static ParallelOptions NoParallelOptions = new ParallelOptions();

        public static void InParallelWhile(this Action<ParallelLoopState> action, Func<bool> condition, ParallelOptions options = null)
        {
            if (options == null) options = NoParallelOptions;

            Parallel.ForEach(IterateForever(), options , 
                (ignored, loopState) =>
                {
                    if (!condition())
                    {
                        loopState.Stop();
                    }
                    else
                    {
                        action(loopState);
                    }
                });

        }

        private static IEnumerable<bool> IterateForever()
        {
            while (true)
            {
                yield return true;
            }
        }
    }
    class Program
    {
        static void Main(string[] args)
        {
            ConcurrentQueue<int> queue = new ConcurrentQueue<int>();

            for (int i = 0; i < 10; i++)
            {
                queue.Enqueue(i);
            }

           
            Action<ParallelLoopState> processQueue = (lps) =>
            {
                int item;
                if (queue.TryDequeue(out item))
                {
                    Console.WriteLine(" Thread Id = {0} Task Id = {1} : {2}", Thread.CurrentThread.ManagedThreadId, Task.Current.Id, item);
                }
            };

            Func<bool> queueContainsItems = () => queue.IsEmpty == false;

            processQueue.InParallelWhile(queueContainsItems);
        }
    }
}

The key difference is the first example you use the following line to process the queue in parallel

ParallelUtil.While(new ParallelOptions(), queueContainsItems, processQueue);

As opposed to the second version with a perhaps more fluent implementation

processQueue.InParallelWhile(queueContainsItems);

Linq for NHibernate

2009-07-31T08:08:00.001+01:00

Its been a while coming, but there it is no possible to use LINQ against NHibernate..I think its a shame that MS didn’t get behind the Hibernate family when they first created LINQ as its a marriage made in heaven, combining a mature ORM with a language integrated query. Rather than spending all that time ( and still ) on trying to create their own ORM.

I’ve yet to try out the integration, but hopefully will be over the next few weeks…

Click Here for the announcement

WARNING..Cancellation support may makes things go slow….

2009-07-30T11:44:00.002+01:00

.NET 4 Tasks offers much better support for task cancellation, unlike QueueUserWorkItem tasks can be cancelled before commencing, and the Task library offers a standard way for running tasks to detect and report cancellation. I recently recorded a screencast that demonstrated the new Task API including the cancellation support. This blog post isn’t so much about the cancellation mechanics but some guidance on how best to use it if you don’t want to change the throughput of your task.

Below is some code that calculates pi, it is expecting to be run inside a task and is supporting the notion of cancellation.

private static double AbortableCalculatePi()
        {
            double pi = 1;
            double multiplier = -1;

            const int N_ITERATIONS = 500000000;

            for (int nIter = 3; nIter < N_ITERATIONS; nIter += 2)
            {
                
                    if (Task.Current.IsCancellationRequested)
                    {
                        Task.Current.AcknowledgeCancellation();
                        return 0.0;
                    }
                

                pi += (1.0 / (double)nIter) * multiplier;
                multiplier *= -1;

            }
            return pi * 4.0;
        }

So all well and good until you benchmark it and compare it to the version with no cancellation support and it runs almost twice as slow. The reason being that the cost of detecting cancellation is high in relation to the work being done. The important aspect to cancellation is being able to respond in a meaningful time for the client, at present we are being way over aggressive in checking.

One option would be to only check every N iterations..

private static double BetterAbortableCalculatePi()
        {
            double pi = 1;
            double multiplier = -1;

            const int N_ITERATIONS = 500000000;

            for (int nIter = 3; nIter < N_ITERATIONS; nIter += 2)
            {

                if ((nIter - 3) % 100000 == 0)
                {
                    if (Task.Current.IsCancellationRequested)
                    {
                        Task.Current.AcknowledgeCancellation();
                        return 0.0;
                    }
                }


                pi += (1.0 / (double)nIter) * multiplier;
                multiplier *= -1;

            }
            return pi * 4.0;
        }

This took a third less time than the previous more aggressive version. However the if block’s effect on the pipeline and the additional maths is still an additional cost over the version that had no cancellation support.

So a third approach is called for this time refactoring the algorithm to use two loops instead of one, were the check for cancellation is done once per iteration of the outerloop, this results in little additional cost.

private static double OptimisedAbortableCalculatePi()
        {
            double pi = 1;
            double multiplier = -1;

            const int N_ITERATIONS = 500000000 / 2;

            const int OUTER_ITERATIONS = 10000;
            const int INNER_ITERATIONS = N_ITERATIONS / OUTER_ITERATIONS;

            int i = 3;
            for (int outerIndex = 0; outerIndex < OUTER_ITERATIONS; outerIndex++)
            {
                for (int nIter = 0; nIter < INNER_ITERATIONS; nIter++)
                {
                    pi += (1.0 / i) * multiplier;
                    multiplier *= -1;
                    i += 2;
                }

                if (Task.Current.IsCancellationRequested)
                {
                    Task.Current.AcknowledgeCancellation();
                    return 0.0;
                }
             
            }
            return pi * 4.0;
        }

Here are the timings are got from the various approaches

NoAbortableCalculatePi = 3.14159264958921 took 00:00:03.7357873

AbortableCalculatePi = 3.14159264958921 took 00:00:09.6137173

BetterAbortableCalculatePi = 3.14159264958921 took 00:00:06.3826212

OptimisedAbortableCalculatePi = 3.14159265758921 took 00:00:03.6883268

As the figures show there is virtually no difference between the first and last run, but a considerable difference when cancellation is inserted into the core of the computation.

So to sum up whilst cancellation support is good the frequency you check for could have an impact on the overall performance of your algorithm. Cancellation is something we want to support but in general users probably don’t need it so we need to strike the right balance between throughput and responding to cancellation in an appropriate timeframe.

.NET 4 Tasks and UI Programming

2009-07-30T09:49:00.001+01:00

Just upload a new screencast covering how to marshal results from .NET 4 Tasks back on to the UI thread. One method is to continue to utilise the same API’s from previous versions of .NET thus utilising SynchronizationContext.Post, the Task based API offers an alternative and in some cases more elegant solution using the ContinueWith method.

My First Silverlight 3 App

2009-07-23T15:15:00.001+01:00

Had a brief rest from patterns and parallel stuff to have a quick play with Silverlight 3. I mainly wanted to see the out of browser aspect, as I think the idea of being able to build RIA that also run on the desktop is very compelling….

So what to build, I really like the iPhone weather app so I thought I’d have a go at reproducing it in Silverlight, below is a screen shot showing it running out of the browser.

I’m using isolated storage to store the list of weather centre’s of interest, a more typical line of business app would store app config on the web server/cloud and potentially locally to support true client roaming, but I’ll leave that for another day.

One thing to note is that the method for enabling Out Of Browser mode for you application is now different from pre-release versions of Silverlight 3, so there are many old blog posts that are unfortunately wrong now. The good news is now it is trivial, view the project properties, and under the Silverlight tab there is an option to enable the app to support out of browser. This then creates the OutOfBrowserSettings.xml file.

You can download the source via here or to just see the app in action click here. I’ll let you decide if its as cool as the iPhone….

Testing on varying number of cores

2009-07-16T18:42:00.001+01:00

I’ve written many blog articles in the past that show that the performance of a piece of parallel code can vary dramatically based on the number of available cores. With that it mind, its obviously desirable even when given a machine with 8 cores that you test your code against a machine that could have substantially less. You can resort to task manager and set Process Affinity and reduce the number of cores available for the process, but this is tedious. There is a .NET API that allows access to controlling which cores to make available for a process. The API requires the use of a bitmask to identity which cores to use, that's a bit ( no pun intended) overkill for what I'm trying to do, so I created a facade that allows me to simply say use N cores.

public static class Cores
    {
        public static int Max
        {
            get
            {
                return Environment.ProcessorCount;
            }
        }
        public static int CoresInUse 
        {
            get
            {
                IntPtr cores = 
                Process.
                    GetCurrentProcess()
                    .ProcessorAffinity;


                int nCores = 0;
                while( cores != IntPtr.Zero )
                {
                    if ( ((int)cores & 1) == 1 )
                    {
                        nCores++;
                    }
                    cores = (IntPtr)((int)cores >> 1);
                }
                return nCores;
            }

            set
            {
                if ((value < 1) || (value > Environment.ProcessorCount))
                {
                    throw new ArgumentException("Illegal number of cores");
                }

                int cores = 1;
                for (int nShift = 0; nShift < value-1; nShift++)
                {
                    cores = 1 | (cores << 1);
                }

                Process.GetCurrentProcess().ProcessorAffinity = (IntPtr)cores;
            }
        }
    }

The following code prints out the number of active cores and then reduces the number of cores to 4

Console.WriteLine("Using {0} out of {1}" , Cores.CoresInUse , Cores.Max);
Cores.CoresInUse = 4;
Console.WriteLine("Using {0} out of {1}", Cores.CoresInUse, Cores.Max);

Guerilla .NET Demos from 6th July 2009

2009-07-11T10:51:00.001+01:00

Had loads of fun as normal teaching Guerilla .NET with Rich and Marcus , All the demos from class here

Parallel.For and Parallel.ForEach…moving past Thread.Sleep

2009-06-26T08:26:00.001+01:00

Just released a screen cast demoing Parallel.For and Parallel.ForEach where we are not demoing the virtues of Parallel Sleep….

Click here to see this and many other Rock Solid Knowledge Screen casts

Short and Long running tasks in .NET 4

2009-06-24T13:19:00.001+01:00

Just added a screen cast on how to create short and long running tasks in .NET 4. You can get to all the Rock Solid Knowledge screen casts via www.rocksolidknowledge.com/screencasts

Rock Solid Knowledge at Software Architect 2009

2009-06-15T19:49:00.001+01:00

Just to say that we will be delivering talks at Software Architect 2009, here is a full list of our sessions.

Long or Short running tasks

2009-06-15T18:10:00.001+01:00

Been playing around with the new version of Pfx for .NET. I must say Ive been very impressed with the improvements since the last CTP for .NET 3.5. So here goes a series of blogs and screen casts on some on various bits of Pfx for .NET 4 BETA 1.

The one thing that really shouts is that the original Pfx types are no longer something for just fine grained parallelism its is a unification of the various threading API’s. Unifying Api’s is something not new to the .NET framework its been happening since 1.1. As developers its great that whilst the framework is evolving a constant effort is being made to refactor and simplify previous complexity.

So previously If I wanted to create a short running piece of background work I would favour the thread pool, if I was to write a long running piece of background activity I would have to create my own custom thread. Creating a short running thread using the thread pool meant either QueueUserWorkItem or Delegate.BeginInvoke, and a long running via new Thread(), and calling Start.

Now for either types of situation you simply create a new Task using the new Task type, either via the Task Factory ( not a real GOF Factory, but don’t get me started on that ), or via new Task()

Task bgTask = Task.Factory.StartNew( MyShortRunningTask);

For a long running task we use the same API but this time giving it a hint that this is a long running task and shouldn’t therefore use a thread pool thread, but create a new thread outside the thread pool

Task bgTask = Task.Factory.StartNew( MyLongRunningTask, TaskCreationOptions.LongRunning);

So too very similar calls but leaving it up to the framework to decide how best to schedule the work.

VS2010 Edit window outside the IDE

2009-06-11T14:34:00.001+01:00

Just noticed that I can drag source windows outside the IDE in VS2010, this is something Ive wanted for ages…Now I can really use multiple monitors

Pfx team examples moving into the real world

2009-06-10T12:13:00.001+01:00

I’ve written many articles on my blog about how Im sick of trade show demos of Pfx ( Parallel framework extensions ). You know the ones using simple Parallel.For with Thread.Sleep or Thread.SpinWait as the piece of work. These examples scale wonderfully but the moment people take those simple examples and apply them to their own for loops terrible performance often results. Thankfully the Pfx team have written a blog article offering some suggestions about what to do when the piece of work inside the for loop is too small. ( Blog article ). Interesting this is the first time I’ve seen them utilise the number of processors in the machine to determine the number of tasks, something I’ve advocated many times in the past for tasks of equal cost.

Trailing comma in C#

2009-06-03T19:58:00.001+01:00

Since C# v3 you have been able to initialise collections leaving a trailing comma…

List<int> vals = new List<int>

{

};

I was asking students in a recent class if they liked it or not…One said that he didn’t like the final comma as he thought it suggested something was missing…My original reason for liking them was that it meant I could easily add to the list. However one student said he liked them as it meant that if you ever did a source control diff on the different versions the diff would only show one line change as opposed to two. I never thought of that…

.NET Screencasts

2009-05-20T18:48:00.001+01:00

Just to say Rock Solid Knowledge of which Im one of the founders have now started to produce a series of screen casts on various bits of .NET technology. With Beta 1 of VS2010 just out expect a load more in the not too distant future covering all the new goodness You can checkout the screen casts either via

http://www.rocksolidknowledge.com/screencasts or subscribe via RSS

Entity Framework and Coarse grain locking

2009-05-12T15:54:00.001+01:00

I’ve recently had to come back and visit the Entity Framework (EF), as part of this I had the need to provide editing for many entities as a single edit operation. In effect what I'm really saying here is that my collection of entities is really one entity as far as my user is concerned, e.g. An Order entity is comprised of a series of Order Item entities, as far as the user request is concerned they are editing an order.

Now out of the box Entity Framework provides support for Optimistic Locking on a per entity basis. However in my case I don’t want to manage optimistic locking at the level of each EF entity, I wish to do it at the collection/or aggregate level. There is no direct support for this in EF, so it has to be done by the application developer. Martin Fowler presents a serious of patterns for solving this problem which he calls the Coarse Grained Lock. The Coarse Grained lock is in effect using a single lock for a collection of items, rather than have a lock per item.

Two possible approaches are put forward one is to nominate a root entity to act as the lock for all other entities, the other is to create a separate entity that represents a shared lock.

The root entity approach works assuming that from any entity in the aggregate you can easily find and fetch the root entity in order to perform the necessary locking.

The shared lock approach requires each entity to have a direct reference to the shared lock.

Neither of these approaches looked especially simple to implement in EF, after playing around with both approaches I finally opted for the shared Optimistic lock pattern.

The use case I opted for was a simple one were I’m modelling an Order and a series of Order Items.

Using simple EF optimistic concurrency I would place a version column in each entity table. The problem with this is that if two users are editing the same order, by simply modifying different order items or adding items there is no way to ensure that at the end of both edits we have a valid order, since the result in the store will be a merge of both edits whilst each edit session will only see the original and its own changes. If the two edits were modifying the same order item a concurrency conflict would be detected. Now in some cases having the ability for two users to edit the same Order concurrently is great, but what if I want to ensure that the composite as a result of both users modifications is in fact valid, perhaps you wish to run some validation logic across the entire composite before saving it.

You could start a system transaction per user update, perform the users updates, reload the entire composite, validate it and if it checks out commit the transaction. However this may be too expensive, and if most of the time there is no contention then this approach is expensive.

By having a shared version across all entities that make up the order we can be sure that when we save the parts of the composite that we have changed that the parts that we haven’t changed are still the same.

When any entity that references a shared version is modified it must update the version entity’s version number, this will only be successfully persisted if the version number in the store is the same as it was when the entity was loaded.

Now to see if EF can implement this, when building the mappings we end up with an entity for each table, and a reference from the Orders and OrderItems to a Versions Entity.

The norm in EF when doing optimistic locking is to use a database timestamp column for the version number, in this case this wasn’t possible since we needed to make the version entity be persisted when a change was made to any entity that referenced it, so the version is represented as an integer, when a change is made to any entity that references that version the version number needs to be updated, and thus it will be persisted by EF when we decide to save our changes.

All of this could be done manually when ever you load an entity be sure to include the Version its associated with and when you wish to perform a modification increment its associated version, version number. This seems tedious and in a production environment missing this step is very likely, the result of which would result in possible corruption assuming two edits happen at the same time. So I wasn’t happy with that, so what I decided to do was to automate this part of the process.

Normally in EF you call SaveChanges on the context to persist any changes, the changes are stored in the context and are available to see what has changed, so all I would need to do was to provide my own SaveChanges and look at the change set and for every entity in that set that references a shared version I increment that version, thus modifying the version and now placing it into the change set.

Unfortunately you can’t override SaveChanges, ( apparently you will in EF 2 ) so you end up writing a new method on the ObjectContext in my case I called it Submit, which performs the necessary checks to see what has changed, and then calls SaveChanges. Note that the type SharedVersion represents the entities in the Version table.

1:

   2:  public partial class AcmeWidgets

   3:  {

   4:  public void Submit()

   5:  {

   6:      // Establish all the entities that have changed

   7:      //

   8:      var entities = this.ObjectStateManager

   9:          .GetObjectStateEntries(EntityState.Modified | EntityState.Added );

10:

  11:      var toIncrement = new Dictionary<int, SharedVersion>();

12:

  13:      // Foreach entity that has changed, if it has a shared version

  14:      // then increment it.

  15:      foreach (ObjectStateEntry attachedObject in entities.Where(e => e.IsRelationship == false ) )

  16:      {

  17:          // If No property of type SharedVersion then it will not attempt to increment the version

  18:          // if many it will throw an exception

  19:          // if just one will increment the version number, assuming it has not

  20:          // previously been incremented

  21:          PropertyInfo versionProperty = (from prop in attachedObject.Entity.GetType().GetProperties()

  22:                                          where prop.PropertyType == typeof(SharedVersion)

  23:                                          select prop)

  24:                                          .DefaultIfEmpty(null)

  25:                                          .Single();

26:

  27:          if (versionProperty != null)

  28:          {

  29:              SharedVersion versionEntity = (SharedVersion) versionProperty.GetValue(attachedObject.Entity, null);

30:

  31:              if (versionEntity != null)

  32:              {

33:

  34:                  if (!toIncrement.ContainsKey(versionEntity.id))

  35:                  {

  36:                      toIncrement.Add(versionEntity.id, versionEntity);

  37:                      versionEntity.Increment();

  38:                  }

39:

  40:              }

  41:              else

  42:              {

  43:                  throw new SharedVersionException("You must load the version of the entity if you wish to modify it");

  44:              }

  45:          }

  46:      }

47:

  48:      // Now call down to the entity framework to do persist

  49:      // any version entities used will now be marked.

  50:      SaveChanges();

  51:  }

  52:  }

In addition to my own Submit method, I also needed to write strongly typed Delete methods on the context, so that if the final part of the composite was deleted the version entity associated with the composite would also be deleted.

Finally it would be nice to automatically assign the version reference to any new object that becomes part of the composite. To do this I register for change on the OrderItems collection of the order and assign it the same version object of the Order its part of.

   1:  public partial class Order

   2:  {

   3:      public Order()

   4:      {

   5:          this.OrderItems.AssociationChanged += new System.ComponentModel.CollectionChangeEventHandler(OrderItems_AssociationChanged);

   6:      }

7:

   8:      void OrderItems_AssociationChanged(object sender, System.ComponentModel.CollectionChangeEventArgs e)

   9:      {

  10:          if (e.Action == System.ComponentModel.CollectionChangeAction.Add)

  11:          {

  12:              OrderItem item = (OrderItem)e.Element;

  13:              if (item.Version == null)

  14:              {

  15:                 item.Version = this.Version;

  16:              }

  17:          }

  18:          else if (e.Action == System.ComponentModel.CollectionChangeAction.Remove)

  19:          {

  20:              this.Version.Increment();

  21:          }

  22:      }

The last part and probably the bit I like least is the fact that you need to ensure that for every entity that is loaded you must insure that the associated version entity is loaded at the same time. This can’t be lazily loaded as its important to know the entity and associated version are in step with each other.

   1:   Order order = ctx.OrderSet

   2:                      .Include("Version")

   3:                      .Include("OrderItems")

   4:                      .Include("OrderItems.Version")

   5:                      .First();

So whilst this all works, its a real shame coarse grained locking has not been considered by the framework providers, EF is at best an ok ORM for simple entities but fails to support out of the box moderately complex real world examples. I think its having to go to this level of effort that still puts developers off betting on a ORM. All the code can be downloaded from here. Note the database creation script is for SQL 2008.

Microsoft's Parallel Framework extensions (Pfx) isn’t always the free lunch its cracked up to be.

2009-04-01T16:05:00.001+01:00

I’ve seen numerous demos of Microsoft's Parallel Framework Extensions ( Pfx), and as you would expect they focus on showing how easy it is to make sequential code go faster on multiple cores. Whilst there is no disputing that the code that is run certainly scales as you add new processors, you can’t get away from the fact that a lot of these demos are contrived, heavy use of Thread.SpinWait to simulate lots of computation for each parallel task. Take away the SpinWait and do some simple one line piece of computation and you will almost certainly end up with a far worse performance than you had with sequential approach. The reason being that creating , scheduling and correlating results is not free. So when deciding to utilise Pfx you need to ensure that you each piece of parallel activity is sufficient in size to benefit from the additional cores. This is were the problem lies whilst its easy to drop in a Thread.SpinWait for demo purposes its often not easy to take your piece of real world computation you wish to parallelise and make each piece parallel activity be of significant size.

One demo I’ve seen over and over again is summing the nodes of a binary tree. The sequential version looks like this

private static int SumTree(Node root)
{

return root.Value + (root.Left == null ? 0 : SumTree(root.Left)) + (root.Right == null ? 0 : SumTree(root.Right));
}

This could then be parallelised like so

private static int ParallelSumTree(Node root)
        {
            int lhsTotal = 0;
            int rhsTotal = 0;

                Future<int> lhs = Future<int>.Create(() => root.Left == null ? 0 : ParallelSumTree(root.Left));
                Future<int> rhs = Future<int>.Create(() => root.Right == null ? 0 :ParallelSumTree(root.Right));
                lhsTotal = lhs.Value;
                rhsTotal = rhs.Value;
            return root.Value + lhsTotal + rhsTotal;
        }

Future<T> is a Pfx type that is used to create a parallel task that yields a result in this case an int. The instance of Future<T> has a Value property that you can use to get the result of the parallel task. If you request the Value and the task has not yet completed you block waiting for the result. I personally really like this model but they have scrapped this type in .NET 4.0 release of Pfx and have the concept that a parallel task can return a value as opposed to void.

This form of parallelisation can work if the computation of both of the futures takes a reasonable time. To demonstrate this I have an implementation that allows the amount of computation to vary, utilising a for…loop. I varied the length of the loop from 0 to 50,000 iterations. With the case of 0 iterations simulating the real task in hand to sum all nodes in the binary tree, and 50,000 to simulate a more expensive operation to perform on each node. I ran the code over a binary tree of 500k nodes with 2 cpus.

What the result shows is that you need to have a workload equivalent to 42k loop iterations for the parallel version to be faster, but note not twice as fast. I then ran the same piece of code on a 8 core machine, here the point in which the parallel version was more effective was around 10k loop iteration mark. This shows that the size of the workload to be effective for parallelisation is also a function of the number of available cores. Today you may well rule out a parallel variant of an algorithm as it doesn’t offer a speed up, whilst in the future when we have say 32+ cores it may well do so, which is why the MS guys are keen to advocate you build for future concurrency not necessarily what you have today. However we obviously don’t want to have a slower parallel version today for the possibility of some future benefit. The reason for this speed up as we add more cores means we run more code in parallel this is not only our algorithm but also the task creation code.

The eight processor version is a lot more compelling than the two processor version, however I'm not comfortable with this, as the speed up I’m getting is not close to 8x speed up which is what the ideal case would be, in fact it is at best about 4 times. In some cases it may be reasonable to accept this, as the effort to make the 4x speed up was very little work, and if we wait a while until we have say 32 procs we will continue to get a speed up just in the same way we did in the past with greater clock speed.

If you do want to squeeze every last drop out of the machine then you will need to think of ways to optimise the algorithm so that the overhead of parallelisation is reduced. In this case its about not having to manage lots of separate tasks. For this algorithm a couple of things stand out that will help here.

The tree is perfectly balanced
The computation for each node is identical

We can take advantage of these two facts along with the knowledge of how many cores we currently have available, and simply divide the tree up between all the cores. That way we take minimal hit for task creation, and each core can run flat out on its given piece of work, and since all work items are equal we would expect all cores to be utilised the same. The two graphs below show this new implementation which has a far better performance.

The two above facts about the tree made producing an efficient parallel implementation reasonably easy. If you had a unbalanced tree it may be worth balancing it or producing a series of other containers that contain the collection of nodes spread evenly. For this to be effective the computational work has to be of greater than the cost of scanning the tree. Alternatively a more dynamic approach could be undertaken were by the algorithm kept track of how many cores it was utilising and when it detected full utilisation it keeps all new activities on the same core as part of the same task. It is often worth considering parallelisation when deciding how to structure your data.

Conclusion

Parallel.For and use of lots of tasks will not necessarily make your code go faster, it may well go slower. The piece of work being parallelised must be of sufficient size to benefit from the overhead of parallel framework and finding that piece of work is often not as easy as you might expect. Going forward as machines have greater and greater number of cpu’s that size of computation necessary to take advantage of the framework will get less. However this will not always result in the most effective us of the cores. If you want to take full advantage of the cores to get maximum performance you will need to structure the algorithm so that understands the environment that it is running in, in order to best utilise resources, and it is this part that often requires the hard thinking. So is the free lunch back? Well to some degree if you want an ok lunch, but for a gut busting lunch you will still have to work for it.

Ruby Ranges in .NET

2009-03-29T20:50:00.001+01:00

Managed to attend some talks whilst I was at DevWeek 2009, one such talk was by Oliver Sturm on the topic of other cool stuff you can do with C# v3 other than Linq. Really enjoyed the talk introduced me to topics like Fluent Api's. If a series of types have a fluent API they allow you to build up code that reads more like a regular sentence.

E.g. invoices.All.OverDueBy(30.Days).SendReminders();

One of the demo's Oliver did was to show a C# implementation of a standard Ruby technique called Ranges.

numbers = [ 1..10]

Here the numbers variable represents all values between 1 and 10. Some obvious things you would like to do with the range is consume it via foreach, or in Ruby speak Each, or determine if a given value is inside the range.

Oliver took based his implementation from Don Box's implementation and modified a few bits.. Don's implementation is based on .NET 2.0 generics and thus when he builds a Range<T> type you are required to provide delegate instances that provide the next item in the range and determine if a given value is inside the range. So that your don't have to provide delegate instances for the basic primitive types Don supplied a series of static helper methods that supply the necessary delegate instances.

I started thinking if this was necessary in C# 3, could we not use the new Expression syntax to automatically build the ranges without the need to supply delegate instances. The idea is that the Range<T> builds the necessary delegate instances by building expressions, I blogged about this technique a while back that whilst you can't do numeric operations on T as part of standard generics you can build expressions on T that you can attempt to perform an Add operation on.

Based on the previous experience of building a Generic Sum method using Expressions I got to work and built an implementation of Range<T> that attempts to build the the necessary next and isIn delegate instances for you.

public class Range<T> : IEnumerable<T>
{
public T Start { get; private set; }
public T End { get; private set; }

private Func<T, T> next;
private Func<T, bool> isIn;

    public Range(T start, T end, T step)
        : this(start, end, step, false)
    {

}

    public Range(T start, T end, T step, bool highToLow)
    {
        InitStartAndEnd(start, end);

ParameterExpression current = Expression.Parameter(typeof(T), "current");

        Expression<Func<T, T>> nextExpr = Expression.Lambda<Func<T, T>>(
            Expression.Add(
             current,
             Expression.Constant(step)),
             current);

Expression<Func<T, bool>> isInExpr = null;

        if (highToLow)
        {
            isInExpr = Expression.Lambda<Func<T, bool>>(
                Expression.And(
                    Expression.GreaterThanOrEqual(current, Expression.Constant(end)),
                    Expression.LessThanOrEqual(current, Expression.Constant(start))),
                    current);
        }
        else
        {
            isInExpr = Expression.Lambda<Func<T, bool>>(
                Expression.And(
                    Expression.GreaterThanOrEqual(current, Expression.Constant(start)),
                    Expression.LessThanOrEqual(current, Expression.Constant(end))),
                    current);
        }

        isIn = isInExpr.Compile();
        next = nextExpr.Compile();
    }

    public Range(T start, T end, Func<T, T> next, Func<T, bool> isIn)
    {
        InitStartAndEnd(start, end);
        this.next = next;
        this.isIn = isIn;
    }

    public bool IsIn(T val)
    {
        return isIn(val);
    }

    public override string ToString()
    {
        return String.Format("{0}..{1}", Start, End);
    }

    private void InitStartAndEnd(T start, T end)
    {
        Start = start;
        End = end;
    }

    public IEnumerator<T> GetEnumerator()
    {
        T current = Start;
        yield return current;

        while (isIn(current))
        {
            current = next(current);
            yield return current;
        }
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

}

With this type defined you now write code like this

var digits = new Range<int> (0,9,1);

I can now consume all the digits via a simple foreach loop, or perhaps use the range as part of a Linq expression since my Range<T> implements IEnumerable<T>.

digits.Except( new int[] { 0,2,4,6,8 } ).ToList().ForEach( i=>Console.WriteLine(i) );

This will print out all the odd numbers from the range..

With the addition of another C# v3 feature extension methods, we can write a "fluent api" style method.

public static class RangeUtil
{

        public static bool In<T>(this T val, Range<T> range)
        {
            return range.IsIn(val);
        }
    }

I can now write

bool isSingleDigitValue = 5.In( digits ) ;

I really wanted to support enumerations without the need to define your own next and IsIn delegate instances but couldn't quite get there. Im sure its possible with a bit more thought. But for now enum ranges need to be defined by creating a new type derived from the Range<T> type.

public enum Places { First, Second, Third, Fourth };

   public class PlacesRange : Range<Places>
   {
       public PlacesRange(Places start, Places end)
           : base(start, end, p => p + 1, p => p>= start && p <=end )
       {

}
}

Note this only works if there are no gaps in the enumeration values.

There are other situations were the normal Add operator is not sufficient

public class DayRange : Range<DateTime>
{
      public DayRange(DateTime start, DateTime end )
          : base(start.Date, end.Date, d => d.AddDays(1), d => d >= start && d <= end )
      {

}
}

To use the DayRange type

DayRange days = new DayRange(new DateTime(2009, 1, 1), new DateTime(2009, 12, 30));

          if (DateTime.Now.In(days))
          {
              Console.WriteLine("You are in Year 2009");
          }

In conclusion Ranges do look pretty cool..however it should be noted that using ranges to represent a simple iteration is far more expensive than a normal for loop, but they are great for building fluent api’s.

You can download the complete source here

Devweek 2009

2009-03-26T18:21:00.001Z

Just posted all my demos from my sessions at devweek, checkout all the Rock Solid Knowledge guys demos via Rock Solid Knowledge Conferences. Also just like to say what a fantastic conference Devweek is, met loads of interesting people and had a lot of fun delivering the talks. Patterns Dartboard worked out really well...Tim tried hard to get me to consider moving to Ruby...and Oliver scared me with F#...

New email client

2009-02-19T13:07:00.001Z

Im sure its not just me but outlook has always seemed very sluggish...and this week I had finally had enough when it stopped synchronising with my IMAP inbox folder...So I reverted back to using Mozilla Thunderbird...And boy its so much sleeker...

Install took seconds...and best of all with a couple of free addins it fully integrates with google calendar and contacts, which is perfect for me especially since I have my iPhone sync directly with google calendar and contacts via a great free service www.nuevaSync.com. This removes the need to sync my iPhone except for Media content...My contacts and calendar are sync'd all the time..

Power Point Merge

2009-01-06T19:43:00.001Z

Im currently preparing my slide decks for Dev Week 2009, and for the day of design patterns Im having to merge five individual decks to make a single deck, now I could do this by hand or I could write a tool to do, and since Ive managed to get the hang of the power point automation api I did just that. You can download the tool from here

Design Patterns Crossword

2008-12-18T16:59:00.001Z

Whilst teaching Developmentors "Code Smarter with Design Patterns in .NET" class a student pointed me towards this Gang of Four patterns crossword puzzle. To test your GOF patterns click here Enjoy!

Sql Server 2008 and TransactionScope just got better

2008-12-18T16:47:00.001Z

Its hard to deny that the TransactionScope programming model created in .NET 2.0 was pretty cool. Initial feelings when I saw it was wow I can now place transactional directives in the business layer and have everything inside that block right down to the data layer enlist in the transaction. However disappointment was only just around the corner when you saw that multiple Open/Close pairs inside the same transaction caused the transaction to be promoted to the DTC ( see previous post ) event though you were talking to the same database instance.

There have been many solutions to this problem, all effectively resulting in changing the way your data layer works to maintain an open connection across multiple data access layer calls. My implementation can be found here, and Alazel Acheson's here

The good news is that this is no longer needed as of Sql Server 2008, the transaction is only promoted when you do indeed involve multiple resources.

Power point to Microsoft One Note

2008-11-19T10:41:00.001Z

I often find the need to produce a one note book of a set of slides so that I can annotate and scribble notes against a given slide. In the past Ive imported the slides manually into a one note book, I finally got around to building a tool that automates the process. You can download it from http://www.beaconsoft.co.uk

PDC 2008, Day one

2008-10-28T13:58:00.001Z

This is my first PDC, so not sure what to expect in terms of quality of speakers and content. Key note on cloud services the new windows platform Windows Azure was a little disappointing based on the fact that it was a developer audience. In essence Microsoft are providing a platform for other companies to take advantage of large distributed data centres. Microsoft has some experience in this space in providing Windows Update, MSDN content , Hotmail , Live services, these services will be ported to the new platform to allow them to self validate the platform, and offer others the ability to out source the data centre services. There will be a short period of time were these services will be free, but in 2009 MS will start to charge for the cloud services based on usage and SLA. Google's cloud based computing has been a serious threat to MS traditional inside the enterprise model, companies have started looking to outsource basic IT, Daily Telegraph recently announced the use of Google for email and office based apps..Its still yet to be seen if large corporations with lots of IPR will ever rely on a third party to hold and manage their data, and this fact plays nicely into Microsoft's offering as they have concentrated on allowing enterprises to utilise the cloud with existing applications inside the enterprise, thus enabling perhaps companies to shift over time, and deciding what is ok to put into the cloud. This is a key advantage they have over the google offering...Other key advantages is that as an MS developer I continue to use the same tools and languages to target the cloud as I do inside the enterprise, and thats huge.

Whilst all that fluffy cloud stuff was interesting, for me the best talk was on C# 4.0 by Anders. Anders is not only a seriously smart guy but a great presenter too. The high lights where

Dynamic language feature support

Co-variance on generic interfaces

Default/Optional parameters

The dynamic language support was very cool.. Microsoft has been working on a DLR ( Dynamic Language Runtime ) for some time to make it easier to write dynamic languages, IronPython and IronRuby utilise common functionality to deliver dynamic language behaviour. C# will now utilise some of that same functionality, by introducing the dynamic keyword. So I can write

dynamic foo = 10

foo = "hello"

foo = 5.6

When you call a method on foo it is late bound, this late binding is done by the fact that foo is of type DynamicObject which has various methods on it like GetMember , InvokeMethod which take a string to represent a Member/Method name. The main aim of this is to allow better interop between staticly typed languages and dynamic ones. Thus calling java script from silverlight is now something like this

<h1 id="title">Hello</h1>

dynamic element = HtmlPage.Document.title;

element.innerText = "Good Bye";

Also COM interop via this mechanism means no PIA's. and a programming model that looks very easy to consume. It also has massive ramifications for Active Record pattern utilised by Ruby on Rails, and for processing XML documents using a dot notation that matches the names of elements. The creation of REST proxies would also seem extremely straight forward.

Support for co-variance List<string> can now be turned into an IEnumerable<object>. Something that has been very frustrating in the past.

Finally we can write C# methods

public int Add( int lhs = 1, int rhs = 1 )

We can therfore call Add(), Add(1) and Add(1,1) to produce the same result or

Add( rhs: 1 , lhs : 1 )

Day 2 is now calling...so hopefully plenty of goodness coming

WARNING...Look after your speakers badge

2008-10-01T10:26:00.001+01:00

I noticed today that there is a site called http://www.whereisandyclymer.com/ and so I clicked on it...and yo and behold its a series of strange locations featuring my Speakers card from Software Architecture week. A quick whois lookup revealed the culprit a fellow Developmentor instructor....

Think I'll be looking after my speaker badge in future...

Random Sampling extension method

2008-10-01T08:48:00.001+01:00

Had reason recently to select a random sample of data from a stream of elements. The amount of samples I needed to take was finite but what was unknown was the number of elements in the input stream. I managed to find various implementations of an IEnumerable extension method on the net, but all the ones I found would cache the entire stream before selecting the sample. This clearly isn't scalable as the input stream increases in size.

After a look around I found this blog article that describes how to implement Reservoir sampling without the need to keep all the items. This then allowed me to write a Random Sample extension method to IEnumerable<T> to do what I needed.

public static IEnumerable<T> RandomSample<T>(this IEnumerable<T> rows, int nSamples)
{
Random rnd = new Random();

T[] sample = new T[nSamples];

int nSamplesTaken = 0;

          foreach (T item in rows)
          {
              if (nSamplesTaken < sample.Length)
              {
                  sample[nSamplesTaken] = item;
              }
              else
              {
                  // As the amount of samples increases the probability
                  // of including a value gets less..due to the fact
                  // that it has a greater chance of surviving if it gets
                  // placed into the sample over earlier selections

                  if (rnd.Next(nSamplesTaken) < nSamples)
                  {
                      sample[rnd.Next(nSamples)] = item;
                  }
              }

nSamplesTaken++;
}

          if (nSamplesTaken >= nSamples)
          {
              return sample;
          }
          else
          {
              return sample.Take(nSamplesTaken);
          }
      }

Chrome support for Silverlight

2008-09-18T15:18:00.001+01:00

The Beta version of Chrome does not fully support silverlight the app appears to start but no screen updates happen unless you keep resizing the browser. Ive just switched to the Dev branch for Chrome and it appears now to be working .

You can switch via this web site

.NET 3.5 SP1 produces a re think of Optimised Lazy Singleton

2008-09-17T07:01:00.001+01:00

In the past I've used a version of double check locking in order to implement efficient lazy creation of a singleton in .NET. This I've found to be the most performant implementation. However I've noticed recently that things are changing, when I run the code on a 64 bit CLR I see no difference between a type that has the before field init attribute present on or not. Both get initialised lazily in release mode and extremely quickly. The double check locking on 64 bit CLR is found to be less performant and therefore an unnecessary complication in the implementation.

In addition Ive recently installed .NET 3.5 SP1 and noticed now that there is a change in the 32 bit CLR for type initialisation. Whilst I still get the normal behaviour of before field init, the lazy CLR based initialisation is now extremely quick, almost the same as non lazy initialisation of the type. Again more importantly faster than my optimised double check locking implementation

My conclusion therefore is that moving forward, I'm comfortable in the fact that the very simple of singleton implementations is now acceptable even in cases where I'm aggressively requesting an instance of the singleton.

Below are two implementations of a singleton, the first using optimised locking the second using the CLR type initialisation to ensure lazy initialisation even in a multi threaded environment.

public class OptimisedLazyLogger
    {
        private static volatile OptimisedLazyLogger instance;
        private static object creationLock = new object();

        private OptimisedLazyLogger()
        {
        }

        public static OptimisedLazyLogger GetInstance()
        {
            if (instance == null)
            {
                CreateInstance();
            }

return instance;
}

        private static void CreateInstance()
        {
            // only one thread at any one time
            // can be inside this block
            lock (creationLock)
            {
                if (instance == null)
                {
                    instance = new OptimisedLazyLogger();
                }
            }
        }

}

Implementation relying on CLR lazy initialisation

class LazyLogger
{
private static LazyLogger instance;

        // Type constructor
        static LazyLogger()
        {
            instance = new LazyLogger();
        }

        // ensure new can not be used outside the type
        // to create an instance
         private LazyLogger()
        {
            Console.WriteLine("Lazy Logger Created");
        }

        // Public method to allow clients to get to the single
        // instance
         public static LazyLogger GetInstance()
        {
            return instance;
        }

        public void LogMsg(string msg)
        {
            Console.WriteLine("Logger: {0}" , msg );
        }

}

Google Chrome, new browser or new desktop os ?

2008-09-03T09:23:00.001+01:00

Today I watched the google chrome demo, and I must say it looks awesome. For me its not just a new browser its an attempt to create a desktop os to run web apps. Even the language they use suggests that the overall aim

"We see chrome as more of a window manager"

Other highlights that elude to this are :-

Each tabs run in separate process and sand boxed, even has its own task manager to kill tabs and plugins

Compiled Java Script, allowing you to utilise java script more in your pages

Web Apps can be run without all the normal browser decoration, so they look like a normal desktop app, good uses of this are google calendar and gmail

And to top it all its open source....and works with any search provider...Other cool features Incognita windows allowing you to search the web with out recording the urls in the web history. This proves to be important based on the fact that it utilises browser history to optimise behaviour.

Google have created a desktop web platform that is optimised for searching and running web apps, perhaps in not so distant future all we will care about is having a machine that runs chrome and thats all we care about....Im for one going to be downloading chrome and start using it...

http://google.client.shareholder.com/Visitors/event/build2/MediaPresentation.cfm?MediaID=33101&Player=1

Developmentor Pfx Article

2008-08-19T08:47:00.001+01:00

I have written an article for Developmentor Developments magazine on the June 2008 CTP of Microsoft Parallel Extensions known as Pfx. To read the article click here

Weak Observer

2008-07-30T14:37:00.001+01:00

During a recent teach of Code Smarter for Design Patterns in .NET in which we teach the Observer pattern showing both traditional GOF implementation as well as a more appropriate .NET version using events. One student after class told me he has never been keen on encouraging the use of event subscriptions as he was never confident that his developers would remember to unsubscribe from events when they no longer needed them. This is in fact has been a common problem in .NET code that can often induce a memory leak.

The typical case that is often presented is one where a Windows Form subscribes to an event from a different object say some application object. When the form subscribes to the event on the application object, the application object therefore has a strong reference to the form so that it can deliver the events. When the form is closed a failure to unsubscribe from the event means that there will still be a strong reference the form that will ultimately prevent it from being gc'd. If this form has a large data table bound to a data grid this could have seriously impact on available memory during the applications life, as this data set will also not be gc'd

Clearly the right thing is for all programmers to remember to unsubscribe, but we know that often programmers forget and these leaks often go unnoticed until a customer screams. An alternative would be for the application object not to hold a reference and thus will not stop the window from being garbage collected, the obvious problem then is how can it deliver the event. There is though a half way house through the use of a Weak Reference. A Weak reference provides a means to reference an object without preventing it from being GC'd, but still allowing you to get to it whilst the GC'd hasn't decided to remove it.

As I said at the start we provide two implementations of the observer pattern one using interfaces, IObserver/IListener the other using .NET events. Implementing a version of the observer pattern using the interface approach combined with Weak references is relatively trivial extension to the standard observer pattern. The subject does not hold on to the observer/listener using a strong reference it is simply held as a weak reference. When the subject wishes to inform the observers of an event it walks through its list of weak refrerences turning each one into a strong reference for the duration of the notification. If it fails to turn a weak reference into a strong reference it knows the observer has been gc'd.

When attempting to implement this behaviour using events things get a little bit more involved as the delegate chain holds the reference to the observer as a normal strong reference, that is something we can't change. This is a shame since we would clearly love to have the choice to hold on to it as a weak reference, thus giving us a similar solution to Weak Observer. In order to give an event style api we will need to be a bit more cunning, and also take a performance hit. What I have done here is to continue to have the normal event subscribe/unsubscribe using a delegate, but then crack open the delegate to reveal the target instance and target method. Once I have the target method and instance I wrap the instance up in a weak reference wrapped up by what I call a WeakEventHandle. After the construction of the handle it is placed into a handle list. When asked to fire the event each of the handles is visited in turn and the target method is invoked via reflection. Invoking the method via reflection is not optimal in terms of performance and limits the possibilities of running the application with reduced permissions .

private WeakEventMgr<SomeEventArgs> someEvent = new WeakEventMgr<SomeEventArgs>();

public event EventHandler<SomeEventArgs> SomeEvent
{
add { someEvent.Add(value); }
remove {someEvent.Remove(value); }
}

// .....

protected virtual void OnSomeEvent(SomeEventArgs someEventArgs)
{
someEvent.Notify(this, someEventArgs);
}

Above is the code necessary to expose a WeakEvent, each event has its own instance of WeakEventMgr<T>. The event subscribe/unscribe method is explicitly implemented to use WeakEventMgr as the backing store for the subscribers. Finally to maintain the Microsoft pattern for event dispatch a virtual protected method exists to dispatch the event. To make adding weak events easier I created a snippet.

Using one of the implementations of Weak Observer means there is now less need to worry about ensuring that all subscribers have unregistered, since event subscriptions will not keep an object alive.

You can download the code here. The code demonstrates the classic problem of form leakage.

Parallel Boids

2008-07-20T17:42:00.001+01:00

The last two weeks Ive found myself immersed in looking at the new Pfx CTP release. I must say it has come on loads since the last drop in December. Some of the issues I spoke about in a previous blog post have now been fixed, things like better algorithms for Parallel.For. Perhaps some of the best addition is the set of concurrent data structures ( cds ).

The main reason for digging into Pfx over the last two weeks was to write an article for the August edition of Developmentor's "development's" monthly news letter, this article attempts to cover the highlights of the recent CTP, so after completing that part of it I had to find some fun use of Pfx, so I ported a developmentor favorite "Boids". Anyone attending a Developmentor Esessntial .NET or Guerilla .NET will have probably encountered this app, but for those not in the know it is a simulation of flocking behaviour.

I took the initial code written by Jason Whittington and started tuning it for performance initially just using one core. The main task was to reduce the number of loops, so instead of having two loops one after each other that iterate over the same set of data I simply combined them, and the second optimisation was to replace foreach with a regular for loop. After that I replaced the main outer for loop with Parallel.For , and viola a reasonable speed up...

I've run the code on my dual core and on an eight way and in both cases observed a reasonable speed up...Not bad considering not a massive amount of effort.

You can download the code from here

Software Architect Week Demos

2008-06-06T10:08:00.001+01:00

Had a lot of fun again at Software Architect Week, the demos for the day of Design patterns can be downloaded from here.

Classic Snake in Silverlight 2

2008-05-18T22:53:00.001+01:00

Inspired by Dave Wheeler's article in VSJ magazine, I decided it was time to write a simple game in Silverlight 2. The game I chose was the classic game of Snake.

Whilst there is no doubt Silverlight 2 is a massive step closer to a .NET environment in the browser, there are still a few bits and pieces that trip you up. Some of the collections data structures are not supported, LinkedList, Queue, Stack...there has been an underlying effort to keep the install of the platform as small as possible, so classes that have functionality that can be easily written using other classes have been tossed out.

As for WPF feature parity things have massively improved since we now have controls, not just simple shapes. There is also now support for data binding and resources. Things still trip you up though, transforms don't support data binding and results in less than obvious errors.

You can play the game here and download the source from here

CodeGen for base decorator

2008-05-03T12:48:00.001+01:00

One of the patterns in developmentor's "Code Smarter with design patterns in .NET course" is the decorator pattern. The decorator pattern allows you to extend functionality at runtime, this works if the client is written in terms of an abstract type. Using the class hierarchy below the client would be written in terms of the Component class. The code wishing to call the client has been given an instance of ConcreteComponent, but it wants to extend the functionality of that object, in order to do this it must create another object that looks like what the client is expecting. This is where the decorator comes in, the component decorator class below is compatible with the client since it also derives from Component.

The componentDecorator class is a null decorator it simply supports the ability to wrap an object of type Component, and when any of its methods are invoked the call is simply passed on to the wrapped object. Since it has no real functionality it is also declared abstract. Each of the types DecoratorOne and DecoratorTwo overrride only the methods they wish to extend, the methods they do not wish to extend are simply handled by the base class ComponentDecorator and forwarded to the wrapped object. Before calling the client a decorator object is created, and is given the object it is to decorate, in this case something of type component. It is now this decorated object that is passed into the client.

The building of the ComponentDecorator class can become very tedious if the Component class has lots of methods that need to be implemented. This week I finally got around to writing some code that via reflection and the CodeDom classes that will generate the base decorator class, saving a reasonable amount of time and effort.

The code can be found

Web Guidelines from the street

2008-04-03T09:44:00.001+01:00

Dom found this link,

Generic Sum

2008-04-02T18:01:00.001+01:00

Its been a while since I blogged about trying to re-implement C++ STL using .NET generics. Last night I was skimming through an excellent book on Linq "Linq in Action" published by Manning, and whilst reading the section on how Expression trees work in C# v3.0 it occured to me that I could potentially use them for creating a generic Sum method.

First a quick look at C# v3.0 expressions, the lambda expression here is not being turned into a method as would be the norm for anonymous methods but it is turned into an AST. Abstract Syntax Tree used to describe the function of the lambda expression.

Expression<Func<int, int, int>> addExpr = (lhs, rhs) => lhs + rhs;

Func<int, int, int> addMethod = addExpr.Compile();

Console.WriteLine(addMethod(5, 5));

In this case that produces a root node to the expression tree that is a Lambda node, that then contains a single Add node, which is a binary operator thus containing two further nodes for the left hand side and right hand side of the add operation.

Reflector spits out the following code

private static void Main(string[] args)
{
    ParameterExpression CS$0$0000;
    ParameterExpression CS$0$0001;
    Console.WriteLine(

      Expression.Lambda<Func<int, int, int>>(

        Expression.Add(

           CS$0$0000 = Expression.Parameter(typeof(int), "lhs"),

           CS$0$0001 = Expression.Parameter(typeof(int), "rhs")),

       new ParameterExpression[] { CS$0$0000, CS$0$0001 }

     ).Compile()(5, 5));
   }

This got me thinking could I build an expression using generic arguments, the code as spit out by reflector certainly looks like its possible. My first crack was as follows

public static T Sum<T>(T[] vals)
{

Expression<Func<T, T, T>> addExpr = (lhs, rhs) => lhs + rhs;

T total = default(T);

Func<T, T, T> addMethod = addExpr.Compile()

          foreach (T val in vals)
         {
             total = addMethod(total, val);
         }

return total;
}

Unfortunately this failed, with the typical "Operator + cannot be applied to operands of type T and T" error that has been the reason why you can't easily write generic Sum in .NET. However not put off I built the expression tree by hand

private static Func<T, T, T> CreateAdd<T>()
        {
            ParameterExpression lhs = Expression.Parameter(typeof(T), "lhs");
            ParameterExpression rhs = Expression.Parameter(typeof(T), "rhs");

Expression<Func<T, T, T>> addExpr = Expression<Func<T, T, T>>.

Lambda<Func<T, T, T>>(
Expression.Add(lhs, rhs),

new ParameterExpression[] { lhs, rhs });

Func<T, T, T> addMethod = addExpr.Compile();

return addMethod;
}

The method above now returns a delegate instance that adds two values both of type T, and returns a value as type T.

Generic Sum can now be rewritten as follows

public static T Sum<T>(T[] vals)
{

T total = default(T);

Func<T, T, T> addMethod = CreateAdd<T>();

     foreach (T val in vals)
     {
           total = addMethod(total, val);
      }

return total;
}

This now works for arrays for ints, doubles etc. It will not work for types not supported by the Add Expression, so only works for simple numeric types, but it is certainly a lot simpler than the approach I took a year or so ago although not as flexible. Its a shame that the C# compiler expression tree builder won't accept generic arguments since it is possible to represent a expression tree this way, as this would result in a lot easier way to generate such code.

Looking for a C# Design Patterns book

2008-04-02T17:26:00.001+01:00

I co-author developmentor's Code smarter with Design Patterns in .NET course with Kevin Jones, and ever since the creation of the course I've been on the look out for a good book to recommend to students. The one I currently recommend is the excellent Head First Design Patterns, although the only rub is its in Java, but in terms of teaching patterns its awesome.

I recently decided to order the latest Design Patterns for C# v3.0 from OReilly written by Judith Bishop, and see if I could recommend that text. Yesterday it arrived, and I skimmed though it looking at the various classic patterns Observer, Template..and my was I shocked...the implementation of some of the patterns is just plane weird, and some of the patterns basic structure is plainly wrong as defined by GOF.

I've not been through the entire catalogue of patterns, but here are some examples.

Template pattern is shown to use composition as opposed to inheritance to bind the template method to the template steps, this is plainly wrong and the implementation is therefore a strategy pattern. As for the strategy pattern examples none of them really effectively show the use of supplying runtime specific behaviour to a subsystem or algorithm.

The observer pattern uses a weird combination of delegates and IObserver interface, its either one or the other, and for .NET is almost certainly pure delegate/event. The is no obvious reason why she has used both interface and delegates/events

The Abstract Factory pattern makes use of a generic interface for the Abstract Factory, but no where in the interface is the generic argument references, and thus is seemingly redundant.

The command pattern mixes the implementation of the invoker with the implementation of the command.

In conclusion I'm not convinced the author either knows the patterns as defined by the GOF, nor is highly proficient in C# v3.0. So my quest continues to find a good C# patterns book I can recommend for class..

True Generic Singleton

2008-02-13T21:44:00.001Z

Last weekend I was teaching Code Smarter with Design Patterns in .NET, yep that's right a weekend...The guys at Intelliflo decided they didn't have time during the regular week for training that wanted me to pack a 4 day course into 3 days over the weekend so we worked from 9 until 19:00...

Whilst teaching them the Singleton pattern they bought out a version they were using built on Generics.

public
class
AlmostSingleton<T> where T:new()

{

private
static T instance;

private
static
object initLock = new
object();

public
static T GetInstance()

{

if (instance == null)

{

CreateInstance();

}

return instance;

}

private
static
void CreateInstance()

{

lock (initLock)

{

if (instance == null)

{

instance = new T();

}

public
class
Highlander : AlmostSingleton<Highlander>

{

public Highlander()

{

Console.WriteLine("There can be only one...");

}

static
void Main(string[] args)

{

Highlander highlander = Highlander.GetInstance();

Highlander highlander2 = Highlander.GetInstance();

Debug.Assert(object.ReferenceEquals(highlander, highlander2));

}

A bit of a google around and it appears a few people are advocating similar solutions. On the surface all looks well, but to be really true to the singleton pattern the type which is the singleton should only ever have a single instance. This implementation requires T to have a public constructor, so that whilst it is certainly possible to always get back to a common instance via the GetInstance method, what is lacking and what the singleton pattern strictly requires is the enforcing of only a single instance of the type. Since the fact that the type has a public constructor means that another client could easily create an instance as opposed to calling GetInstance().

static
void Main(string[] args)

{

Highlander highlander = Highlander.GetInstance();

Highlander highlander2 = Highlander.GetInstance();

Debug.Assert(object.ReferenceEquals(highlander, highlander2));

// This will create a second instance of Highlander..

// not what we want support

Highlander highlander3 = new
Highlander();

}

In fact supporting the true singleton behaviour isn't too hard, it simply requires the use of a bit of reflection to create the object instance rather than rely on the typical language new construct.

public
class
Highlander : Singleton<Highlander>

{

private Highlander()

{

Console.WriteLine("There can be only one...");

}

public
class
Singleton<T>

{

private
static T instance;

private
static
object initLock = new
object();

public
static T GetInstance()

{

if (instance == null)

{

CreateInstance();

}

return instance;

}

private
static
void CreateInstance()

{

lock (initLock)

{

if (instance == null)

{

Type t = typeof(T);

// Ensure there are no public constructors...

ConstructorInfo[] ctors = t.GetConstructors();

if (ctors.Length > 0)

{

throw
new
InvalidOperationException(

String.Format("{0} has at least one accesible ctor making it impossible to enforce singleton behaviour",

t.Name));

}

// Create an instance via the private constructor

instance = (T)Activator.CreateInstance(t, true);

}

Now any attempt to create the instance outside the Highlander class using the new keyword would produce a compiler error, thus enforcing the true singleton pattern. I've also added a guard to make sure that when you use the Generic wrapper it ensures that you do not have a public constructor which would ultimately allow a client to by bass the singleton functionality.

Safe Event snippet

2008-01-31T17:54:00.001Z

This week Im teaching the course I co-wrote with Kevin Jones "Code Smarter with Design Patterns in .NET", as part of the course we introduce the observer pattern and whilst the first part of the deck deals with the basic principles we soon move on to the prefered way of doing it in .NET using delegates/events.

When using events in a multi threaded environment care needs to be taken as whilst register/unregister of the events is thread safe, care needs to be taken when raising the event.

Typically we see code like this

Public class Subject

{

public event EventHandler<EventArgs> Alert;

// ...

protected virtual void OnAlert(EventArgs args)

{

If ( Alert != null )

{

Alert( this , args )

}

Here a check is being made to make sure the delegate chain has at least one instance, however whilst this works in a single threaded environment in a multithreaded environment there is a race condition between the if and the firing of the event.

To get around this problem, one way is to take a local copy of the delegate reference, and compare that. Since register/unregister events build new lists thus making the register/unregister thread safe. However there is a simpler approach and that is to use the null pattern

public event EventHandler<EventArgs> Alert = delegate { };

This will simple create a delegate instance at the head of the delegate chain which is never removed. Thus its then safe to simply do

Alert( this , args )

Without checking for null, since that can't happen. Ok there is an overhead, but it certainly makes the logic simpler and less prone to occasional hard to find race conditions. Remembering to do this could be a pain so I wrote a simple snippet to do it...So now I simply type safeevent TAB TAB and I always get an event with a null handler...another great use of anonymous methods...

Not so much killer app, but killer content

2008-01-25T11:00:00.001Z

It used to be the case that a computer/os would succeed if it had the killer app, the app everyone needed. The spreadsheet was one of the first of that kind, now in the new media streaming age perhaps its killer content that will help to push the adoption of new technology. A student told me last week that Microsoft has done a deal so that MSN is the official home of the 2008 Olympic, the site will provide hours and hours of online coverage, bought to you by Silverlight. Could there be any better vehicle to get Silverlight plugin onto every home/office pc..

The potential danger of too much access to too much data

2008-01-21T16:54:00.001Z

The UK government is set to build a series of large scale databases containing children's data. The charity Action on Rights for Children has put together a three part video detailing some of their concerns to this scheme.

http://www.youtube.com/watch?v=LRQr2VrtX-0

http://www.youtube.com/watch?v=KH-1IumXZbI

http://www.youtube.com/watch?v=IyXCSg-lRkA

Ensure local transaction snippet

2008-01-18T17:27:00.001Z

Whilst teaching developmentors enet 3 class this week I had an idea for a new snippet. It came to me whilst teaching the deck on System.Transactions, I was in the middle of demonstrating to them how you need to be really careful not to cause a transaction to be promoted to the DTC ( See Previous Blog Article ). Two strategies I suggest are

Stop the DTC, net stop MSDTC
Add an Debug.Assert statement inside your TransactionScope block to ensure the current transaction is not a distributed transaction, by checking that the DistributedIdentifier is an empty GUID.

The first option is reasonable if you never need the DTC, but if you do have bits of code that does need the DTC its clearly not an option. The second approach is a bit tedious ut if you use a snippet, then its not all that bad. Below is the snippet code..

<Title>Local Transaction</Title>

<Author>Andrew Clymer</Author>

<Shortcut>lscope</Shortcut>

<Description>Creates a transaction scope and ensures at the end of the block that the transaction is only a local transaction </Description>

<SnippetType>SurroundsWith</SnippetType>

<SnippetType>Expansion</SnippetType>

</SnippetTypes>

</Header>

<ID>scope</ID>

<Default>scope</Default>

</Literal>

</Declarations>

<![CDATA[

using (TransactionScope $scope$ = new TransactionScope())

{

$selected$

$end$

Debug.Assert(Transaction.Current.TransactionInformation.DistributedIdentifier == Guid.Empty,

"Unexpected!! Transaction is now distributed");

}

]]>

</Code>

</Snippet>

</CodeSnippet>

Pfx, Parallel Extensions take advantage of multiple cores, but be careful...

2008-01-13T18:25:00.001Z

Microsoft has now made available a parallel library for .NET, its still in early stages with its aim is to make it easier to write applications to take advantage of multiple cores. Pfx has really two layers one which offers a relatively high level of abstraction, providing a Parallel.For, Parallel.Do and parallel LINQ and a lower level layer allowing programmers to schedule individual Tasks.

This has been a long time coming, .NET support for concurrency is pretty primitive even in .NET 3.0. Simple things like kick of N tasks and wait for them all to complete requires a reasonable amount of code, but on the surface Pfx looks to assist here. I spent a couple of days over xmas looking at it and understanding how it works.

So first task was to find some code I wanted to parallelise I frequently use a piece of logic I stole from the book "How long is a piece of string" that uses many iterations to work out the value of PI as a good async demo, so I thought I would have a go at parallelising this piece of code.

private
const
int N_ITERATIONS = 1000000000;

private
static
double CalcPi()

{

double pi = 1.0;

double multiply = -1;

for (int nIteration = 3; nIteration < N_ITERATIONS; nIteration += 2)

{

pi += multiply * (1.0 / (double)nIteration);

multiply *= -1;

}

return pi * 4.0;

}

Looking at the algorithm it doesn't look that complicated all I need to do is to make each core do a different part of the for statement and then combine the results of each core and multiple by 4.0. In fact Ive done this using the basic BeginInvoke functionality before so I know it scales...

So I first opted for the high level approach offered by Pfx called Parallel.For. Which it states allows me to spread loop iterations across multiple cores. In other words nIteration in my case could be 3 on one core and 5 on another at the same time.

private
static
double CalcPiSimpleParallel()

{

double pi = 1.0;

object sync = new
object();

//for (int nIteration = 3; nIteration < 1000000000; nIteration += 2)

Parallel.For<double>(0, N_ITERATIONS / 2 , 1, () => { return 0; }, (nIteration, localPi) =>

{

double multiply = 1;

// 3 5 7 9 11

if (nIteration % 2 == 0 )

{

multiply = -1;

}

localPi.ThreadLocalState += multiply * (1.0 / (3.0 + (double)nIteration* 2.0 ));

(localPi) => { lock (sync) { pi += localPi; } });

return pi * 4.0;

}

This unfortunately this did not achieve the results I had hoped for...it in fact took twice as long...Bizare...Ok so not suprising since it is using delegates. Using reflector you see that Parallel.For is using the underlying Task class provided by Pfx.

A Task is a piece of work that you give to a task manager and it queue's for an available thread. Now the first thing to mention is that tasks do not run on thread pool threads, a task manager has its own pool of threads which it controls depending on the number of available cores and if the tasks running are currently blocked.

Task task = Task.Create((o) =>

{

Console.WriteLine("Task Running..");

} );

task.Wait();

The code above will create a new task, and it will be run on one of the default Task Managers threads, and the main thread will wait for its completion.

Parallel.For, uses a special variant of Task called a replicating task, which means when you create this task the task manager will effectively not create a single instance of the task but in fact as many instances as there are cores. Each of these task clones are then expected to work in harmony to achieve the overall task. A typical implementation might therefore have an outerloop looking for a new piece of work to do in the overall task, when no more work it simply ends the task.

In the case of a Parallel.For you could expect the outerloop to simply perform an Interlocked.Increment on the loop counter until the required value is reached, and then inside the loop to simply invoke the required piece of work, and here lies the problem in my example in that the piece of work that is executing inside the for loop is extremely trivial and doesn't take that many cycles, and so the overhead of invoking a delegate for each iteration is having a large impact on the overall performance.

So first word of warning when using Pfx, you need to make sure that the piece of work inside a given task is of a reasonable size in order for it to actually scale. Ok so it's not as trivial as it first looks to parallelise the calc Pi method. Not put off I refactored the code to make the algorithm have an inner loop, thus increasing the amount of work for each parallel for iteration. To start with I simple broke it down into ten blocks, thinking each block would now be a reasonable amount of work and we would be only paying the overhead of a function call 10 times, as opposed to N_ITERATIONS/2

private
static
double CalcPiDualLoop()

{

double pi = 1.0;

object sync = new
object();

int stride = N_ITERATIONS / 2 / 10;

//for (int nIteration = 3; nIteration < 1000000000; nIteration += 2)

Parallel.For<double>(0, N_ITERATIONS / 2, stride, () => { return 0; }, (nIteration, localPi) =>

{

double multiply = 1;

// 3 5 7 9 11

if (nIteration % 2 == 0)

{

multiply = -1;

}

double piVal = 0;

for (int val = nIteration; val < nIteration + stride ; val++)

{

piVal += multiply * (1.0 / (3.0 + (double)val * 2.0));

multiply *= -1;

}

localPi.ThreadLocalState += piVal;

(localPi) => { lock (sync) { pi += localPi; } });

return pi * 4.0;

}

But only partial luck, I got a less than 10% speed up...not what I was expecting. So I had another look at the implmentation of Parallel.For to understand why, and digging into it you realise that each of the task created by parallel for, doesn't just take the next value in the loop and work on that, but takes the next eight values. So what this means is that in my case I have an 2 core CPU, the first task to run takes the first 8 iterations of the loop to work on, and the second task gets just 2. The second task completes its work, and since there is no more ends, leaving the first task to complete its remaining 6 all running on one core, leaving my other core cold.

To make sure I wasn't seeing things I made the number of itterations of Parallel a factor of 8 * number of cores, and hey presto it took just over half the time.

I can see that taking chunks of work at a time reduces contention for work allocation, but Im struggling to see why this would be neccessary since this contention will not be that great especically when in order to make Parallel.For to work efficiently each iteration needs to be of a reasonable size. I really hope they address this issue when they finally release the library, since having to know how many cores you are running on and magic number 8 just seems rather odd and goes against what I believe they were trying to achieve.

I also implemented the method using the low level parallel task library and that scales beautifully too..

private
static
double ParallelCalcPi()

{

object sync = new
object();

int nIteration = 3;

double pi = 1;

int stride = N_ITERATIONS / 4;

Task task = Task.Create((o) =>

{

Console.WriteLine("Running..{0}", Thread.CurrentThread.ManagedThreadId);

double localPi =0.0;

double multiply = -1;

long end;

while ((end = Interlocked.Add(ref nIteration, stride)) <= N_ITERATIONS + 3)

{

//Console.WriteLine("Next Chunk {0} from {1} to {2}", Thread.CurrentThread.ManagedThreadId,

// end - stride, end);

for (int val = (int) (end - stride); val < end; val += 2)

{

localPi += multiply * (1.0 / (double)val);

multiply *= -1;

}

lock (sync) { pi += localPi; }

}, TaskCreationOptions.SelfReplicating);

task.Wait();

return pi * 4.0;

}

First impressions is that the Task Library seems ok, but the Parallel.For seems ill thought out. I have also done some experiments with Parallel.Do which allows me to kick of multiple tasks, by supplying a series of Action delegates as parameters. Parallel.Do then blocks waiting for them to complete. Unfortuantly no timeout parameter is present whih means it waits forever, so if an action goes into a rogue state and never completes, your thread that invoked Parallel.Do will hang, not desirable....So my second request is that they add a timeout parameter.

Mobile Bingo

2007-10-22T11:49:00.001+01:00

Just couldn't help myself in writing a bingo caller for my Mobile phone the other week, you see it was my daughter's birthday and she wanted bingo at the party and my wife had decided the low tech approach of pulling numbers out of a bag was the way to go...I wasn't having it "I could write a .NET App for my phone in 10 minutes to do that" I said. Kind of brave since it was my first mobile phone app, I paid £300 for this phone so it just has to do more than just make calls...You know what a bit more than 10 minutes say 30 and I had it working...It just goes to show easy things have become these days....Im thinking of porting wack a mole now...although I'm not sure my touch sensitive screen will last...

You can download the app from here

WPF Extension method

2007-10-10T16:31:00.001+01:00

I used my first extension method yesterday with WPF. I wanted to bring a control to the foreground, that means changing its position in the child list to the last child.

Parent.Children.Remove(element);

Parent.Children. Add(element);

What I would prefer to do is call SendToFront on the collection with the appropriate child. Since I don't have any control over the type being used to hold the collection, I would need to resort to extension methods to get a more object style syntax. Resulting in the following code.

public static class UIElementCollectionExtensions

{

public static void SendToFront(this UIElementCollection collection, UIElement element )

{

collection.Remove(element);

collection.Add(element);

}

All this is well and good but it would have been virtually as elegant with an old fashioned static call on some Util class. It then dawned on me why these are perhaps so useful, for me anyway I rely on intelli sense to see what I can do with a lot of the WPF controls, with any Util class methods I write I need to have knowledge of them, but with extension methods intelli sense can potentially pick them up. So I refactored my code once more into a separate assembly, but placing the code into the System.Windows.Controls namespace, thus whenever I use WPF controls and reference my additional extension assembly I get my new methods. On a large scale project I can see how this could be aid productivity.

However there is something smelly about placing my code into someone else's namespace so on reflection I think it would be far nicer to place all my extension methods inside my own namespace, and just make sure I bring them into scope by using my extension namespace.

Too much reliance on encryption

2007-10-04T10:08:00.001+01:00

A friend of mine was telling me recently how safe manufactures grade their safes, they don't simply say this is unbreakable/uncrackable ( only a fool would say that ). What they do say is that they say you need X amounts of TNT or N hours to crack this safe, clearly limiting their liability you may think, but actually its useful and practical information to anyone who has a safe. Firstly they know that there are no guarantees, but they also know what level of additional security they may need to layer on to get closer to full peace of mind. In the case of the fact that it takes 2 hours to crack the safe the owner could employee a security guard to patrol the safe location every 1 hour, hopefully not creating a window of opportunity for the cracker to do the deed. When you want to secure digital data similar considerations need to be taken into account.

Ive recently been involved with debating the security of biometric systems used in schools with a biometric firms Principal Sales Engineer based in the US. The issue we have as a group is that whilst Im sure all measures today are being take to secure the data in terms of encryption technology, the plain fact remains that history has shown us that what ever cryptography we used today is likely to be compromised in a reasonable time frame say 5-10 years. Therefore when we encrypt any sort of date we need to be aware of this since if the data has uses outside this time window then clearly we cannot rely just on this means of security.

At this goes to the very heart of the debate in biometrics in schools, the Engineer in question dismissed are complaints about encryption technology not being adequate for 10 years plus by first acknowledging this fact ( which is a great step forward ),

"I personally believe their will be another breakthrough in the next 10

to 15 years. Whether is it quantum computing or the DNA processor they

have been working on for the last 10 years. They can now beat a person

in tic tac toe. 5 years ago they could count to 10 with 80 percent

accuracy. They are much faster because they don't have to calculate

they just know the answer. But it is going to be a while and belive it

or not there are higher levels of encryption out there. There are 512

and even 1028 based encryption. Like the computer industry, there is

always someone out there building a stronger based encryption."

Further that statement shows that the industry as a whole knows we need stronger encryption because we know its only a matter of time for it to be broken, but he then goes on to say that

"As I said above the great thing about using encryption on keys and or

files is the fact that if there is a problem with a key or the actual

encryption you can encrypt the info with a better encryption or even

encrypt the encryption such as is done with 3 DES. It is DES encrypting

DES encrypting DES. The US government went from a standard of 3 DES to

AES 256. Not because 3 DES had been broken…. It has not. But because

they saw there were some weekensses that could be exploited and maybe in

the next 10 years or so it may be broken. Now do you think that all the

info that they have stored in 3 DES is still in 3 DES… I think not.

They reencrypted it in the new standard."

Whilst this is all well and good there is a piece to this solution that makes the US government solution ok but not for the average school. In that it relies on the fact that the person who is responsible for the re-encryption has guaranteed sole access to the data, in other words no one has taken an illicit copy, or more likely has some backup media, or an old hard disk. Whilst I can imagine that the US government has plenty of physical security measures in place to make sure they own the only copy of the data, I can't imagine that the average school will have similar system in place, and let's be realistic they can't with theft being the obvious one.

In fact I encountered a similar experience when working for Cisco, we were trying to pitch wireless networking to a large bank. Whilst they accepted the notion that the encryption technology we had chosen prevented illegal access to the network, we could not demonstrate to them that any data sniffed off the network could not be decrypted in a time frame that still made the data useful to an outsider. These guys were smart and did truly understand the nature of securing the business data.

So to summarise a responsible biometric manufacture would secure biometric data as best they can today, but once the software has been deployed if that data is to be truly secure it needs have sufficient physical security measures in place provided by the owner to ensure that in the future the encryption based solution still has adequate merits, the moment you do not have complete ownership of the data all bets are off...and by their own admission the biometric provider in this case said their guarantees is for appx. 10 years, in the case of biometric data for kids that data is sensitive for 60-70 years.

Patterns, Patterns everywhere

2007-09-28T21:55:00.001+01:00

I've not blogged for a while now...mainly due to being snowed under writing the developmentor's "Code Smarter with Design Patterns" course with my co-author Kevin Jones (Course Details ) and working at a local firm ( yes Chippenham has industry). My head is now full of a handful of posts I want to make, and I've finally found time to write this one...

Whilst writing the design pattern course I managed to stumble upon various variations of standard patterns, in the course we discuss various forms of the standard Singleton pattern. Such as ThreadScoped singleton where each thread has access to a single instance, rather than a single instance app domain wide, this is potentially useful for preventing contention where multiple threads are competing over the use of the singleton. But oddly enough I stumbled upon another use of the singleton pattern when developing an example of a CopyOnWrite proxy. There are times when multiple threads request a collection of items from some object. If all the threads are only reading from the collection life is good...If there is a potential for the threads to update the collection for their private use then we should really be returning a copy of the collection to each thread, this could be considered inefficient if the collection is only occasionally privately updated, and that is where the CopyOnWrite Proxy comes in. By returning a proxy to the collection as opposed to the collection itself we allow ourselves to control the access to the collection in such a way as to provide access to the shared view of the collection whilst the thread is only reading but the moment it attempts to update the collection the proxy takes a copy of the collection it is providing the proxy for, and from that moment on the proxy is now wrapping up the copy. The client is blissfully unaware of this.

Ok, so inside my proxy implementation all the methods that are deemed write operations need to determine if a copy has been made previously and if not make a copy. In other words I will have a piece of code that I only wish to run once for the life of the object, and only once. This sounded similar to the fact that in the case of a singleton type I only want one instance. In my case I wanted to define a method in a class that only ever gets run once.

For example I was creating a proxy for a List, thus inside my proxy type I implemented the method

public
void Insert(int index, T item)

{

// Write operation, so make a private copy of the list now

MakeCopy();

subject.Insert(index, item);

}

Where subject is the actual List. Clearly I don't want MakeCopy to run every time the insert method is called just the first time on this proxy. Simple you might say just test a Boolean flag to see if you have made a copy. Whilst this would be guaranteed to work in a non threaded environment it would not in a threaded environment, so now you end up writing code that performs some kind of synchronization prior to determining if you have made a copy or not.

In this scenario I could simply create a method called MakeCopyIfNeccessary() and call that from each write method, and make that method do the appropriate synchronization, to do this efficiently requires double check locking. However this got me thinking if there was a more re-usable way of doing this so that if I needed this functionality again I could somehow re-use not just the pattern but the code. Below is a type I defined which wraps up an Action delegate such that it will only invoke the delegate once, irrespective of the number of times you call DoOnlyOnce();

public
class
SingletonAction<T>

{

object onlyOnceLock = new
object();

private
Action<T> action;

public SingletonAction(Action<T> action)

{

this.action = action;

}

public
void DoOnlyOnce(T arg)

{

if (action != null)

{

SynchronizedDoOnlyOnce(arg);

}

private
void SynchronizedDoOnlyOnce(T arg)

{

lock (onlyOnceLock)

{

if (action != null)

{

action(arg);

action = null;

}

Inside my proxy class I create an instance of this type supplying it the corresponding method I wish to only be executed once.

class
CopyOnUpdateList<T> : IList<T>

{

private
IList<T> subject;

private
SingletonAction<object> makeCopy;

public CopyOnUpdateList(IList<T> list)

{

makeCopy = new
SingletonAction<object>(MakeCopy);

subject = list;

}

// ....

public
void Insert(int index, T item)

{

makeCopy.DoOnlyOnce(null);

subject.Insert(index, item);

}

However if it wrapped up as an anonymous method this would be more true to the values of the singleton pattern as that way the only way the code could be invoked was via my wrapper. Wrapping up an arbitrary method doesn't stop it from being invoked elsewhere.

public CopyOnUpdateList(IList<T> list)

{

makeCopy = new
SingletonAction<object>(delegate

{

// Code required to perform the subject copy

});

subject = list;

}

Now the "only" way that code can be run is via my wrapper class which ensures it only ever gets run once...

Obtaining a balanced view

2007-05-17T08:45:00.001+01:00

I was asked yesterday to visit Gloucestershire county council to discuss the use of biometrics in schools. The council have set up a committee with the responsibility of forming a policy around the use of biometrics in the county; I must congratulate Gloucestershire here as they are the first government body that I'm aware of that are actually attempting to tackle this issue fully. Currently schools are allowed to spend their money as they wish, which in one sense is great but on the other side there is always the worry that can a head of a school actually devote sufficient time to do full due diligence on a piece of technology. Certainly in the case of biometrics a lot of the issues are not obvious and present new challenges to the school such as correctly disposing of the data in a safe and secure manor, understanding the potential long term consequences of how the data may be used and abused in the future. These issues require you to be a technologist at minimum but really require someone to be a technologist visionary.

Investing time and effort in understanding the implications of biometrics does not need to be done by every school, it needs to be done centrally were money can be spent employing the experts from both sides of the argument to present a balanced and informed view, that will further enable the decision makers to make an efficient and informed decision. This information needs to be made available in a prominent place, and not buried on some government web site.

I also think it's important here to include parents in the decision making process, providing them with the information in a clear and concise form. There is precedence here, it's pretty well accepted that smoking is bad for you, people can still smoke but the government takes the responsible act of making it very clear that this could affect your long term life expectancy. Perhaps a similar mandatory warning should be made on all acceptance forms for biometric systems, just above the signature line.

"WARNING: Some experts feel that in the future it will be possible for this information to be used to steal your child's identity"

All I'm asking for is a fair representation of the facts, surely we owe it to all decision makers to make both arguments equally available and allow them to make the decision based on the full range of facts.

Minimum Debugging tool set

2007-05-14T23:38:00.001+01:00

This week I've been teaching DM's Effective .NET course, one theme that run through the course is the ability to debug applications in the field. The tools in your arsenal here are the obvious ones like perfmon, but also the native windows debugging. The native debugger can be used to take snap shots of the suspect process (dump's seems to get a giggle from a few students). You therefore need to put the debugger on the client's machine, the typical way to do this is to download the debugging tools for windows, this is a reasonable size once unzipped 37MB...This includes a graphical debugging a few other bits and bobs, whilst these tools are useful on a developers machine they are not needed to take a snapshot and therefore not relevant for anyone other than a developer. This prompted a question from a student as to what is the minimum set of files needed to take a dump from a users machine.

After a few trial and errors we came up with the following list of files

tlist.exe

dbgeng.dll

dbghelp.dll

adplus.vbs

cdb.exe

This results in a 4.6 MB unzipped install for a client's machine; this is more of a reasonable size that could be included in all employees desktop configuration.

Free Running Threads

2007-03-21T18:25:00.001Z

Last week I was teaching Developmentor's Effective .NET 2 course, as part of that course we spend at least a day looking at various patterns for building multi threaded applications. With the rise of multi core machines it is becoming increasingly more important to write algorithms that can scale with the availability of new cores. One of the topics we cover is thread safety, and I typically write an application that simply performs an i++ operation on multiple threads, on a single core does not typically generate a problem but on a multi core means we rarely get the value we expected since i++ is in fact multiple CPU instructions, e.g

MOV R0,i

INC R0

MOV i,R0

One thread could perform the load of i into a register and at the same time another is also doing this thus they both cause i to be simply incremented by 1 and not by 2. To solve this problem we have a variety of synchronization techniques at our disposal some are lighter weight than others

Interlocked.Increment , Monitor.Enter/Exit and OS synchronization primitives like Mutex

In my stress test I conducted in class I had multiple threads updating a single integer...First I demonstrated on a multi core machine that i indeed didn't have the correct value after 10 threads had attempted to increment it 10000 times. I then moved to using Interlocked.Increment and now everything was as expected for the result, but it was slower than a simple i++. All well and good so far...I then moved to using Monitor.Enter and Monitor.Exit and to my amassment that took pretty much the same time as Interlocked...so as all engineers do...we run it again cause that result was just a glitch....but after numerous runs it kept coming out the same....So when I first developed this demo I did so on a single core machine and this was its first outing on a dual core, so what went wrong...all my multi threading life I've been told that interlocked is far cheaper than the heavier weight mutex style of synchronization. I then re-ran the demo with CPU Affinity set to a single CPU, and got the results I would have expected with Interlocked being at least an order of magnitude quicker than mutex

During the lab break I stepped back and had a think, about what could have caused this, the light eventually went on in that all the threads were in fact sharing a common variable, with dual core each core is going to be attempting to cache this value, this has a very negative effect on the cache, why well the cores try and maintain cache coherency by marking parts of the cache dirty when they update them, and forcing the other core to reload its data from main memory...If this wasn't the case in our example we would have had the wrong numbers again...So ok possibly what you would have expected....however the process of marking a piece of memory as dirty is not as simple as marking a single word, its less granular than that the core will mark whats called a cache line as dirty meaning any data on the same cache line as the value being updated is effectively marked as invalid.

So imagine two integers next two each other in memory. Thread A increments integer one and Thread B increments integer two, from a high level programming language perspective this is perfect, each thread has its own private resource and there is no need to synchronise and the threads therefore run freely...The perfect threaded app....however not so fast if both integers occupy the same cache line...we take a performance hit...

To demonstrate this fact I wrote some code that simply implements a parallel i++. There are three scenario's

Interlocked.Increment( ref i )
Interlocked.Increment( ref ManyCounters[Thread.CurrentThread.ManagedThreadId ] )
Interlocked.Increment( ref ManyCounters[Thread.CurrentThread.ManagedThreadId * 10000 ] )

Whilst the last two variants will not produce the correct total, the middle version shows that although the threads are not sharing the same variable the fact that the fact that they probably all live on the same cache line is the crucial factor. Below is the results of running the code using 1 and 2 threads. In the first case, with the high level of contention we see that a single thread would have been more efficient.

Single shared counter with 1 Threads took 00:00:00.2065516

Single shared counter with 2 Threads took 00:00:00.4303586

In the second case with multiple counters we are still taking a hit on performance even though the counters are different ints, and thus we are not incrementing the same location

Multiple counters with 1 Threads took 00:00:00.2184554

Multiple counters with 2 Threads took 00:00:00.5177658

Only in this last case were we ensure the counters are not on the same cache line do we see both cores being used efficiently.

Multiple Sparse Counters ( ~4k apart ) with 1 Threads took 00:00:00.2180101

Multiple Sparse Counters ( ~4k apart ) with 2 Threads took 00:00:00.1603323

So what does all this mean, well it certainly shows that just because your code at a high level programming language has no contention it doesn't mean that it will not have contention when the code finally meets the hardware. This is obviously a big issue and more so with virtualisation in .NET, how do I know the size of the cache lines, or how my data will be laid out in memory...Computing just got fun again.....Code

Reduce your risk; only store what you really need

2007-02-08T11:18:00.000Z

I've been drawn further into the campaign to prevent biometric information being used by schools, on Tuesday I attended a briefing session for MP's with the aim to highlight the issues with adopting this technology in the context of school children. The BBC were present and did an article for BBC online..

http://news.bbc.co.uk/1/hi/uk_politics/6336799.stm

What we as campaigners are increasingly finding hard to understand is how the department for education fails to understand the difference between data that has a relatively short validity and immutable data that last a life time ( I can't change my finger print)

For me one of the best quotes from Dfes with regard to schools holding biometric data is:-

"They are well used to handling all kinds of sensitive information to comply with data protection and confidentiality laws.

Schools have historically failed here; a forensic computer science faculty bought hard drives off ebay, and extracted school records (http://www.theregister.co.uk/2005/02/17/hard_drive_data/ ). A colleague also told me recently how he took a school computer out of a skip...Personally I don't condemn the schools here after all their primary focus is on education and not on securing personal information. In fact this is also the case in business the IT system the security are all additional burdens which do not enhance the core functionality of the business, it is seen as a necessary evil, and you often find these are given second or third rate priorities.

Information security can never be guaranteed and so we should therefore only gather the least amount of information required to perform our function. Software engineers have been aware of running applications with least privilege, thus limiting the risk that there application exposes to a system if it was to be hacked; even Microsoft is adopting this strategy at last with Vista. This therefore poses the question do schools need biometric information in order to educate our children? If the answer is no then it should not be used in schools...since this creates a further burden on a system which already showing signs of failing under the current security workload.

Custom ToString() for Flag based Enums, and a splatter of Unit Testing

2007-02-02T16:53:00.000Z

A project that I'm currently working on has a enum type, where the enum is defined like so

[Flags]

public
enum EventDayMask

{

NONE = 0,

SUNDAY = 1,

MONDAY = 2,

TUESDAY = 4,

WEDNESDAY = 8,

THURSDAY = 16,

FRIDAY = 32,

SATURDAY = 64

}

Calling ToString on a value of type EventDayMask would result in a comma separated list of the various set bits..

EventDayMask weekend = EventDayMask.SATURDAY | EventDayMask.SUNDAY;

Console.WriteLine( weekend.ToString() );

Would produce SATURDAY,SUNDAY..Whilst this is a massive improvement on what we had with C and C++ I in fact would like a more appealing string like Saturday, Sunday.

You can't override ToString for Enum's which means you can't write your custom string generate as part of the enum class. There are many blog posts on how you can achieve this for a non Flagged based enums, by applying custom attributes to your enum definition and then having a static method on a Utils class that first determines the value of the enum and then determines if there is a custom attribute associated with that value and if so uses the string associated with that custom attribute.

E.g

public
enum EventDayMask

{

NONE = 0,

[Description("Sunday")]

SUNDAY = 1,

[Description("Monday")] // etc..

MONDAY = 2,

TUESDAY = 4,

WEDNESDAY = 8,

THURSDAY = 16,

FRIDAY = 32,

SATURDAY = 64

}

However with flag based enum's it gets a bit more complex, since an enum does not have a single value but a combination of many valid values, as in the DayMask example above. To extend the technique to Flag'd based enums you need to effectively test the enum value against each possible flag value. I managed to write a version that did this by stepping through each possible value and performing a logical "AND" against the underlying enum value, and that worked fine, until I added an or'd value to my enum definition.

public enum EventDayMask

{

...

[Description("Weekend")]

WEEKEND = SATURDAY | SUNDAY

}

The built in ToString() works as expected if the underlying value is SATURDAY | SUNDAY it outputs WEEKEND. Obviously I wanted the same behaviour, it was at this point that I realised this small task was now going to escalate, a quick cup of tea latter and I decided to change my approach why not simply use the built in ToString() to generate the initial string and then replace each of its component parts with the value stored in the attribute. This simplified the code greatly..the downside being that I'm tightly coupled with the output format of Enum.ToString(). To counteract this I have a handful of unit tests as part of the project that tests this functionality, so if MS ever changed the formatting algorithm for Enum.ToString(), I'm alerted immediately..This for me is another example why unit testing is so powerful, I can make expedient decisions that I wouldn't have dared made without them....

[AttributeUsage(AttributeTargets.Field,AllowMultiple=false)]

public
class
EnumValueDescriptionAttribute : Attribute

{

public EnumValueDescriptionAttribute(string description)

{

Description = description;

}

public
string Description;

}

public
static
class
EnumUtils

{

private
const
char ENUM_FLAGGED_VALUE_SEPERATOR_CHARACTER = ',';

public
static
string EnumToString(Enum enumValue)

{

StringBuilder enumValueAsString = new
StringBuilder();

Type enumType = enumValue.GetType();

string[] enumToStringParts = enumValue.ToString().Split(ENUM_FLAGGED_VALUE_SEPERATOR_CHARACTER);

foreach (string enumValueStringPart in enumToStringParts)

{

FieldInfo enumValueField = enumType.GetField(enumValueStringPart.Trim());

EnumValueDescriptionAttribute[] enumDesc = enumValueField.GetCustomAttributes(typeof(EnumValueDescriptionAttribute), false) as
EnumValueDescriptionAttribute[];

if (enumValueAsString.Length > 0)

{

enumValueAsString.Append(ENUM_FLAGGED_VALUE_SEPERATOR_CHARACTER);

}

if (enumDesc.Length == 1)

{

enumValueAsString.Append(enumDesc[0].Description);

}

else

{

enumValueAsString.Append(enumValueStringPart);

}

return enumValueAsString.ToString();

}

TransactionScope and DataAdapters

2007-01-12T11:11:00.000Z

Recently I had to write some code that persisted changes too two DataTable's to a SQL Server database, with both updates inside a single transaction. So since I was using .NET 2.0 I decided to use TransactionScope.

using( TransactionScope tx = new TransactionScope()))

{

firstTableAdapter.Update( firstTable );

secondTableAdapter.Update( secondTable );

tx.Complete();

}

I was careful to ensure that both adapters used the same database connection object, however there was still a nasty side effect. First a bit of background

In order for a resource manager (in this case SQL Server) to take part inside a transaction it must first enlist in the transaction, SQL server determines if to do this when a connection is opened. Opening a connection when there is a transaction associated with the current thread will result in SQL server placing that connection inside a transaction, if no other enlistment has happened then SQL server will create a local transaction and thus manage the transaction.

If another resource manager wishes to enlist in the transaction then no one resource manager can be responsible for coordinating the transaction and the DTC is invoked (Distributed Transaction Coordinator), the role of the DTC is to ensure the properties of the transaction across multiple resources. Such that a failure to commit one set of resources through one resource manager causes a rollback in the other. SQL Server is the only resource manager that currently allows the promotion of a local transaction to a distributed transaction; other resource manages like MSMQ will always create a distributed transaction irrespective of the number of previous enlistments.

Ok, so to sum up if you enlist multiple resource managers inside a single transaction the DTC is invoked and the transaction becomes a distributed transaction. The obvious consequence of this is that the transaction management is now more expensive. So we obviously want to avoid having a distributed transaction unless we really need to.

When first playing with TransactionScope we can end up with a distributed transaction when from a high level logical perspective it might seem strange.

using( TransactionScope tx = new TransactionScope() )

{

conn.Open();

// Do DB Work

conn.Close();

conn.Open();

// Do DB Work

conn.Close();

tx.Complete();

}

In the above example our transaction is promoted to a distributed transaction, why because on the second open SQL Server will attempt to enlist in the transaction, because there is already a transaction enlisted System.Transaction will attempt to promote the existing transaction to be managed by the DTC, and the new transaction will also be managed by the DTC. This does seem odd at first since you are in fact only using a single Resource Manager, there is a rumour that this will be fixed in the future. If you do not have the DTC service running on your machine the code above will throw an exception. If you have the DTC running then the code runs to completion, and you are non the wiser of the promotion, except for a slight pause. To see when the promotion has taken place simply make a call to the function below at the various points to see the transaction identifier.

private
static
void PrintTransaction(Transaction transaction)

{

Console.WriteLine("Local Id = {0}" , transaction.TransactionInformation.LocalIdentifier );

Console.WriteLine("Global Id={0}" , transaction.TransactionInformation.DistributedIdentifier);

}

Back to the data adapter example, when the data adapter attempts to do an update if the connection is not currently open it opens the connection and performs the update. If it opens the connection it would be polite for it close the connection when it is done. So when the first Update runs on opening the connection it creates a local transaction which gets enlisted, when the second update runs it also needs to open the connection and so it enlists a second transaction and thus causes the behaviour observed above. To fix the problem we simply have to ensure that the connection is only opened once.

using( TransactionScope tx = new TransactionScope()))

{

conn.Open();

try

{

firstTableAdapter.Update( firstTable );

secondTableAdapter.Update( secondTable );

tx.Complete();

}

finally

{

conn.Close();

}

Now when the calls are made to update the data adapter sees that the connection is already open and thus the sql commands now run in the context of the existing transaction and no promotion is necessary. It is therefore essential that when you create the data adapters you create them with the same connection object you are using inside the transaction scope.

All this poses the question how can I ensure that I don't accidently cause a promotion to take place; there are two solutions that come to mind

Disable the DTC on your machine
Add a Debug.Assert prior to calling the outer Complete

The only way I could find to determine if a promotion had happened was to look at the Distributed Transaction Identifier.

Debug.Assert(IsInDistributedTransaction() == false);

tx.Complete();

...

private
static
bool IsInDistributedTransaction()

{

return ((Transaction.Current != null) &&

(Transaction.Current.TransactionInformation.DistributedIdentifier != Guid.Empty));}

So whilst TransactionScope has certainly improved the programming model for transactions by hiding a lot of the complexities required for distributed transactions it is essential to understand how it works if you are to avoid accidently creating a distributed transaction.

WPF Pong

2007-01-05T14:46:00.000Z

Couldn't keep still over the xmas break, my kids were playing some of those classic games we used to play as kids although now inside a tiny console you plug in to your tv. This got me thinking about writing a classic game using WPF, not being much of a games writer I kept things simple and decided to implement the classic game Pong...My main aim is to continue to ramp up on WPF, one area I wanted to explore further was the use of Content Templates and styles, what I wanted to be able to do was not only clearly separate UI layout from UI behaviour but also allow different Skins to be applied to the game one being the classic black and white look another being a more 21^st Centaury look and feel.

To implement Skining in WPF you make use of Styles contained in resource dictionaries, when a control makes reference to a named style the control tree is walked for each control its resource dictionary is searched for a style by that name if none is found the search continues up the tree. Styles allow you to set properties of the control, very similar to ASP.NET skins. So in the case of the bats you can set a style that sets the colour, width and length of the bats. You can then apply this style to both player one's and player two's bat keeping both bats looking the same. In order to allow the interchanging of Skins I placed all the resource definitions into their own resource dictionaries. Each Skin is then an instantiation of a resource dictionary, when the window is launched I simple associate my window resource dictionary with the currently selected skin. When any of the controls wish reference styles located in resources they walk up the control tree looking for the resource and should find it in the resource dictionary associated with the window. If the user wishes to change the current skin I simply change the resource dictionary associated with the window and the look feel changes to reflect the new skin.

this.Resources = newSkin;// this is the current window

Below is an extract from GameWindow.xaml, the style of the stack panel is defined through a resource named StatusPanel, notice that DynamicResource is used as opposed to StaticResource. If you want to have any changes in resources immediately reflected then you need to bind using a DynamicResource, these are slightly more expensive since the infrastructure needs to keep track of any changes.

One really cool feature of WPF is the ability to change the inner content of a control via the template property. I used this to great effect when defining for displaying the scores using block character style numbers. I used a data trigger to select the appropriate inner content template based on the value of the score.

When I have some more free time I want to add WPF animation effects so that the bat shudders when the ball hits it...Oh and my QA engineer ( AKA my son ) wants to enhance the game so you can move the bats forwards and backwards too...( "Just like real tennis." )

Oh I also perhaps need to have some instructions in regard to keys etc...but for now

A and Z player one

K and M player two

And Space to serve..

First to 9 wins...

When I first started hearing about WPF and how designers would now rule over UI I I started to get the same feelings I had when I first saw VB 6, but what seems to be coming clear whilst the designers can really do cool stuff they really need to have good exposure to the underlying model, so there certainly is still a significant role for us UI developers in exposing the appropriate information all be it perhaps slightly removed from the coal face...

You can download the full source from WPFPong , or for any Click Once Jen fans you can install it via ClickOnce

SynchronizationContext assists layering

2006-10-21T22:22:00.000+01:00

Having just got back from two weeks travelling I was confronted with the need that my oldest daughter needs to learn her times tables for a tester after half term. Being away a lot and the children having loads of after school clubs etc means we often find it hard to find the right moment to practice. So I decided to write an app to help test them on it…Ok so I’m sure there are tons of them out there…but you can’t beat a home grown solution ;-)

Seriously there is a couple of areas I am focusing on at the moment,

Ramping up for DM’s .NET architecture course.
WPF.

This seemed like a good exercise where I could put both these skills to good use. Now even DM instructors have time pressures this app needed to be written like yesterday if the kids are to start using it to get ready for their test. My WPF skills aren’t there yet that I could confidently build the UI that quick, but in a few weeks I do want to add a WPF gui. So this creates a great architectural opportunity for me to structure the solution into clear layers…Model, View and Controller. The View and Controller layer is specific to WinForms and I obviously want to write as little code here as possible, putting most of my focus on the Model, in this case a Times Table Test engine.

First I built the model defining a type to represent the test, and exposing properties for three possible answers and a running score. The type also had methods for requesting the next question and submitting an answer. In order to make the view be updated by changes in the model I had to implement the INotifyPropertyChanged interface on each model type in order for them to notify the UI of any changes to the model.

Then I moved onto building a form to display the current question. Labels were used to pose the question, and to display the score etc. The possible answers were represented by buttons. Having laid out the form it was then just a question of wiring up, the various labels and buttons to display the correct values, this was achieved through the use databinding via bindingSources, all the wiring up done inside the designer. I then had to wire up the actions for the various buttons to communicate the users response to the question and to move the question forward. But So far so good…virtually no UI code written.

However whilst this was ok to make the app really useful the user needs to enter the response in a finite period of time. So this created two options implement a timer in the UI layer or in the model…For me I obviously wanted to place it into the model since that piece of code would be reused when I implement the WPF UI.

So off I went and added a Timer thread to my model, the activity on this timer thread would cause various bound properties in the model to be updated, which then had the effect of firing PropertyChanged events to all objects bound to these properties. Since some of these bindings where to UI components this would result in the UI component being updated on a thread other than the thread used to create the UI component. Attempting to update a UI component on a foreign thread is technically not allowed, and in .NET 2.0 if you have a debugger attached and you attempt to do so you get a cross thread exception…to my surprise I didn’t get one…just the update’s failed to be propagated to the UI. This I thought was a bit odd…but since the update didn’t happen I guess it’s by design Im not 100% sure of the exact reasoning behind this..Anyway for me I need to fix this situation since I did want my UI to be updated.

In order to solve this problem I would need to cause the event to actually be delivered on the UI thread. The conventional way to do this in WinForms is to use a pair of methods on every control called InvokeRequired and BeginInvoke. But this would mean that my model code would need knowledge of winforms to ensure that all events fired on the UI thread Yuk and breaks my architectural model.

There is however a more general way of doing this using something called SynchronizationContext, this is an abstract class that provides a way of marshalling a delegate onto a specific thread. You can ask WinForms for the current SynchronizationContext and it will return an object that can be used to post work back onto the UI thread. You write your code against the abstract class, and assuming other UI implementations that require thread affinity also support this then my mode can be written just the once. In fact WPF does have support for this so cool…the model can be written to deliver PropertyChanged events on the correct thread. To ensure my property events are marshalled onto the appropriate thread I have a method which looks as follows.

private
void SafePropertyChangeNotification(string property)

{

if (syncCtx != null)

{

syncCtx.Post(delegate

{

OnPropertyChanged(new
PropertyChangedEventArgs(property));

} , null);

}

else

{

OnPropertyChanged(new
PropertyChangedEventArgs(property));

}

What I’m struggling to understand is why they MS never added support in the BindingSource type to perform the marshalling operation, they are after all detecting the attempt to perform a cross thread operation and silently ignoring it, giving you the option to configure the binding source to marshal the update would have been dead cool.

Anyway the SynchronizationContext does allow me to decouple my model from my UI implementation, and does allow me to write a single piece of marshalling code that will work with WPF or WinForms..

The first version of the times table tester is now complete, the UI layer is extremely thin…and I even managed to deploy it in our home network using ClickOnce…I know scary I actually taught the kids how to just say ok to the not trusted publisher prompt….

Over the next few weeks I’m going to build a WPF version of the UI and assuming no hidden gotcha’s Im just simply going to replace the UI layer with a new one

You can download the full code from my download page

Korean’s get excited over VSTS

2006-10-10T14:16:00.000+01:00

Whilst teaching the other week Jason pointed me to this cool VSTS video where Korean developers had to make their own jingle to spread the news….

Check out the Video ignore all the requests to install the language packs unless you want Korean characters on your machine..

BioGuard fingerprinting booted out

2006-10-10T14:04:00.000+01:00

If you have read my earlier posts you may have read my rants on how my kids primary school was attempting to implement fingerprinting for school registration. Finally after a whole year and some money they eventually got it that the idea was just plain dumb….how they ever expected to integrate such a system such that they had a reliable and fault tolerant registration system at a reasonable price was beyond me and any else who had any reasonable amount of IT experience.

So whilst I’m pleased I’m just hoping that no other school gets approached by Bio Guard or its resellers in the UK. It also made me chuckle today as their web site hinted that finger prints are globally unique something we now know to be false.

More than just coffee

2006-10-10T13:08:00.000+01:00

The whole identity theft took a new twist today whilst at Starbucks I noticed that the electronic credit card machine had a sticker on it and in big letters were the Network Address of the device and other network details…..Wow some hacker would just luv to lap those numbers up along with their espresso….The nice girl behind the counter did remove it…

Slicker Labels

2006-09-05T18:27:00.000+01:00

Whilst thinking up some more interesting WinForms demo's I just thought Id have a go and at making a the bog standard label control a bit more interesting...so I knocked up a new version that fades text in and out when the label's text property is modified.

The Source code can be downloaded from my downloads page

WHAK a Mole WPF

2006-07-24T21:41:00.000+01:00

Just started to play with WPF, and just had to port Jason’s W infamous WHAK a mole game to WPF. Hasn’t exactly pushed the boundaries of my GPU, but more about starting getting to grips with XAML, WPF event model and inner content.

Download Source

Every OS Sucks

2006-07-12T19:36:00.000+01:00

Currently teaching a DM Guerrilla, and we have been playing some short funny geeky type movies, a student offered this one to be considered….

Every OS Sucks…

Visual Studio and ASP.NET Master Pages

2006-07-07T16:58:00.000+01:00

I finally got tierd of the bug in the Visual studio designer for ASP.NET master pages. The issue is this, if you don’t hardcode the master page for the page in the page itself the designer will not see that you are using master pages, even if you define one in the pages section of web.config. This greatly defeats one of the objectives of master pages in that I would like to tweak the master page at deployment time, and the default behavior is page value overrides config file.

Possible solutions are have a default.master page, and make all pages point to that. Then have the actual master pages as separate files and simply copy over the default.master with the one you want at deployment time.

That seems clunky to me, so another option would be to make it use the value in web.config as opposed to the one defined on the page. You can do this by creating a new class that sits between your form and the System.Web.UI.Page class. This class will simply inspect the value held in web.config and if its set override the page setting.

using System;

using System.Data;
using System.Configuration;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;
using System.Configuration;
using System.Web.Configuration;

class SiteMaster : Page
{
protected override void OnPreInit(EventArgs e)
{
PagesSection pagesSection = (PagesSection) ConfigurationSettings.GetConfig("system.web/pages");

if (pagesSection.MasterPageFile != String.Empty)
{
this.MasterPageFile = pagesSection.MasterPageFile;
}

base.OnPreInit(e);

}
}

}

Simply modify your code behind file to look like

public partial class DefaultForm : SiteMaster

And you will now get the master page as defined in the web.config. For pages you want to set by override the site default continue to derive directly from System.Web.UI.Page.

Finger printing in schools just gets hotter…

2006-07-07T08:49:00.000+01:00

A while back I mentioned that the primary school my children are attending was planning to introduce fingerprinting for school registration. I believe I’ve managed to get that stopped now, however this is not the case for many parents who are currently discovering that without their knowledge or consent their child’s biometrics have been taken by the school and stored for such trivial uses as borrowing a library book.

The companies involved in this technology keep stating there is no problem and parents have never objected…So thankfully a new web site has been launched which is now clearly stating for the record that this is clearly not the case.

Please check the web site out, and make your voice heard….Leave Them Kids Alone

Custom Controls and Cross Threading

2006-07-06T11:19:00.000+01:00

In .NET 2.0 a feature was introduced so as to raise an exception if an application attempted to access a Windows Control on a thread other than the thread in which the control was initially rendered. This is because Windows Controls are deemed not thread safe, and only expect to interact with a single thread, and thus have no state lock protection. In 1.1 you could blissfully ignore this and most of the time your code would work..

The feature in 2.0 is disabled by default, but when you run under the debugger it is enabled. The debugging environment sets the static member on the Control class CheckForIllegalCrossThreadCalls to true.

Whenever you attempt to get hold of the underlying Windows handle for the control this static member is first inspected and if it evaluates to true the InvokeRequired method is invoked to see if you are indeed on the correct thread. If you are not on the correct thread the exception is thrown

The code below was generated by Reflector, taken from the base Control class

public IntPtr get_Handle()

{

if ((Control.checkForIllegalCrossThreadCalls && !Control.inCrossThreadSafeCall) && this.InvokeRequired)

{

throw new InvalidOperationException(SR.GetString("IllegalCrossThreadCall",

new object[] { this.Name }));

}

if (!this.IsHandleCreated)

{

this.CreateHandle();

}

return this.HandleInternal;

}

There is a slight extra piece of complication in that it is possible to have some control operations that are thread safe, these calls use a class called Control.MultithreadSafeCalllScope which implements IDisposable as a way to wrapup blocks of code that are in fact thread safe and still need to have access to the underlying windows handle.

So using Reflectors Analyze feature I decided to see where this class was used in the framework so that you can see what Windows control methods are Thread safe…

The results were as follows

System.Windows.Forms.Control.BeginInvoke(Delegate, Object[]) : IAsyncResult

System.Windows.Forms.Control.CreateGraphics() : Graphics

System.Windows.Forms.Control.EndInvoke(IAsyncResult) : Object

System.Windows.Forms.Control.get_InvokeRequired() : Boolean

System.Windows.Forms.Control.get_WindowText() : String

System.Windows.Forms.Control.Invalidate(Boolean) : Void

System.Windows.Forms.Control.Invalidate(Rectangle, Boolean) : Void

System.Windows.Forms.Control.Invalidate(Region, Boolean) : Void

System.Windows.Forms.Control.Invoke(Delegate, Object[]) : Object

A few posts back I saw that Invalidate was possible, but now I can see that I can also get the text associated with a windows control.

Anyway back to the plot….the reason for all this was to make my own custom controls behave the same as the underlying windows controls. I basically wanted to ensure that my control was only ever interacted with on a single thread. Since with UI apps you generally expect everything to happen on the main UI thread you don’t want to put the overhead of locking etc in as the majority of cases everything is happening on the UI thread.

So I’ve added a new method to my control called CheckForIllegalCrossCall I make a call to this method inside each of my properties..

private void CheckForIllegalCrossCall()

{

if ((Control.CheckForIllegalCrossThreadCalls) && (InvokeRequired))

{

throw new InvalidOperationException("Illegal cross Thread operation");

}

Now I don’t need to worry about state corruption as state can now only be manipulated on a single thread. When debugging we will make the call to InvokeRequired and when not we will simply assume that the programmer knows what he’s doing…

Last two weeks

2006-07-04T10:07:00.000+01:00

Last two weeks been very manic….went up to Edinburgh to deliver a course at the Edinburgh training centre. Didn’t get to see much of the city but will certainly go back for a weekend visit, what I did see was pretty spectacular. What was great was the fact that although you were in the city you could still get to see some pretty amazing “Vista’s”… ( wow that word is now so much in my vocabulary now ).

Second week doing a gig for MS Academic Insight were I spoke to nearly 50 academics on the wonders and joys of .NET. Had a lot of fun and the hotel was fantastic. More often and not Im in London hotels and going to a rural hotel was a breath of fresh air, the staff spoke English ok in this case with a Brummie dialect but that’s ok Im a Brummie too. There were extremely helpful, and just little touches like Breakfast cooked to order goes a long way…A fellow speaker felt the need to praise the hotel on his blog too ( http://www.robmiles.com/ ), mainly for the in room tea tray. The hotel was the Stourport Manor Hotel

Rob was there to deliver a talk on Windows Mobile, and I must say things have come on along way since my original IPaq and NSBasic. I can now develop using full loaded Visual Studio and .NET code…Rob was very passionate about Mobile, mainly due to richness in applications that you can now develop. Since in the case of a smart phone you have access to camera, blue tooth, phone GPS, and GPRS/3G ( Internet ). Such a vast array of devices compact and mobile…

He demo’d a neat space invaders game were you can take a picture of your co-worker and then use that picture as the evil alien and blaster them to smithereens… The funniest app was the SMS Roulette were game would select a random name from your contacts and then a random greeting be it nice or bad and send it….A great pub game as Rob put it…

So this was indeed neat you had full access to all the resources on the phone…After the initial laughter of this app, it got me and at least one other delegate thinking.. At no point did we get a warning from the app saying it was about to use the SMS service it just did…So the application spent my money without actually asking me…What else could it do…could it farm my contact list and email it to evilspammers.com..Who knows…

When I asked the question about security, there is a step you need to go through to get your app onto a phone whereby you need to get it signed by someone like Orange. However they don’t see what the app does, they simply confirm that you are supplier. It would seem to me that MS has missed the boat here again on security, I will feel very nervous about running smart phone apps unless I can sandbox them.

For example it would seem to me that the first time an app is installed its permissions should be restricted to CAS level “internet zone” . If it wishes to elevate permissions to do such things as SMS I should get a list of services I will be happy to grant to it. For example SMS, and potentially to which number. Thankfully they did see the security issue with the camera and managed code cannot access the camera, it can only load a picture from the store..So at least some one was awake at Redmond for the 5 minute mobile security meeting. Come on MS you have made manage code have the ability to be so much more secure please use it…..

Rant over…..After seeing it though I want one…so I’m going to order a smart phone ASAP….I have a feeling that one of the next killer apps will live on a smart phone…

Another Security Rant

2006-07-03T19:38:00.000+01:00

A while back I started playing with the managed support for secure stream protocols available for windows. The managed wrappers of NegotiateStream wrap up access to SSPI. In a previous post on my previous blog (Previous Rant) I mentioned that even though you asked for a secure stream to be mutually authenticated, windows could decide not to bother and use NTLM and only send your credentials. Personally I felt that the API should have thrown an exception since it did not do what was said on the tin…Having discussed it with others it was pointed out that there is a property you can check on the stream called IsMutuallyAuthenticated prior to actually using the stream, so not too bad..Assuming you remember to check..A little internal wrapper method and you could keep yourself safe.

NegotiateStream is used as a foundation building block for higher level communication abstractions such as remoting. When I use .NET remoting I have no access to the underlying stream I simply make a method call in the same way as I make a local call, the communication context is hidden from me. So if in my remoting configuration I actually specify I want to use mutual authentication and it fails to achieve it because the other end refuses to play ball it falls back to NTLM and I’m none the wiser.

One reason for opting for mutual authentication was the fact that the information Im sending to the server is for its eyes only, it may be sensitive and I don’t want it consumed by any other party. Personally I find this totally unacceptable….

So from a pure security point of view that may rule out .NET remoting which would be a real shame, since I really don’t want to go back to the days of creating my own wire format for RPC between my client and server.

So this got me thinking if there was some kind of workaround. The strategy that I think you can adopt is the following

On the server, declare your remoting service in the normal way, and in addition provide a simple tcp listener on a well known port. This tcp listener is the real trusted entity, clients should negotiate a stream to the listener using mutual authentication and after establishing it check that it was successful.. The server sends back along the stream the URL for the remoting service. The client then uses the contained URL to connect back to host..

This way the client can be sure of using mutual authentication to validate the service, after the service has been validated the client continues to use RPC based protocols.

At least one more Thread Safe Method on Windows Controls

2006-07-03T19:37:00.000+01:00

Its been long understood that it is unsafe to call methods and properties on Windows controls unless you are on the same thread that created the control. Windows controls have always been designed with single threaded environment in mind. In .NET 1.1 you could actually perform the operations and on the whole things did seem to work, until perhaps the application is deployed and then it breaks….Thankfully in 2.0 MS have added programmatic protection when the your application is connected to a debugger, and will fire an InvalidOperationException.

To work around this issue you can deploy the standard InvokeRequired pattern, were by you call InvokeRequired method on the control you wish to access and if that returns true call BeginInvoke on the control passing it a delegate instance that represents the work you wish to have carried out on the controls owning thread. Hold on you said you can’t access a control unless you are on its owning thread, that’s true in the whole accept for some select methods

InvokeRequired
BeginInvoke
Invoke
CreateGraphics

The above list was my understanding before today, but what I realised today was that another method can be added to the list. This holy method is Invalidate… I stumbled upon this whilst creating a more funky progress bar control. The control had a progress property that was being set by my background task, and once set called invalidate In order for the progress bar to be updated, this produced no exception when the invalidate was called from the non owning thread, leading me to conclude that it is perfectly safe to do so….

Which makes sense when you think about it since all it is doing is telling the OS that it needs to generate a Paint event for a given control, which ultimately will be delivered on the message queue for that controls owning thread. Since there is no underlying control interaction on the non UI thread no exception is raised.

Delegate binding in .NET 2

2006-07-03T19:36:00.001+01:00

In .NET 1.1 a delegate defined like

delegate void Proc(string foo);

Could only be instantiated with a method which had that exact signature. In .NET 2.0 they relaxed it so that a method defined as

void MyProc( object bar );

Could be wrapped by the above delegate, since MyProc parameter is more restrictive, which makes sense.However what you can't do is

delegate void Proc(int foo);

And make it wrap up

void MyProc(object bar );

The compiler will not let you...Hold on, int ultimately derives from System.Object so why not. My guess is that it comes down to boxing, since in order to make code work the method MyProc must see a reference type. The code invoking the delegate, or the CLR generated code would have to be smart enough to see that the actual implementation was expecting a reference type and perform a boxing operation on the parameters being passed. So perhaps the seamless type system is starting to show a few cracks....

NegotiateStream or is that bend over and take what’s given

2006-07-03T19:36:00.000+01:00

Been playing around with NegotiateStream, this class allows you to wrap up any other type of stream typically a socket stream with a tunnel. As part of the tunnel setup you specify the source credentials and the destination identity. The source credentials are used by the server to validate who you are and the destination identity is used in order to confirm that the server is who it says it is. The first phase of setting up the tunnel is authentication, and Windows supports two types NTLM and Kerberos. NTLM is not capable of performing mutual authentication so only cares about the client credentials. Where as Kerberos does support both and will validate that the remote end is infact who you think it should be.</p><p>So great in environments where it is important to know what’s on the other end of the wire, you need to use Kerberos, but here\'s the rub you would think that if you made a call as below, that it would enforce the use of Kerberos

NegotiateStream s = new NegotiateStream

s.AuthenticateClient( CredentialCache.DefaultCredentials , “RemoteServiceUser” );

However if the RemoteServiceUser was unknown by Windows AD you would think that it would reject the call, but oh no it simply fails back to NTLM. The only way you know it’s not done mutual authentication is when you check the s.IsMutuallyAuthenticated flag. In my mind I think there needs to be an additional methods, one that specifies the remote identity and one that doesn't. That way it’s clear that I want or don\'t want mutual authentication...If I want it and it doesn't happen then an exception needs to be thrown, I shouldn't have to check a property on the stream after the fact, and worse still if I don't want it I shouldn't have to specify anything not even String.Empty.

Fingerprinting in schools

2006-07-03T19:35:00.001+01:00

I’ve been campaigning in my kids school to stop the introduction of finger printing for school registration.Â I was interviewed by the Guardian on Tuesday, the article can be found here

.NET Generics and C++ STL

2006-07-03T19:35:00.000+01:00

A fellow instructor was commenting, well moaning to be exact about the fact that .NET generics were not as flexible as C++ templates. In this particular case he wanted to produce a generic Complex number type, such that he could write a single version and instantiate it for ints, doubles etc. This is trivial in C++ templates, but not so in .NET. I did manage to get something close C++ templates in terms of functionality, the details are too long for my primitive blogging editor so Ive written up my findings here .

Whilst undertaking this task there is a feature I would really like from the C# compiler and that is a pragma to turn auto boxing off. Why well when writing code that you wish to tune to death, such as a generic algorithm it would be nice to be told at compile time that boxing will be occurring. I guess I could write a generic unit test that inspects a method and fails if it finds a boxing op code

.NET Generics and static methods

2006-07-03T19:34:00.001+01:00

Been playing around with generics a bit more and have discovered something a little strange. When you run it you will see that you get the same result even though you instantiate the generic method with two different types that have the opposite functionality. If you know how generics work under the covers then I guess you will think I wouldn\'t be that stupid to write the code below, but If you are not completely up to speed with how it is implemented in the CLR then you may think the behaviour below is a little strange.

using System.Collections.Generic;

using System.Text;

namespace Constraints

{

public class A

{

public static bool operator ==(A lhs, A rhs)

{ return true; }

public static bool operator !=(A lhs, A rhs)

{ return false; }

}

public class B : A

{

public static bool operator ==(B lhs, B rhs)

{ return false;}

public static bool operator !=(B lhs, B rhs)

{ return true; }

}

class Program

{

public static void PerformOperation<T>(T lhs , T rhs ) where T:A

{

if (lhs == rhs)

{

Console.WriteLine(“Equal...”);

}

else

{

Console.WriteLine(“Not Equal...”)

}

static void Main(string[] args)

{

A v1 = new A();

A v2 = new A();

PerformOperation<A>(v1, v2);

B v3 = new B();

B v4 = new B();

PerformOperation<B>(v3, v4);

}

In fact the C# compiler tries to stop you falling into this trap with a generic method with no constraints. The code below will not compile, the reason being the way in which genercis have been implemented. In .NET a generic method or type are constructed using common piece of IL using special generic IL instructions. All it needs to be know at compile time is which method to call to perform a piece of functionality, the binding to the final implementation either happens at JIT time for valuetypes or using runtime polymorphism in the case of ref types. For static methods and non virtual methods the binding happens when the generic method or type is compiled only the correct specific behaviour is used if the methods are polymorphic, or you supply the same type as defined by the constraint and not a more specialised version. It is for this reason that the C# compiler will not compile the code below as it will always perform an object comparison ( E.g. Reference equality ) even if the method is called PerformOperation<string>( string1 , string2)

public static void PerformOperation<T>(T lhs , T rhs )

{

if (lhs == rhs)

{

Console.WriteLine(“Equal...”);

}

else

{

Console.WriteLine(“Not Equal...”;)

}

What I’m failing to understand is why in my case it allows the compilation since it basically suffers the same issue. Perhaps generic methods should not be allowed to use non polymorphic methods.

ASP.NET Themes

2006-07-03T19:34:00.000+01:00

The new theme support in ASP.NET is pretty cool. Although personally I don\'t think they took it far enough. When you design your page you can add an attribute to the @Page tag that states which theme to use when rendering this page, this gives the web site designer some flexibility in that they can change the look and feel of the site with a simple change. The simplest way to do this is with a single global change in the web.config

<pages theme=”MyTheme&quot”/>
</system>

</configuration>

For me though the cool thing about themes is to allow the user of the web site to select the theme they would like to use. As far as I can tell there is no direct support in ASP.NET to do this for you. So I set about writing a component that could do this. So what we need to do is. For each page as it is about to be rendered change the theme used for that page select the theme based on the user preferred theme

In order to intercept the rendering of each page at the appropriate point you can provide a handler for the Page PreInit event, inside this handler you can add the appropriate code to change the theme to use for just that page. In order to hook up to this event I decided on a design where if you wanted a page to have the user selected theme then instead of creating a WebForm that derives from System.Web.UI.Page you actually derive from my new UserThemedPage type which ultimately derives from System.Web.UI.Page. This can be simply done inside the forms Code view, Inside the constructor for UserThemedPage I add the necessary event registration code. Thus when the WebForm is created the code Â is in place to handle the PreInit event and apply some policy to determine what theme to actually use. For the second part we need to provide a way for the user to select a theme and also for the theme to be retained across multiple pages and hopefully multiple visits to the site. I didn’t really want to introduce the need for non anonymous access and store the users preferences in a database, so I opted for the use of cookies to store the user\'s prefered theme. Since not every user wishes to enable cookies, I also opted to store the theme in the session store allowing users to retain their preferred theme for the duration of the session, but would need to reselect it on re-entering the site.</p><p>As for selecting the theme to use, I did not want to provide the actual UI for this but simply a method that would allow simple iteration of possible themes to use and then a simple set/get property interface for setting the chosen theme. The iterator simply returns the set of themes that are defined for the current application, its then up to the chosen web app to decide how to ask the user for their chosen theme. So now in order to have a user select their own theme and have it persist across multiple pages and visits, you simply need to derive from my UserThemedPage as opposed to System.Web.UI.Page and provide a form to select the chosen theme. Trivial...You can download the component source and example from the Download Page