Software Architect Week Demos

Had a lot of fun again at Software Architect Week, the demos for the day of Design patterns can be downloaded from here.

Classic Snake in Silverlight 2

Inspired by Dave Wheeler's article in VSJ magazine, I decided it was time to write a simple game in Silverlight 2.  The game I chose was the classic game of Snake.

Whilst there is no doubt Silverlight 2 is a massive step closer to a .NET environment in the browser, there are still a few bits and pieces that trip you up.  Some of the collections data structures are not supported, LinkedList, Queue, Stack...there has been an underlying effort to keep the install of the platform as small as possible, so classes that have functionality that can be easily written using other classes have been tossed out.

As for WPF feature parity things have massively improved since we now have controls, not just simple shapes.  There is also now support for data binding and resources.  Things still trip you up though, transforms don't support data binding and results in less than obvious errors.

 

You can play the game  here and download the source from here

CodeGen for base decorator

One of the patterns in developmentor's "Code Smarter with design patterns in .NET course" is the decorator pattern.  The decorator pattern allows you to extend functionality at runtime, this works if the client is written in terms of an abstract type.  Using the class hierarchy below the client would be written in terms of the Component class.  The code wishing to call the client has been given an instance of ConcreteComponent, but it wants to extend the functionality of that object, in order to do this it must create another object that looks like what the client is expecting.  This is where the decorator comes in, the component decorator class below is compatible with the client since it also derives from Component. 

The componentDecorator class is a null decorator it simply supports the ability to wrap an object of type Component, and when any of its methods are invoked the call is simply passed  on to the wrapped object. Since it has no real functionality it is also declared abstract.  Each of the types DecoratorOne and DecoratorTwo overrride only the methods they wish to extend, the methods they do not wish to extend are simply handled by the base class ComponentDecorator and forwarded to the wrapped object.  Before calling the client a decorator object is created, and is given the object it is to decorate, in this case something of type component.  It is now this decorated object that is passed into the client.

The building of the ComponentDecorator class can become very tedious if the Component class has lots of methods that need to be implemented.  This week I finally got around to writing some code that via reflection and the CodeDom classes that will generate the base decorator class, saving a reasonable amount of time and effort.

The code can be found

Web Guidelines from the street

Dom found this link,

Generic Sum

Its been a while since I blogged about trying to re-implement C++ STL using .NET generics.  Last night I was skimming through an excellent book on Linq "Linq in Action" published by Manning, and whilst reading the section on how Expression trees work in C# v3.0 it occured to me that I could potentially use them for creating a generic Sum method.

First a quick look at C# v3.0 expressions,  the lambda expression here is not being turned into a method as would be the norm for anonymous methods but it is turned into an AST.  Abstract Syntax Tree used to describe the function of the lambda expression.

Expression<Func<int, int, int>> addExpr = (lhs, rhs) => lhs + rhs; Func<int, int, int> addMethod = addExpr.Compile(); Console.WriteLine(addMethod(5, 5));

In this case that produces a root node to the expression tree that is a Lambda node, that then contains a single Add node, which is a binary operator thus containing two further nodes for the left hand side and right hand side of the add operation.

Reflector spits out the following code

private static void Main(string[] args) { ParameterExpression CS$0$0000; ParameterExpression CS$0$0001; Console.WriteLine( Expression.Lambda<Func<int, int, int>>( Expression.Add( CS$0$0000 = Expression.Parameter(typeof(int), "lhs"), CS$0$0001 = Expression.Parameter(typeof(int), "rhs")), new ParameterExpression[] { CS$0$0000, CS$0$0001 } ).Compile()(5, 5)); }

This got me thinking could I build an expression using generic arguments, the code as spit out by reflector certainly looks like its possible.  My first crack was as follows

public static T Sum<T>(T[] vals) {      Expression<Func<T, T, T>> addExpr = (lhs, rhs) => lhs + rhs;       T total = default(T);          Func<T, T, T> addMethod = addExpr.Compile()           foreach (T val in vals)          {              total = addMethod(total, val);          }          return total; }

Unfortunately this failed, with the typical "Operator + cannot be applied to operands of type T and T" error that has been the reason why you can't easily write generic Sum in .NET.  However not put off I built the expression tree by hand

 

private static Func<T, T, T> CreateAdd<T>()         {             ParameterExpression lhs = Expression.Parameter(typeof(T), "lhs");             ParameterExpression rhs = Expression.Parameter(typeof(T), "rhs");             Expression<Func<T, T, T>> addExpr = Expression<Func<T, T, T>>.             Lambda<Func<T, T, T>>(                     Expression.Add(lhs, rhs),                    new ParameterExpression[] { lhs, rhs });              Func<T, T, T> addMethod = addExpr.Compile();             return addMethod;         }

The method above now returns a delegate instance that adds two values both of type T, and returns a value as type T.

Generic Sum can now be rewritten as follows

public static T Sum<T>(T[] vals) {       T total = default(T);       Func<T, T, T> addMethod = CreateAdd<T>();      foreach (T val in vals)      {            total = addMethod(total, val);       }       return total; }

 

This now works for arrays for ints, doubles etc.  It will not work for types not supported by the Add Expression, so only works for simple numeric types, but it is certainly a lot simpler than the approach I took a year or so ago although not as flexible.  Its a shame that the C# compiler expression tree builder won't accept generic arguments since it is possible to represent a expression tree this way, as this would result in a lot easier way to generate such code.

Looking for a C# Design Patterns book

I co-author  developmentor's Code smarter with Design Patterns in .NET course with Kevin Jones, and ever since the creation of the course I've been on the look out for a good book to recommend to students.  The one I currently recommend is the excellent Head First Design Patterns, although the only rub is its in Java, but in terms of teaching patterns its awesome.

I recently decided to order the latest Design Patterns for C# v3.0 from OReilly written by Judith Bishop, and see if I could recommend that text.  Yesterday it arrived, and I skimmed though it looking at the  various classic patterns Observer, Template..and my was I shocked...the implementation of some of the patterns is just plane weird, and some of the patterns basic structure is plainly wrong as defined by GOF.

I've not been through the entire catalogue of patterns, but here are some examples.

Template pattern is shown to use composition as opposed to inheritance to bind the template method to the template steps, this is plainly wrong and the implementation is therefore a strategy pattern.  As for the strategy pattern examples none of them really effectively show the use of  supplying runtime specific behaviour to  a subsystem or algorithm. 

The observer pattern uses a weird combination of delegates and IObserver interface, its either one or the other, and for .NET is almost certainly pure delegate/event.  The is no obvious reason why she has used both interface and delegates/events

The Abstract Factory pattern makes use of a generic interface for the Abstract Factory, but no where in the interface is the generic argument references, and thus is seemingly redundant.

The command pattern mixes the implementation of the invoker with the implementation of the command. 

In conclusion I'm not convinced the author either knows the patterns as defined by the GOF, nor is highly proficient in C# v3.0.  So my quest continues to find a good C# patterns book I can recommend for class..

True Generic Singleton

Last weekend I was teaching Code Smarter with Design Patterns in .NET, yep that's right a weekend...The guys at Intelliflo decided they didn't have time during the regular week for training that wanted me to pack a 4 day course into 3 days over the weekend so we worked from 9 until 19:00...

 

Whilst teaching them the Singleton pattern they bought out a version they were using built on Generics.

 

public
class
AlmostSingleton<T> where T:new()

{

private
static T instance;

private
static
object initLock = new
object();

 

public
static T GetInstance()

{

if (instance == null)

{

CreateInstance();

}

 

return instance;

}

 

private
static
void CreateInstance()

{

lock (initLock)

{

if (instance == null)

{

instance = new T();

}

}

}

}

 

public
class
Highlander : AlmostSingleton<Highlander>

{

public Highlander()

{

Console.WriteLine("There can be only one...");

}

}

 

static
void Main(string[] args)

{

Highlander highlander = Highlander.GetInstance();

Highlander highlander2 = Highlander.GetInstance();

 

Debug.Assert(object.ReferenceEquals(highlander, highlander2));

}

 

A bit of a google around and it appears a few people are advocating similar solutions. On the surface all looks well, but to be really true to the singleton pattern the type which is the singleton should only ever have a single instance. This implementation requires T to have a public constructor, so that whilst it is certainly possible to always get back to a common instance via the GetInstance method, what is lacking and what the singleton pattern strictly requires is the enforcing of only a single instance of the type. Since the fact that the type has a public constructor means that another client could easily create an instance as opposed to calling GetInstance().

 

static
void Main(string[] args)

{

Highlander highlander = Highlander.GetInstance();

Highlander highlander2 = Highlander.GetInstance();

 

Debug.Assert(object.ReferenceEquals(highlander, highlander2));

 

// This will create a second instance of Highlander..

// not what we want support

 

Highlander highlander3 = new
Highlander();

}

 

 

In fact supporting the true singleton behaviour isn't too hard, it simply requires the use of a bit of reflection to create the object instance rather than rely on the typical language new construct.

 

public
class
Highlander : Singleton<Highlander>

{

private Highlander()

{

Console.WriteLine("There can be only one...");

}

}

 

public
class
Singleton<T>

{

private
static T instance;

private
static
object initLock = new
object();

 

public
static T GetInstance()

{

if (instance == null)

{

CreateInstance();

}

 

return instance;

}

 

private
static
void CreateInstance()

{

lock (initLock)

{

if (instance == null)

{

Type t = typeof(T);

 

// Ensure there are no public constructors...

 

ConstructorInfo[] ctors = t.GetConstructors();

if (ctors.Length > 0)

{

throw
new
InvalidOperationException(

String.Format("{0} has at least one accesible ctor making it impossible to enforce singleton behaviour",

t.Name));

}

 

// Create an instance via the private constructor

instance = (T)Activator.CreateInstance(t, true);

}

}

}

}

 

Now any attempt to create the instance outside the Highlander class using the new keyword would produce a compiler error, thus enforcing the true singleton pattern. I've also added a guard to make sure that when you use the Generic wrapper it ensures that you do not have a public constructor which would ultimately allow a client to by bass the singleton functionality.

 

 

Safe Event snippet

 

This week Im teaching the course I co-wrote with Kevin Jones "Code Smarter with Design Patterns in .NET", as part of the course we introduce the observer pattern and whilst the first part of the deck deals with the basic principles we soon move on to the prefered way of doing it in .NET using delegates/events.

When using events in a multi threaded environment care needs to be taken as whilst register/unregister of the events is thread safe, care needs to be taken when raising the event.

 

Typically we see code like this

 

Public class Subject

{

public event EventHandler<EventArgs> Alert;

 

    // ...

 

    protected virtual void OnAlert(EventArgs args)

    {

        If ( Alert != null )

        {

            Alert( this , args )

        }

    }

}

 

Here a check is being made to make sure the delegate chain has at least one instance, however whilst this works in a single threaded environment in a multithreaded environment there is a race condition between the if and the firing of the event.

 

To get around this problem, one way is to take a local copy of the delegate reference, and compare that. Since register/unregister events build new lists thus making the register/unregister thread safe. However there is a simpler approach and that is to use the null pattern

 

public event EventHandler<EventArgs> Alert = delegate { };

 

This will simple create a delegate instance at the head of the delegate chain which is never removed. Thus its then safe to simply do

 

    Alert( this , args )

 

Without checking for null, since that can't happen. Ok there is an overhead, but it certainly makes the logic simpler and less prone to occasional hard to find race conditions. Remembering to do this could be a pain so I wrote a simple snippet to do it...So now I simply type safeevent TAB TAB and I always get an event with a null handler...another great use of anonymous methods...

 

 

Not so much killer app, but killer content

It used to be the case that a computer/os would succeed if it had the killer app, the app everyone needed. The spreadsheet was one of the first of that kind, now in the new media streaming age perhaps its killer content that will help to push the adoption of new technology. A student told me last week that Microsoft has done a deal so that MSN is the official home of the 2008 Olympic, the site will provide hours and hours of online coverage, bought to you by Silverlight. Could there be any better vehicle to get Silverlight plugin onto every home/office pc..

 

Read More

The potential danger of too much access to too much data

The UK government is set to build a series of large scale databases containing children's data. The charity Action on Rights for Children has put together a three part video detailing some of their concerns to this scheme.

 

http://www.youtube.com/watch?v=LRQr2VrtX-0

 
http://www.youtube.com/watch?v=KH-1IumXZbI


http://www.youtube.com/watch?v=IyXCSg-lRkA

 

Ensure local transaction snippet

Whilst teaching developmentors enet 3 class this week I had an idea for a new snippet. It came to me whilst teaching the deck on System.Transactions, I was in the middle of demonstrating to them how you need to be really careful not to cause a transaction to be promoted to the DTC ( See Previous Blog Article ). Two strategies I suggest are

 

1. Stop the DTC, net stop MSDTC
   
2. Add an Debug.Assert statement inside your TransactionScope block to ensure the current transaction is not a distributed transaction, by checking that the DistributedIdentifier is an empty GUID.
   

 

The first option is reasonable if you never need the DTC, but if you do have bits of code that does need the DTC its clearly not an option. The second approach is a bit tedious ut if you use a snippet, then its not all that bad. Below is the snippet code..

 

 

<CodeSnippet
Format="1.0.0"
xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">

<Header>

<Title>Local Transaction</Title>

<Author>Andrew Clymer</Author>

<Shortcut>lscope</Shortcut>

<Description>Creates a transaction scope and ensures at the end of the block that the transaction is only a local transaction </Description>

<SnippetTypes>

<SnippetType>SurroundsWith</SnippetType>

<SnippetType>Expansion</SnippetType>

</SnippetTypes>

</Header>

<Snippet>

<Declarations>

<Literal>

<ID>scope</ID>

<Default>scope</Default>

</Literal>

</Declarations>

<Code
Language="CSharp">

<![CDATA[

using (TransactionScope $scope$ = new TransactionScope())

{

 

$selected$

$end$

 

 

Debug.Assert(Transaction.Current.TransactionInformation.DistributedIdentifier == Guid.Empty,

"Unexpected!! Transaction is now distributed");

}

]]>

</Code>

</Snippet>

</CodeSnippet>

Pfx, Parallel Extensions take advantage of multiple cores, but be careful...

Microsoft has now made available a parallel library for .NET, its still in early stages with its aim is to make it easier to write applications to take advantage of multiple cores. Pfx has really two layers one which offers a relatively high level of abstraction, providing a Parallel.For, Parallel.Do and parallel LINQ and a lower level layer allowing programmers to schedule individual Tasks.

This has been a long time coming, .NET support for concurrency is pretty primitive even in .NET 3.0. Simple things like kick of N tasks and wait for them all to complete requires a reasonable amount of code, but on the surface Pfx looks to assist here. I spent a couple of days over xmas looking at it and understanding how it works.

 

So first task was to find some code I wanted to parallelise I frequently use a piece of logic I stole from the book "How long is a piece of string" that uses many iterations to work out the value of PI as a good async demo, so I thought I would have a go at parallelising this piece of code.

 

private
const
int N_ITERATIONS = 1000000000;

 

private
static
double CalcPi()

{

double pi = 1.0;

double multiply = -1;

 

for (int nIteration = 3; nIteration < N_ITERATIONS; nIteration += 2)

{

pi += multiply * (1.0 / (double)nIteration);

multiply *= -1;

}

 

return pi * 4.0;

}

Looking at the algorithm it doesn't look that complicated all I need to do is to make each core do a different part of the for statement and then combine the results of each core and multiple by 4.0. In fact Ive done this using the basic BeginInvoke functionality before so I know it scales...

 

So I first opted for the high level approach offered by Pfx called Parallel.For. Which it states allows me to spread loop iterations across multiple cores. In other words nIteration in my case could be 3 on one core and 5 on another at the same time.

 

private
static
double CalcPiSimpleParallel()

{

double pi = 1.0;

 

object sync = new
object();

 

//for (int nIteration = 3; nIteration < 1000000000; nIteration += 2)

 

Parallel.For<double>(0, N_ITERATIONS / 2 , 1, () => { return 0; }, (nIteration, localPi) =>

{

double multiply = 1;

// 3 5 7 9 11

 

if (nIteration % 2 == 0 )

{

multiply = -1;

}

 

localPi.ThreadLocalState += multiply * (1.0 / (3.0 + (double)nIteration* 2.0 ));

},

(localPi) => { lock (sync) { pi += localPi; } });

 

return pi * 4.0;

}

 

 

This unfortunately this did not achieve the results I had hoped for...it in fact took twice as long...Bizare...Ok so not suprising since it is using delegates. Using reflector you see that Parallel.For is using the underlying Task class provided by Pfx.

 

A Task is a piece of work that you give to a task manager and it queue's for an available thread. Now the first thing to mention is that tasks do not run on thread pool threads, a task manager has its own pool of threads which it controls depending on the number of available cores and if the tasks running are currently blocked.

 

Task task = Task.Create((o) =>

{

            Console.WriteLine("Task Running..");

        } );

 

task.Wait();

 

The code above will create a new task, and it will be run on one of the default Task Managers threads, and the main thread will wait for its completion.

 

Parallel.For, uses a special variant of Task called a replicating task, which means when you create this task the task manager will effectively not create a single instance of the task but in fact as many instances as there are cores. Each of these task clones are then expected to work in harmony to achieve the overall task. A typical implementation might therefore have an outerloop looking for a new piece of work to do in the overall task, when no more work it simply ends the task.

In the case of a Parallel.For you could expect the outerloop to simply perform an Interlocked.Increment on the loop counter until the required value is reached, and then inside the loop to simply invoke the required piece of work, and here lies the problem in my example in that the piece of work that is executing inside the for loop is extremely trivial and doesn't take that many cycles, and so the overhead of invoking a delegate for each iteration is having a large impact on the overall performance.

So first word of warning when using Pfx, you need to make sure that the piece of work inside a given task is of a reasonable size in order for it to actually scale. Ok so it's not as trivial as it first looks to parallelise the calc Pi method. Not put off I refactored the code to make the algorithm have an inner loop, thus increasing the amount of work for each parallel for iteration. To start with I simple broke it down into ten blocks, thinking each block would now be a reasonable amount of work and we would be only paying the overhead of a function call 10 times, as opposed to N_ITERATIONS/2

private
static
double CalcPiDualLoop()

{

double pi = 1.0;

 

object sync = new
object();

int stride = N_ITERATIONS / 2 / 10;

 

//for (int nIteration = 3; nIteration < 1000000000; nIteration += 2)

 

Parallel.For<double>(0, N_ITERATIONS / 2, stride, () => { return 0; }, (nIteration, localPi) =>

{

 

double multiply = 1;

// 3 5 7 9 11

 

if (nIteration % 2 == 0)

{

multiply = -1;

}

 

double piVal = 0;

for (int val = nIteration; val < nIteration + stride ; val++)

{

piVal += multiply * (1.0 / (3.0 + (double)val * 2.0));

multiply *= -1;

}

 

localPi.ThreadLocalState += piVal;

},

(localPi) => { lock (sync) { pi += localPi; } });

 

return pi * 4.0;

}

 

But only partial luck, I got a less than 10% speed up...not what I was expecting. So I had another look at the implmentation of Parallel.For to understand why, and digging into it you realise that each of the task created by parallel for, doesn't just take the next value in the loop and work on that, but takes the next eight values. So what this means is that in my case I have an 2 core CPU, the first task to run takes the first 8 iterations of the loop to work on, and the second task gets just 2. The second task completes its work, and since there is no more ends, leaving the first task to complete its remaining 6 all running on one core, leaving my other core cold.

To make sure I wasn't seeing things I made the number of itterations of Parallel a factor of 8 * number of cores, and hey presto it took just over half the time.

 

I can see that taking chunks of work at a time reduces contention for work allocation, but Im struggling to see why this would be neccessary since this contention will not be that great especically when in order to make Parallel.For to work efficiently each iteration needs to be of a reasonable size. I really hope they address this issue when they finally release the library, since having to know how many cores you are running on and magic number 8 just seems rather odd and goes against what I believe they were trying to achieve.

 

I also implemented the method using the low level parallel task library and that scales beautifully too..

 

private
static
double ParallelCalcPi()

{

object sync = new
object();

int nIteration = 3;

double pi = 1;

int stride = N_ITERATIONS / 4;

 

Task task = Task.Create((o) =>

{

Console.WriteLine("Running..{0}", Thread.CurrentThread.ManagedThreadId);

 

double localPi =0.0;

double multiply = -1;

 

long end;

while ((end = Interlocked.Add(ref nIteration, stride)) <= N_ITERATIONS + 3)

{

//Console.WriteLine("Next Chunk {0} from {1} to {2}", Thread.CurrentThread.ManagedThreadId,

// end - stride, end);

 

for (int val = (int) (end - stride); val < end; val += 2)

{

localPi += multiply * (1.0 / (double)val);

multiply *= -1;

}

}

 

lock (sync) { pi += localPi; }

 

}, TaskCreationOptions.SelfReplicating);

 

 

task.Wait();

 

return pi * 4.0;

}

 

First impressions is that the Task Library seems ok, but the Parallel.For seems ill thought out. I have also done some experiments with Parallel.Do which allows me to kick of multiple tasks, by supplying a series of Action delegates as parameters. Parallel.Do then blocks waiting for them to complete. Unfortuantly no timeout parameter is present whih means it waits forever, so if an action goes into a rogue state and never completes, your thread that invoked Parallel.Do will hang, not desirable....So my second request is that they add a timeout parameter.

 

Mobile Bingo

Just couldn't help myself in writing a bingo caller for my Mobile phone the other week, you see it was my daughter's birthday and she wanted bingo at the party and my wife had decided the low tech approach of pulling numbers out of a bag was the way to go...I wasn't having it "I could write a .NET App for my phone in 10 minutes to do that" I said. Kind of brave since it was my first mobile phone app, I paid £300 for this phone so it just has to do more than just make calls...You know what a bit more than 10 minutes say 30 and I had it working...It just goes to show easy things have become these days....Im thinking of porting wack a mole now...although I'm not sure my touch sensitive screen will last...

 

You can download the app from here

 

WPF Extension method

I used my first extension method yesterday with WPF.  I wanted to bring a control to the foreground, that means changing its position in the child list to the last child.

 

  Parent.Children.Remove(element);

  Parent.Children. Add(element);

 

What I would prefer to do is call SendToFront on the collection with the appropriate child.  Since I don't have any control over the type being used to hold the collection,  I would need to resort to extension methods to get a more object style syntax.  Resulting in the following code.

 

public static class UIElementCollectionExtensions

{

   public static void SendToFront(this UIElementCollection collection, UIElement element )

   {

        collection.Remove(element);

        collection.Add(element);

    }

}

 

All this is well and good but it would have been virtually as elegant with an old fashioned static call on some Util class.    It then dawned on me why these are perhaps so useful, for me anyway I rely on intelli sense to see what I can do with a lot of the WPF controls, with any Util class methods I write I need to have knowledge of them, but with extension methods intelli sense can potentially pick them up.  So I refactored my code once more into a separate assembly, but placing the code into the System.Windows.Controls namespace, thus whenever I use WPF controls and reference my additional extension assembly I get my new methods.  On a large scale project I can see how this could be aid productivity.

 

However there is something smelly about placing my code into someone else's namespace so on reflection I think it would be far nicer to place all my extension methods inside my own namespace, and just make sure I bring them into scope by using my extension namespace.

Too much reliance on encryption

A friend of mine was telling me recently how safe manufactures grade their safes, they don't simply say this is unbreakable/uncrackable ( only a fool would say that ). What they do say is that they say you need X amounts of TNT or N hours to crack this safe, clearly limiting their liability you may think, but actually its useful and practical information to anyone who has a safe. Firstly they know that there are no guarantees, but they also know what level of additional security they may need to layer on to get closer to full peace of mind. In the case of the fact that it takes 2 hours to crack the safe the owner could employee a security guard to patrol the safe location every 1 hour, hopefully not creating a window of opportunity for the cracker to do the deed. When you want to secure digital data similar considerations need to be taken into account.

Ive recently been involved with debating the security of biometric systems used in schools with a biometric firms Principal Sales Engineer based in the US. The issue we have as a group is that whilst Im sure all measures today are being take to secure the data in terms of encryption technology, the plain fact remains that history has shown us that what ever cryptography we used today is likely to be compromised in a reasonable time frame say 5-10 years. Therefore when we encrypt any sort of date we need to be aware of this since if the data has uses outside this time window then clearly we cannot rely just on this means of security.

At this goes to the very heart of the debate in biometrics in schools, the Engineer in question dismissed are complaints about encryption technology not being adequate for 10 years plus by first acknowledging this fact ( which is a great step forward ),

"I personally believe their will be another breakthrough in the next 10

to 15 years. Whether is it quantum computing or the DNA processor they

have been working on for the last 10 years. They can now beat a person

in tic tac toe. 5 years ago they could count to 10 with 80 percent

accuracy. They are much faster because they don't have to calculate

they just know the answer. But it is going to be a while and belive it

or not there are higher levels of encryption out there. There are 512

and even 1028 based encryption. Like the computer industry, there is

always someone out there building a stronger based encryption."

 

Further that statement shows that the industry as a whole knows we need stronger encryption because we know its only a matter of time for it to be broken, but he then goes on to say that

"As I said above the great thing about using encryption on keys and or

files is the fact that if there is a problem with a key or the actual

encryption you can encrypt the info with a better encryption or even

encrypt the encryption such as is done with 3 DES. It is DES encrypting

DES encrypting DES. The US government went from a standard of 3 DES to

AES 256. Not because 3 DES had been broken…. It has not. But because

they saw there were some weekensses that could be exploited and maybe in

the next 10 years or so it may be broken. Now do you think that all the

info that they have stored in 3 DES is still in 3 DES… I think not.

They reencrypted it in the new standard."

 

Whilst this is all well and good there is a piece to this solution that makes the US government solution ok but not for the average school. In that it relies on the fact that the person who is responsible for the re-encryption has guaranteed sole access to the data, in other words no one has taken an illicit copy, or more likely has some backup media, or an old hard disk. Whilst I can imagine that the US government has plenty of physical security measures in place to make sure they own the only copy of the data, I can't imagine that the average school will have similar system in place, and let's be realistic they can't with theft being the obvious one.

 

In fact I encountered a similar experience when working for Cisco, we were trying to pitch wireless networking to a large bank. Whilst they accepted the notion that the encryption technology we had chosen prevented illegal access to the network, we could not demonstrate to them that any data sniffed off the network could not be decrypted in a time frame that still made the data useful to an outsider. These guys were smart and did truly understand the nature of securing the business data.

So to summarise a responsible biometric manufacture would secure biometric data as best they can today, but once the software has been deployed if that data is to be truly secure it needs have sufficient physical security measures in place provided by the owner to ensure that in the future the encryption based solution still has adequate merits, the moment you do not have complete ownership of the data all bets are off...and by their own admission the biometric provider in this case said their guarantees is for appx. 10 years, in the case of biometric data for kids that data is sensitive for 60-70 years.

 

 

 

 

Patterns, Patterns everywhere

I've not blogged for a while now...mainly due to being snowed under writing the developmentor's "Code Smarter with Design Patterns" course with my co-author Kevin Jones (Course Details ) and working at a local firm ( yes Chippenham has industry). My head is now full of a handful of posts I want to make, and I've finally found time to write this one...

Whilst writing the design pattern course I managed to stumble upon various variations of standard patterns, in the course we discuss various forms of the standard Singleton pattern. Such as ThreadScoped singleton where each thread has access to a single instance, rather than a single instance app domain wide, this is potentially useful for preventing contention where multiple threads are competing over the use of the singleton. But oddly enough I stumbled upon another use of the singleton pattern when developing an example of a CopyOnWrite proxy. There are times when multiple threads request a collection of items from some object. If all the threads are only reading from the collection life is good...If there is a potential for the threads to update the collection for their private use then we should really be returning a copy of the collection to each thread, this could be considered inefficient if the collection is only occasionally privately updated, and that is where the CopyOnWrite Proxy comes in. By returning a proxy to the collection as opposed to the collection itself we allow ourselves to control the access to the collection in such a way as to provide access to the shared view of the collection whilst the thread is only reading but the moment it attempts to update the collection the proxy takes a copy of the collection it is providing the proxy for, and from that moment on the proxy is now wrapping up the copy. The client is blissfully unaware of this.

Ok, so inside my proxy implementation all the methods that are deemed write operations need to determine if a copy has been made previously and if not make a copy. In other words I will have a piece of code that I only wish to run once for the life of the object, and only once. This sounded similar to the fact that in the case of a singleton type I only want one instance. In my case I wanted to define a method in a class that only ever gets run once.

For example I was creating a proxy for a List, thus inside my proxy type I implemented the method

public
void Insert(int index, T item)

{

// Write operation, so make a private copy of the list now    

MakeCopy();

subject.Insert(index, item);

}

Where subject is the actual List. Clearly I don't want MakeCopy to run every time the insert method is called just the first time on this proxy. Simple you might say just test a Boolean flag to see if you have made a copy. Whilst this would be guaranteed to work in a non threaded environment it would not in a threaded environment, so now you end up writing code that performs some kind of synchronization prior to determining if you have made a copy or not.

In this scenario I could simply create a method called MakeCopyIfNeccessary() and call that from each write method, and make that method do the appropriate synchronization, to do this efficiently requires double check locking. However this got me thinking if there was a more re-usable way of doing this so that if I needed this functionality again I could somehow re-use not just the pattern but the code. Below is a type I defined which wraps up an Action delegate such that it will only invoke the delegate once, irrespective of the number of times you call DoOnlyOnce();

public
class
SingletonAction<T>

{

object onlyOnceLock = new
object();

private
Action<T> action;

 

public SingletonAction(Action<T> action)

{

this.action = action;

}

 

public
void DoOnlyOnce(T arg)

{

if (action != null)

{

SynchronizedDoOnlyOnce(arg);

}

}

 

private
void SynchronizedDoOnlyOnce(T arg)

{

lock (onlyOnceLock)

{

if (action != null)

{

action(arg);

action = null;

}

}

}

}

 

Inside my proxy class I create an instance of this type supplying it the corresponding method I wish to only be executed once.

class
CopyOnUpdateList<T> : IList<T>

{

private
IList<T> subject;

private
SingletonAction<object> makeCopy;

 

public CopyOnUpdateList(IList<T> list)

{

makeCopy = new
SingletonAction<object>(MakeCopy);

subject = list;

}

        // ....

public
void Insert(int index, T item)

{

makeCopy.DoOnlyOnce(null);

 

subject.Insert(index, item);

}

}

However if it wrapped up as an anonymous method this would be more true to the values of the singleton pattern as that way the only way the code could be invoked was via my wrapper. Wrapping up an arbitrary method doesn't stop it from being invoked elsewhere.

public CopyOnUpdateList(IList<T> list)

{

makeCopy = new
SingletonAction<object>(delegate

{

// Code required to perform the subject copy

});

subject = list;

}

Now the "only" way that code can be run is via my wrapper class which ensures it only ever gets run once...

Obtaining a balanced view

I was asked yesterday to visit Gloucestershire county council to discuss the use of biometrics in schools. The council have set up a committee with the responsibility of forming a policy around the use of biometrics in the county; I must congratulate Gloucestershire here as they are the first government body that I'm aware of that are actually attempting to tackle this issue fully. Currently schools are allowed to spend their money as they wish, which in one sense is great but on the other side there is always the worry that can a head of a school actually devote sufficient time to do full due diligence on a piece of technology. Certainly in the case of biometrics a lot of the issues are not obvious and present new challenges to the school such as correctly disposing of the data in a safe and secure manor, understanding the potential long term consequences of how the data may be used and abused in the future. These issues require you to be a technologist at minimum but really require someone to be a technologist visionary.

Investing time and effort in understanding the implications of biometrics does not need to be done by every school, it needs to be done centrally were money can be spent employing the experts from both sides of the argument to present a balanced and informed view, that will further enable the decision makers to make an efficient and informed decision. This information needs to be made available in a prominent place, and not buried on some government web site.

I also think it's important here to include parents in the decision making process, providing them with the information in a clear and concise form. There is precedence here, it's pretty well accepted that smoking is bad for you, people can still smoke but the government takes the responsible act of making it very clear that this could affect your long term life expectancy. Perhaps a similar mandatory warning should be made on all acceptance forms for biometric systems, just above the signature line.

"WARNING: Some experts feel that in the future it will be possible for this information to be used to steal your child's identity"

All I'm asking for is a fair representation of the facts, surely we owe it to all decision makers to make both arguments equally available and allow them to make the decision based on the full range of facts.

 

 

 

Minimum Debugging tool set

 

This week I've been teaching DM's Effective .NET course, one theme that run through the course is the ability to debug applications in the field. The tools in your arsenal here are the obvious ones like perfmon, but also the native windows debugging. The native debugger can be used to take snap shots of the suspect process (dump's seems to get a giggle from a few students). You therefore need to put the debugger on the client's machine, the typical way to do this is to download the debugging tools for windows, this is a reasonable size once unzipped 37MB...This includes a graphical debugging a few other bits and bobs, whilst these tools are useful on a developers machine they are not needed to take a snapshot and therefore not relevant for anyone other than a developer. This prompted a question from a student as to what is the minimum set of files needed to take a dump from a users machine.

After a few trial and errors we came up with the following list of files

 

    tlist.exe

    dbgeng.dll

    dbghelp.dll

    adplus.vbs

    cdb.exe

This results in a 4.6 MB unzipped install for a client's machine; this is more of a reasonable size that could be included in all employees desktop configuration.

 

 

 

Free Running Threads

Last week I was teaching Developmentor's Effective .NET 2 course, as part of that course we spend at least a day looking at various patterns for building multi threaded applications. With the rise of multi core machines it is becoming increasingly more important to write algorithms that can scale with the availability of new cores. One of the topics we cover is thread safety, and I typically write an application that simply performs an i++ operation on multiple threads, on a single core does not typically generate a problem but on a multi core means we rarely get the value we expected since i++ is in fact multiple CPU instructions, e.g

 

MOV R0,i

INC R0

MOV i,R0

One thread could perform the load of i into a register and at the same time another is also doing this thus they both cause i to be simply incremented by 1 and not by 2. To solve this problem we have a variety of synchronization techniques at our disposal some are lighter weight than others

Interlocked.Increment , Monitor.Enter/Exit and OS synchronization primitives like Mutex

 

In my stress test I conducted in class I had multiple threads updating a single integer...First I demonstrated on a multi core machine that i indeed didn't have the correct value after 10 threads had attempted to increment it 10000 times. I then moved to using Interlocked.Increment and now everything was as expected for the result, but it was slower than a simple i++. All well and good so far...I then moved to using Monitor.Enter and Monitor.Exit and to my amassment that took pretty much the same time as Interlocked...so as all engineers do...we run it again cause that result was just a glitch....but after numerous runs it kept coming out the same....So when I first developed this demo I did so on a single core machine and this was its first outing on a dual core, so what went wrong...all my multi threading life I've been told that interlocked is far cheaper than the heavier weight mutex style of synchronization. I then re-ran the demo with CPU Affinity set to a single CPU, and got the results I would have expected with Interlocked being at least an order of magnitude quicker than mutex

During the lab break I stepped back and had a think, about what could have caused this, the light eventually went on in that all the threads were in fact sharing a common variable, with dual core each core is going to be attempting to cache this value, this has a very negative effect on the cache, why well the cores try and maintain cache coherency by marking parts of the cache dirty when they update them, and forcing the other core to reload its data from main memory...If this wasn't the case in our example we would have had the wrong numbers again...So ok possibly what you would have expected....however the process of marking a piece of memory as dirty is not as simple as marking a single word, its less granular than that the core will mark whats called a cache line as dirty meaning any data on the same cache line as the value being updated is effectively marked as invalid.

So imagine two integers next two each other in memory. Thread A increments integer one and Thread B increments integer two, from a high level programming language perspective this is perfect, each thread has its own private resource and there is no need to synchronise and the threads therefore run freely...The perfect threaded app....however not so fast if both integers occupy the same cache line...we take a performance hit...

To demonstrate this fact I wrote some code that simply implements a parallel i++. There are three scenario's

• Interlocked.Increment( ref i )
  
• Interlocked.Increment( ref ManyCounters[Thread.CurrentThread.ManagedThreadId ] )
  
• Interlocked.Increment( ref ManyCounters[Thread.CurrentThread.ManagedThreadId * 10000 ] )
  

  
   

Whilst the last two variants will not produce the correct total, the middle version shows that although the threads are not sharing the same variable the fact that the fact that they probably all live on the same cache line is the crucial factor. Below is the results of running the code using 1 and 2 threads. In the first case, with the high level of contention we see that a single thread would have been more efficient.

Single shared counter with 1 Threads took 00:00:00.2065516

Single shared counter with 2 Threads took 00:00:00.4303586

In the second case with multiple counters we are still taking a hit on performance even though the counters are different ints, and thus we are not incrementing the same location

Multiple counters with 1 Threads took 00:00:00.2184554

Multiple counters with 2 Threads took 00:00:00.5177658

Only in this last case were we ensure the counters are not on the same cache line do we see both cores being used efficiently.

Multiple Sparse Counters ( ~4k apart ) with 1 Threads took 00:00:00.2180101

Multiple Sparse Counters ( ~4k apart ) with 2 Threads took 00:00:00.1603323

So what does all this mean, well it certainly shows that just because your code at a high level programming language has no contention it doesn't mean that it will not have contention when the code finally meets the hardware. This is obviously a big issue and more so with virtualisation in .NET, how do I know the size of the cache lines, or how my data will be laid out in memory...Computing just got fun again.....Code

Reduce your risk; only store what you really need

I've been drawn further into the campaign to prevent biometric information being used by schools, on Tuesday I attended a briefing session for MP's with the aim to highlight the issues with adopting this technology in the context of school children. The BBC were present and did an article for BBC online..

http://news.bbc.co.uk/1/hi/uk_politics/6336799.stm

What we as campaigners are increasingly finding hard to understand is how the department for education fails to understand the difference between data that has a relatively short validity and immutable data that last a life time ( I can't change my finger print)

For me one of the best quotes from Dfes with regard to schools holding biometric data is:-

"They are well used to handling all kinds of sensitive information to comply with data protection and confidentiality laws.

Schools have historically failed here; a forensic computer science faculty bought hard drives off ebay, and extracted school records (http://www.theregister.co.uk/2005/02/17/hard_drive_data/ ). A colleague also told me recently how he took a school computer out of a skip...Personally I don't condemn the schools here after all their primary focus is on education and not on securing personal information. In fact this is also the case in business the IT system the security are all additional burdens which do not enhance the core functionality of the business, it is seen as a necessary evil, and you often find these are given second or third rate priorities.

Information security can never be guaranteed and so we should therefore only gather the least amount of information required to perform our function. Software engineers have been aware of running applications with least privilege, thus limiting the risk that there application exposes to a system if it was to be hacked; even Microsoft is adopting this strategy at last with Vista. This therefore poses the question do schools need biometric information in order to educate our children? If the answer is no then it should not be used in schools...since this creates a further burden on a system which already showing signs of failing under the current security workload.