MarkItUp

I seem to have inherited a nasty little bug in the last day or so with Visual Studio 2008 (VS2008) and Team Explorer.  Up until yesterday morning I had been working away quite happily connecting to either our internal TFS server or to our TFSNow TFS server and using VS 2008 to work on projects in them.  But yesterday morning when I loaded up VS2008 I noticed that my working TFS server wasn't already connected up... hrmm.  Oh well, I imagine that these things happen from time to time, so I manually connected to our TFS Now instance again and voila, I was connected as usual.  However when I attempted to load up a solution from Source Control Explorer I was informed that it was not currently configured for integrated source control:

I checked my Source Control settings in Tools | Options and sure enough I had TFS selected as the default source control provider.

Attempting to manually bind the projects didn't work either because there was no recognized server to bind to:

I thought that maybe restarting VS2008 might make the magical difference... but it didn't.  In fact, when I restarted VS, it seemed to have forgotten all about my saved server settings again and so I had to repeat the process of re-connecting to the TFS Server.

I tried lots of things after that, I tried:

1. Blowing away my workspace and creating a new one - nope!
2. Deleting my local TFS cache - nope!
3. Reinstalling TFC - nope!
4. Reinstalling VS2008 and TFS - nope!

Finally, as a sanity check, I called Mitch to see whether he could connect to the TFS instance and open these projects and he could.

So I'm not really sure.  I think that I'll go down the rebuild path.  Darn!

If you are working with TFS then I behoove you to install this great TFS add-in for Outlook:

Outlook 2007 TFS Addin

When installed this add-in surfaces as a little toolbar in Outlook and allows you to create work items directly from emails simply by clicking on a "New" button.  The workflow scenarios that this enables are very useful indeed.  Now I can shoot an email off to my customer with a question, get a detailed reply from them, and convert the whole conversation to a work item.  A great way to ensure that you don't miss any customer requests while helping keep your inbox empty at the same time!

The other day I wrote this post poking fun at how ADNUG had me down - incorrectly - as speaking about EntLib at Code Camp this weekend. In the comments for that article Jeff inquired as to my opinion on EntLib - a question that I've been asked a few times of late.

The short answer is that I'm no fan of EntLib and actively avoid using it in any projects that I'm involved in. I have 2 major objections with EntLib:

1. It's a massive dependency to take
2. It's overly complex

The Dependency Thing
Last time I looked, EntLib was a collection of something like 20 projects. I don't know how many lines of code live in all of those projects but realize that when you take a dependency on EntLib that you are taking a dependency on every one of those lines.

Complexity
The argument for using EntLib (or at least the one that I've heard) seems to be that it's a framework that simplifies common tasks such as data access and adding tracing to applications. Having implemented distributed transaction support in EntLib I definitely do not agree that it simplifies data access! Oh sure it's all config file based and you could do things such as switch databases easily, but a) you can achieve that simply with standard ADO.NET code, and b) nobody ever does that anyway!

As for tracing, the tracing API's in .NET 2.0 along with their standard Listeners make tracing to multiple sources as simple as making a call such as:

Trace.TraceInformation(...) ;

So no. I really do not think that EntLib simplifies tracing either.

And then there's performance, but I won't go there without raw facts, other than to say... think about the cost associated with reading config files, the amount of reflection going on, and the overall number of objects that are being created.

Sorry EntLib - I don't love you.

This week we had a requirement to display many differently filtered views of data within a web page.  In this particular instance, the data represented a resultset of items from one of our line of business apps.  Each view of the data was based on a particular set of logic and needed to be displayed in a grid like so:

The report was for showing various exceptional states of resource scheduling data and so each category represented a different exceptional state, such as:

1. Over allocated entries
2. Under allocated entries
3. Entries where the submitted values were less than the scheduled values
4. Entries submitted on days when the sun was shining and where there were 3 or more red cars parked in the carpark

You get the idea   In creating the report I wanted to ensure that we could easily add new category sections quickly and so I went for a processing system where you could easily create and add new rules to a single processing pipeline.  So we start with a generic definition of a rule and an item that the rule will act upon:

internal class GenericItem {

     private int _id;
     public int ID { get { return _id; } set { _id = value; } }

     public GenericItem(int id) { this._id = id; }

     public override string ToString() { return "Name: " + _id.ToString(); }
}

internal abstract class ItemRule {

     List<GenericItem> _matches = new List<GenericItem>();
     public List<GenericItem> Matches { get { return this._matches; } set { this._matches = value; } }

     abstract public void Evaluate(GenericItem item);
     virtual public void Complete() { this.Clear(); }
     virtual public void Clear() { }
}

You can see from this code that each rule will receive an item to evaluate and will expose a Matches collection of the items which match the rule definition.  The ItemRule class is abstract so the implementation logic for each rule will be contained in the Evaluate method of each of the derived rule classes.  Here's a couple of simple rules which filter items based on whether they are odd or even:

internal class IsEvenRule : ItemRule {
     public override void Evaluate(GenericItem item) {
          if (item.ID % 2 == 0) this.Matches.Add(item);
     }
}

internal class IsOddRule : ItemRule {
     public override void Evaluate(GenericItem item) {
          if (item.ID % 2 == 1) this.Matches.Add(item);
     }
}

These rules are very simplistic.  They perform a simple piece of logic and then if the item matches that logic they add the item to their Matches collection.  Some rules would be much more complex though.  Consider a rule where you need to match where there are a certain number of items in the list - meaning that you couldn't fully report until you had done a complete scan of all of the items.  Here's a rule for such a situation where the Match collection is populated only where there are greater than 10 instances of odd-numbered items in the list:

internal class TenOddsRule : ItemRule {

     List<GenericItem> _evaluatedItems = new List<GenericItem>();

     public override void Evaluate(GenericItem item) {
          if (item.ID % 2 == 1) {

               if (this._evaluatedItems.Count == 9) {
                    foreach (GenericItem evaluatedItem in this._evaluatedItems) {
                         this.Matches.Add(evaluatedItem);
                    }
               }

               this._evaluatedItems.Add(item);

               if (this._evaluatedItems.Count >= 10) {
                    this.Matches.Add(item);
               }
          }
     }

     public override void Complete() {
          this._evaluatedItems.Clear();
          base.Complete();
     }
}

Notice that the rule adds has a private list that it adds items to and only populates its Matches collection when the inner-list grows to beyond 9 items.

Now we can wrap our rules logic within a generic processing pump that will contain the logic for performing iteration and applying each of the rules to the items in the list, like so:

internal class ItemProcessor {

     List<ItemRule> _rules = new List<ItemRule>();
     public void AddRule(ItemRule rule) {
          rule.Clear();
          _rules.Add(rule);
     }

     public void AddRules(IEnumerable<ItemRule> rules) {
          foreach (ItemRule rule in rules) { AddRule(rule); }
     }

     public void ProcessItems(List<GenericItem> items) {

          foreach (GenericItem item in items) {
               foreach (ItemRule rule in _rules) {
                    rule.Evaluate(item);
               }
          }

          foreach (ItemRule rule in _rules) {
               rule.Complete();
          }
     }
}

Then we can create some rules and add them to our processor:

IsEvenRule evensRule = new IsEvenRule();
IsOddRule oddsRule = new IsOddRule();
TenOddsRule tenOddsRule = new TenOddsRule();

ItemProcessor processor = new ItemProcessor();
processor.AddRules(new ItemRule[] { evensRule, oddsRule, tenOddsRule });

processor.ProcessItems(items);

With the items processed, we can now use the Matches collections to bind to our UI elements, like so:

rpt1.DataSource = evensRule.Matches ;

rpt2.DataSource = oddsRule.Matches ;
rpt3.DataSource = tenOddsRule.Matches ;

The real beauty of using this approach is that new rules can be created quickly and easily to meet the needs of the business while keeping the processing logic distinct and easy to maintain.

Generic predicates are a great way to encapsulate the logic for filtering collections.  Recently we had a business application where we had to show various views of a collection based upon certain business rules and generic predicates were the perfect match.  Generic predicates are methods that work on a List<T> and which take a delegate that works on each item in the list to return only the items that conform to a specific piece of logic.  Let's look at an example, we'll start by creating a List<T> which contains 10 items:

List<TestClass> tests = new List<TestClass>();

for (int i = 0; i < 10; i++) {
    tests.Add(new TestClass(i + 1));
}

In the above code snippet we have a List<T> variable named 'tests' which contains 10 elements and within that list each element holds an instance of the TestClass type.  Now let's imagine that this list is a list of Timesheet entries and that we are given a request to display a subset of those entries based on some business rules. 

The List<T> class has a FindAll method on it which takes a Predicate<T> argument and returns a List<T>.  Inside of the the FindAll method, each item in the List<T> is enumerated and evaluated against the Predicate<T> argument passed in to it.  The Predicate<T> is the place where we can add our business rules to return the subset list.  So based upon the list in the above snippet, let's imagine that we have a business rule to return only TestClass instances that are odd numbered.  To do this we would first write a method which represents our business rule that we can use to inspect a TestClass instance and return true or false based on whether that instance is odd numbered or not.  Such a method might look like this:

public static class BusinessRules {
     public static bool IsOdd(TestClass target) {
          return target.ID % 2 == 1;
     }
}

Because List<T>'s FindAll method takes a Predicate<T>  - a signature that our IsOdd method conforms to - we can use our IsOdd as the filter for the FindAll evaluator on our List<T. collection instance like so:

List<TestClass> odds = tests.FindAll(new Predicate<TestClass>(BusinessRules.IsOdd));

Interestingly we can also replace the argument with a delegate which matches the same signature like so:

List<TestClass> odds = tests.FindAll(delegate(TestClass target) { return target.ID % 2 == 1; });

However I like the first method because it's more self-describing to see a delegate named BusinessRules.IsOdd to infer the meaning of what's going on.  Below is the code for a program which displays the output for 2 lists based on a different set of business rules.  1 list displays even numbers while the other displays odd numbers:

private static void RunPredicateRules() {
    List<TestClass> tests = new List<TestClass>();

    for (int i = 0; i < 10; i++) {
        tests.Add(new TestClass(i + 1));
    }

    List<TestClass> evens = tests.FindAll(new Predicate<TestClass>(BusinessRules.IsEven));
    List<TestClass> odds = tests.FindAll(new Predicate<TestClass>(BusinessRules.IsOdd));

    Console.WriteLine("Displaying Evens:");
    evens.ForEach(new Action<TestClass>(DisplayAll));

    Console.WriteLine("Displaying Odds:");
    odds.ForEach(new Action<TestClass>(DisplayAll));

    Console.ReadLine();
}

 

static void DisplayAll(TestClass target) { Console.WriteLine("ID = {0}", target.ID); }

 

public static class BusinessRules {
    public static bool IsOdd(TestClass target) { return target.ID % 2 == 1; }
    public static bool IsEven(TestClass target) { return target.ID % 2 == 0; }
}

public class TestClass {
    public TestClass(int id) { this._id = id; }

    private int _id;
    public int ID { get { return _id; } set { _id = value; } }
}

There appears to be a lot of awesome video content washing around today in line with the kick-off of MIX07. 

Microsoft Expression Knowledge Center Videos

Dozens of Sliverlight videos from the Silverlight site

A bunch of Silverlight screencasts from Tim Sneath's team

As Soma announced, they've just shipped the Beta 1 release for Visual Studio “Orcas” and .NET FX 3.5.  I also saw, via Kent's blog, that the CTP's for the Orcas Express Edition's have shipped too.  I believe that you can grab them from here:

http://msdn.microsoft.com/vstudio/express/future/

Looks like a nice quick way to get me some of that Orcas ASP.NET goodness!

On the forum for my book today a reader linked to this trend graph of Google searches for C# (Blue) against VB.NET (Red):

I'm not sure exactly what that means but I thought that it was interesting data. Here's a similar graph for C# (Blue) versus Java (Red).

Finally, I also read the following interesting article that was linked to from the Google Trends page: C#: Is the party over?

Vance Morrison has written a nice article here which discusses some classes that he provides for measuring the performance of code.  He provides a download which contains the source code for two classes - CodeTimer and MultiSampleCodeTimer - that can be used to measure the performance of code.  Using these classes you can do things like:

// Measure 1000 iterations of concatenation
CodeTimer timer = new CodeTimer(1000);
    string myString = "aString";
    string outString;
    timer.Measure("Measurement Name", delegate {
       outString = myString + myString;  // measuring concatination. 
    });

... or, if you want to run more than 1 sample, you could use the MultiSampleCodeTimer and get statistics from multiple sample runs:

// Take 10 samples of measuring 1000 iterations of concatenation
MultiSampleCodeTimer timer = new MultiSampleCodeTimer(10, 1000);
    string myString = "aString";
    string outString;
    timer.Measure("Measurement Name", delegate {
       outString = myString + myString;  // measuring concatination. 
    });

When I wrote SingleUserBlog (this blogging software) I had a single thing in mind and that was to create a blog that could be totally XCOPY deployable.  In other words, you could just grab the application, copy it into a virtual directory and you'd be away - no database to install.  To do this I decided to use good 'ol XML to store all of the data in the filesystem.  In the space of a few weeks I bashed out the code for my blog and put it online... YEEHAR COWBOY :-)

Everything went fine for a long time after I deployed my blog and I was enjoying the freedom of knowing the source code intimately and therefore being able to make little tweaks at the drop of a hat.  That was until one day when my blog started getting slow, real SLOW.  With every post that I put up there it just got slower and slower until it pretty much just stopped.  My code was failing big time.

You need to measure
When it came time to having to fix my blog I had no real idea where to start - although I had a fair idea that my elite file handling routines were in some way responsible for this mess.  Instead of fumbling around I decided to immediately do some performance tuning and to measure what was going on under the hood.  To do that I used the new Performance Sessions in Visual Studio Team System Developer.  Performance Sessions allowed me to record my running code and view which methods were taking the longest and how often each of them was getting called.  Using this information I was able to pinpoint the culprit within a very short space of time and fix it immediately without the risk of making random changes all over the place.

In this article I'll show the basics of getting a performance session up and running and also show how to use PerfConsole to view the raw data from a recorded Performance Session in a more visual manner.

Finding and Fixing a Bug

For this example, we have an application which makes calls to a business layer to get data which can sometimes be an expensive operation.  Here we see some test code which simulates a bunch of calls to the service agent to test how our Service Agent method performs:

static void Run() {
    Dictionary<string, int> items = new Dictionary<string, int>();
    Random rand = new Random();

    for (int i = 0; i < runs; i++) {
        int id = rand.Next(10);
        string retVal = ServiceAgent.MakeExternalCall(id);
    }
}

Under the covers, the Service Agent calls an internal helper to process the request like so:

public static string MakeExternalCall(int id) {
    return DoesWork(id) ;
}

And it's the DoesWork method which is expensive with each call taking around 500ms to complete.  400ms isn't too bad for a single request but in modern portal applications where data may be fetched many times in a single request each of these 500ms blocks start to add up when they are performed in a linear fashion.  When we run our test we can see that for 40 runs our test takes 20 seconds to complete - 40 runs times 500ms.  In this simple example our bottleneck is pretty clear but when we have hundreds of pages and classes and different assemblies all getting called together, finding the source of a performance issue is not so obvious.  To pinpoint our bottleneck we will now create a performance session to run our test code and record the actions so that we can see some metrics behind what's going on.  To get started select Tools|Performance Tools from the Visual Studio main menu like so:

When you've selected the Performance Tools option from the menu you will be presented with a dialog window for starting a new session as shown in the following image:

Profiling code produces VERY large files - maybe even gigabytes for what might be a short profiling session.  Luckily for us we don't have a lot of code to profile so our session will be quite small this time and so we should select "Instrumentation" from the previous dialog window as opposed to "Sampling".  Instrumentation will create much larger files and so when you are profiling a large amount of code you should start with Sampling turned on until you can zoom in on the problem area as this will create smaller vsp files and so you will be able to get through more performance sessions more quickly.

When you've created your Performance Session you will have a new session in the Performance Explorer window with a selected target for the session - this will typically be a project output that is observed during the session.  In the following image we can see that the HasCacheableMethods project is loaded as a target for PerformanceSession1:

At this point you can press launch on the Performance Explorer window to start the session and fire up the target. 

When the target has finished execution the session will end and you will be presented with a report containing some metrics from the session like so:

These metrics are pretty high level and can be quite ambiguous to begin with but by double-clicking on a row in that report you can drill-down into the next level of detail where things start to make some sense:

Here we start to get a really good view over the time that each method has spent running.  In particular the "Elapsed Exclusive" metric details how long was spent only in that particular method whereas the "Elapsed Inclusive" metric is a rollup of that method and the time spent on methods called by that method and so on.  In this report we can now see clearly that the Thread.Sleep call is where nearly all of the time was spent during our session, double click on that to see where it is called from so that we get some context about the work being done:

PerfConsole

There's another tool that has been released by Microsoft that was written by an internal developer which takes the output of a profiling session and allows you to write queries over it and to view a more graphical representation of the call tree, this tool is called PerfConsole.  Before you can use PerfConsole you must save the results of your performance session out to a CSV file; this is done by right clicking on performance session report and selecting Export from the context menu:

Now you can choose what data to export and the location to create the exported CSV file

Now start up the PerfConsole tool and load the CSV file that was just created by using the load command and passing in the filepath (you can get a complete list of command line syntax for PerfConsole by typing ?? at the command line.

After loading my CSV file I can type in queries to view my performance metrics in a variety of ways, here I've used the following command to create a visual tree of the calltree

> calltree @HasCacheableMethods060811(1) | trim in > 5 | compacttree

Which displays the following result:

I'd highly encourage reading the PerfConsole command line syntax help or reading this blog article to learn more about the query syntax for PerfConsole:

    http://blogs.msdn.com/joshwil/archive/2006/08/04/688599.aspx

After viewing these reports it's now very clear that ServiceAgent.MakeExternalCall is where the Thread.Sleep code is executed from and that, aside from the Thread.Sleep, the ServiceAgent.MakeExternalCall method actually runs fairly quickly - so Thread.Sleep is our enemy here.  Now obviously we cannot optimize the Thread.Sleep code at all but we can start to ask some questions about how to get our code to spend less time in there.  When we look at what is happening in our code (and remember that the Thread.Sleep in this example is a silly placeholder for a database call or some such other expensive operation) we can see that it's getting called on every call to ServiceAgent.MakeExternalCall whereas we could actually cache results for a distinct ID after the first call to it, so we change our MakeExternalCall code to add a layer of caching in like so:

static Dictionary<int, string> cache = new Dictionary<int, string>();
public static string MakeCachedExternalCall(int id) {
    if (cache.ContainsKey(id))
        return cache[id];

    string work = DoesWork(id);
    cache.Add(id, work) ;
    return work;
}

Now that we've fixed our code we can re-run our tests and see that our test code run time has been reduced from 20 seconds to just over 4 seconds.  Not a bad return for a little 30 minute performance fixing session! :-)

After reading this article I've decided to join the dark side for a week or two to see what I think.  One of the major things that pushed me over the line was battery life - but there's also a chance that readability will be improved too.

This morning started just like any other morning... I was sitting in my home office drinking coffee and planning my day when a package arrived from UPS.  It's not terribly uncommon for me to receive parcels these days but this one was a different size and shape to the others that I normally receive.

When I opened the package I discovered that it contained 3 shiny silver cards each of which had a URL engraved on it and a registration number.  It turns out that these cards are invitations that can be given to friends for a 12 month VSTS subscription!  Get that... VSTS free for 12 months!  I thought long and hard before deciding who to give mine to but am happy that, in the end I've given them to people who would love to have VSTS but possibly wouldn't have been able to get it on their own.

I gave one to a good friend in Sydney who seems to have enjoyed receiving it :-)

    http://www.paulstovell.net/Posts/Post.aspx?postId=0846f49b-09e2-4153-a821-f09eb42ddd15

I was thinking that there might be other MVPs who would struggle to find a really good home for theirs and that it would be good if there was a place where people who really are worthy could nominate themselves as recipients.  Maybe a small clearing house for VSTS invites or something :-)

NOTE: I'm too lazy to take a photo of the cards and post the image here on my blog, but here's a picture of Ohad's invites:

I meant to post this the other day actually but I see that Howard Van Rooijen has been posting about a new VSTS template based on Scrum.  You can read more about it at these two posts:

Scrum Template Overview
http://blogs.conchango.com/howardvanrooijen/archive/2005/10/25/2304.aspx

A deeper look at the Scrum Work Item Types
http://blogs.conchango.com/howardvanrooijen/archive/2005/10/27/2311.aspx

Today, Paul blogged about his confusion over this style of C# 3.0 syntax:

var c = new Customer();
var i = 123;
var s = "Hello World";

Nice spotting!  That's not what the "var" keyword was designed for.  "var", as I pointed out in yesterday's post was intended purely to support another new feature - anonymous types.  In fact, if I saw code such as the code above in a code review, I'd be inclined to flunk it.  "var" is intended for situations where you physically cannot write the necessary "dimming" code today, such as this:

var q =
  from c in custs 
  where c.City == "Sydney"
  select new {c.Name, c.City} ;

Unecessary use of the "var" syntax could lead to some misleading code especially When mixed with with another upcoming feature, extension methods.  For example, in the following piece of code, what is the Type of "j" - where Clone() is an extension method.

int i = 10 ;
var j = i.Clone() ;

If you said that it depends then you are quite correct!  Now that we have extension methods, Clone can be anything that you've defined it to be.  So why not be explicit about variable declaration...

int i = 10 ;
int j = i.Clone() ;

In my last post I talked about the new LINQ extensions for C# and mentioned that I thought "extension methods" are pretty cool - and they are.  Extension methods allow you to bind generic predicates to existing classes.  In the case of LINQ they have used this concept to bind a new Where<T>(...) method to IEnumerable<T> meaning that you can now write code such as this to return a filtered listing of customers:

var custs = CustomerDataComponent.GetCustomers().Where(c => c.City == "Sydney") ;
foreach( Customer c in custs ) {
    Console.WriteLine( "{0} : {1}", c.Name, c.City ) ;
}

It gets better though... in addition to extension methods there is another new feature called Anonymous Types.  These are a very powerful concept.  From my current understanding, Anonymous Types can be declared like so:

var t = new {Prop1 = "Hello", Prop2 = "World"} ;
Console.WriteLine( t.Prop1 + " " + t.Prop2 ) ;

In this example, a new Anonymous Type is created and assigned to t; it's type information is inferred to the variable t which now knows about two string properties named Prop1 and Prop2.  You can use Anonymous Types with some new macro keywords in C# to create an Sql-like experience.  Here is the first example re-written using this concept:

var custs = CustomerDataComponent.GetCustomers() ;

var q =
  from c in custs 
  where c.City == "Sydney"
  select new {c.Name, c.City} ;

foreach (var c in q) {
  Console.WriteLine(c.Name);
}

The beauty here is that the variable q is type checked so that, if you attempted to write out c.Foo then the compiler would throw an error.

I've really only just started playing around with this stuff but I'm sure liking what I'm seeing.

As I mentioned yesterday, you can write a custom SchemaImporterExtension component to customize the output from XSD.  Basically I want to write my own extension so that I can control some of the syntax and semantics that are produced by XSD when I run it to create my business entity classes.  For example, out-of-the box XSD will name my fields something like: "rolecodeField" and doesn't create nice, strongly typed collections the way I want.  All-in-all I just want a very generic SchemaImporterExtension that will give me something like:

Fields: _camelCase
Properties: PascalCase
Collections: CustomCollection Inherits List<T>

At this stage I really have no idea how to write such a component but I have worked out how to attach the debugger to XSD to debug the class as you are writing it...

Grab XSD.exe (C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin) and copy it into the build directory for your project - most likely the \Bin\Debug folder.  Next, create an XSD.exe.config for XSD and add the following configuration information to it, this is the bit that tells XSD.exe to hook to your SchemaImporterExtension.

<configuration>
  <system.xml.serialization>
    <schemaImporterExtensions>
      <!-- This should refer to your SchemaImporterExtention class -->
      <add
        name="MarkItUp.GenericSchemaImporterExtention"  
        type="MarkItUp.GenericSchemaImporterExtention, Common, Version=1.0.0.0"
      />
    </schemaImporterExtensions>
  </system.xml.serialization>
</configuration>

Next, open up the Project Properties window for your Project and open up the "Debug" tab.  Set the start action to "Start External Application" and point it to the XSD.exe program that you copied into your \bin folder.  Something like:

C:\Projects\SchemaImporterTests\Common\bin\Debug\xsd.exe

In the start options field, type in the command line arguments that will allow XSD.exe to point to a .xsd file.  This will look something like:

"C:\Projects\SchemaImporterTests\XSD\Test.xsd" /classes /o:"C:\Projects\SchemaImporterTests\XSD\Bin"

This will create classes based on Test.xsd and output a file named Test.cs into the C:\Projects\SchemaImporterTests\XSD\Bin folder.

Finally, create your GenericSchemaImporterExtention class in your project and set a breakpoint somewhere in the ImportSchemaType method:

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml.Serialization.Advanced;

namespace Common
{
  class GenericSchemaImporterExtention : SchemaImporterExtension
  {
    public override string ImportSchemaType(...)
    {
      string foo = "bar";
      return foo;
    }

  }
}

When you press Debug/Start your debug symbol will be hit!   Now you simply need to add the few small lines of code into the ImportSchemaType method and you are done, easy huh? :-)

I've been working on an application that is essentially a data processing pipeline.  Due to the nature of this service we essentially started from a contract-first design principle.  This was interesting as it forced me to look at many of the application development processes from a diffrent viewpoint and, in the process I discovered some really cool tools and articles that can help you design from a schema-first standpoint and wanted to blog about them so that I don't lose some links.

SchemaImporterExtension

After the schema contracts were defined the next task was to get the code generation tools in place to assist with the generation of business classes and some of the business and data component layers.  Here are some articles that describe how to use the SchemaImporterExtension class to meet any shortfalls from the out-of-the-box XSD.exe code.

    John Bristowe's tutorial on implementing SchemaImporterExtension
    Walking XMLSchema objects

ASMX2

    Christian Weyer talking about WebService enhancements in ASMX2

Schematron

We decided to use Schematron to assist with some of the data validation for business rule data constraints.  This is really nice an allows you to easily create standalone rulesets that can be applied against incoming XML documents to validate data constraints:

    http://www.schematron.com/spec.html

I recently toiled with building a designer for use in the EntLib Configuration Tool:

    http://MarkItUp.com/Posts/Post.aspx?postId=adf62fde-bc04-4d52-8212-b4d5c53418f6

One of the things that struck me was how poorly abstracted the classes for doing this are.  I was hoping to get around to factoring out the common stuff into some base classes to help speed up the process but it seems that someone has beaten me to it :-)

    http://weblogs.asp.net/pgielens/archive/2005/08/12/422339.aspx

I've written a bit about Codezone in the past few months and even showed a picture of Dave Glover's MapPoint project appearing within the VS2005 search window - http://markitup.com/Posts/Post.aspx?postId=fef9e926-ccae-4bf4-8935-f9b220d174a0.  Today I'd like to just go over the basics of how it works and what it's value is.

The mechanics of Codezone Community Search Integration
Early this year 'Chuck' told me about the Codezone program and linked me up with it.  After that, I spent the next few months re-wiring parts of ProjectDistributor so that it could be a member of the program. 

Basically, as I've mentioned previously, projects that are added to ProjectDistributor can now also be made visible to Codezone.  This is done by the user checking a checkbox which tells ProjectDistributor to expose the metadata to a Codezone site crawler.  On the ProjectDistributor site I have a landing page for the crawler to access each of the projects that it needs to crawl, you can see it here:

     http://www.projectdistributor.net/Codezone/CodezoneIndex.aspx

The crawler lands there and then, for each project that it finds, it goes off to that page and grabs the metadata from the body of the page and also from the meta tags in the head of the document.  This is done each night.

Once the crawler has aggregated the content from the site some processing is done at the backend and then the projects are made visible via the Search facility within VS2005.

But I already use Google to search
Some people have told me that they currently use MsnSearch or Google to search for snippets or articles or whatever and that they wonder what could possibly make them change from something that already works.  Well, here's the big advantage of Codezone - you get very specific, targetted results.  Think of it this way, when you search through Google for a phrase such as "SOAP" you will get back a plethora of results ranging from the SOAP spec, through to articles about how SOAP is implemented in Java through to articles describing white, foamy, bath substances.  The results that appear within the "Codezone Community" tab for such a search would be very targetted to the context of your question. 

Through the search pane you can filter on: Language, Technology, Topic... heck, you can even filter it down to which Codezone community partner sites you'd like to include in the results (Search/Tools/Options).  So, if I wanted to find a cool SOAP article I could open the search window and configure the following parameters:

   Search Phrase: SOAP
   Language: C#, Visual Basic, XML
   Technology: Web Services, ASP.NET
   Topic: Articles and Overviews

Now I can be  pretty confident that the results that I get back will  conform to my current context and I will be getting the best that there is from not only the local search but also from MSDN and the wider community - and it will be indexed daily!

So the benefits are that now all of the artifacts which have value to solving a problem, whether they be Articles, CodeSnippets, Projects or whatever, can be made available through a single search interface.

I cranked open Visual Studio 2005 today and ran a search for MapPoint.  Look what came back as the first result - Dave's MapPoint sample from ProjectDistributor.  Cool!

I was asking Mitch about CLR Profiler for Framework V2 - which obviously shows you how often I use it :-) - and he pointed me to Brian Johnson's article about it:

    http://blogs.msdn.com/brianjo/archive/2005/06/28/433395.aspx

The other day I was alerted to a new feature in VS2005 whereby you can pre-generate an assembly with strongly-typed XmlSerializers for classes in a project.  Let me step back for a minute to explain that the XmlSerializer can be evil if not used properly. 

What the XmlSerializer does that is evil

The most important thing to be aware of is that when you use the XmlSerializer it actually dynamically creates an assembly containing special (strongly typed) readers and writers which know how to convert the types that you are serializing to and from Xml.  This process has a few gotcha's, one of which is that, to even run an XmlSerializer you need partial trust on the machine so that the compiler can compile the assembly.  The process of creating these strongly typed serializers is very expensive and taxing on machine resources because there's CodeDom, compilers, reflection and a whole host of other resources being used to achieve this.

The XmlSerializer has a few different overloaded constructors which exist to support things such as: loading a type; loading multiple types, supplying evidence.  It's important that there is a huge difference in what happens under the covers if you call one of these constructors: ctor(), ctor(Type), ctor(Type, string) and any of the others.

When you call one of those 2 simple constructors what happens is that a special assembly is created and then stored in a cache the first time that you call it.  On subsequent calls to that constructor, the cache is checked and the assembly will be pulled from there rather than re-created.  This means that you only wear the cost the first time which is good - although remember, it's a big cost. 

When you use any of the other constructors to construct the XmlSerializer, the cache is not checked.  This means that you will wear the full cost of creating the special assembly every time.  Not only that, your application will continue to grow the memory that it requires because the assemblies are created and then loaded into your AppDomain - these cannot be unloaded until the AppDomain is torn down.

Only use the simple constructors

One of the reasons for using the non-simple constructors is when you need to serialize multiple types.  There's a couple of examples of this but a pretty simple one is polymorphism.  Let's say that I have a base class called "Animal" which can have sub-classes for "Human" and for "Dog".  The classes look like this:

[XmlRoot("Animal", Namespace = "urn-MarkItUp-Demo")]
public class Animal {

  public Animal() { }

  protected string _name;
  [XmlAttribute("name")]
  public string FirstName {
    get { return _name; }
    set { _name = value; }
  }
}

[XmlRoot("Human", Namespace = "urn-MarkItUp-Demo")]
public class Human : Animal {

  public Human() { }

  public Human(string name) {
    this._name = name;
  }
}

[XmlRoot("Dog", Namespace = "urn-MarkItUp-Demo")]
public class Dog : Animal {

  public Dog() { }

  public Dog(string name) {
    this._name = name;
  }
}

In the application I will generally be passing Dog and Human classes to methods which take an "Animal" type to take advantage of the polymorphic behaviour.  If, in one of those methods, I try to serialize the instance with the XmlSerializer, my code will throw an exception.  For example the following code throws an InvalidOperation exception:

Human h = new Human("Darren Neimke");
SerializeAnimal(h);

private static void SerializeAnimal(Animal a) {
  XmlSerializer ser = new XmlSerializer(typeof(Animal));
  using (StringWriter sw = new StringWriter()) {
    ser.Serialize(sw, a);
  }
}

The reason for the exception is that, when we created the serializer passing the Animal type as an argument, the special assembly was creating under the covers using Animal as the type to base the new strongly typed reader and writer on.  Then, when we pass the Human to the Serialize method of the serializer it cannot find the type mapping in its cache so it falls over.

A way to get around this is to pass the derivitive types to the overload of the XmlSerializer so that it can create special readers and writers for each of them right up front.  So we change the construction of the XmlSerializer to look like this:

XmlSerializer ser = new XmlSerializer(typeof(Animal), new Type[] {typeof(Human), typeof(Dog)});

This works because we've told the XmlSerializer what to create up front but we've now reverted to one of the evil constructor overloads which is most certainly not what we want - just believe me on that! 

The way to get around this is to use the XmlInclude attribute on the Animal class which the XmlSerializer will use to ensure that, when an Animal is passed as the constructor argument that strongly typed serializers are also created for any other included types.  So we adorn the Animal class like so:

[XmlInclude(typeof(Human))]
[XmlInclude(typeof(Dog))]
[XmlRoot("Animal", Namespace = "urn-MarkItUp-Demo")]
public class Animal {
   ....
}

This means that we can revert to using the simple XmlSerializer constructor and get the benefits of type caching again.

Bypassing the simple constructors

In Visual Studio 2005 there is now a setting on the Project property pages under "Build" which allows you to specify that you want to create the strongly typed serializer classes at COMPILE TIME!  This means that, when you compile your assembly, another assembly will be created and copied into your \bin folder which contains the strongly-typed reader and writer pairs for each class in your project. 

Using this feature not only means that you can always use the simple constructor overload on XmlSerializer without even having to use the XmlInclude attributes but that you will also avoid the initial assembly creation which would normally occur.

How is this magic done?

The custom reader/writer classes are generated by a new tool called SGen.exe which, when you check the "automatically generate serialization assemblies" option on the projects pane, is configured to run as a post-build task.

By default, SGen will be kicked-off to create reader/writer pairs for each type in your input assembly so you should ensure that you are only using the option to automatically generate serialization assemblies for your "enity" projects.  In the case where you don't require all classes within your project to go through this process you could consider not checking the "automatically generate serialization assemblies" option and manually invoking SGen from the post-build event and uing the \t command-line option to specify which type to generate code for.

Update: Joseph pointed me at this article which shows how to do serializer pre-gen in 1.x apps: http://support.microsoft.com/kb/872800

One of the cool takeaways from the course this week was when Martin showed me how hook my own "Open With" tools in Visual Studio.  Let me explain...

What is "Open With"?
In Visual Studio (the .NET developer tool - IDE - of choice) certain files can be opened with certain tools.  For example, I can configure .html files to be opened with an HtmlEditor tool and my code files to be opened by a CodeEditor tool by default.  Associating files with their corresponding editors ensures that you get a good level of developer assistance - such as appropriate intellisense - when editing them.  Here is an image of the "Open With" dialog (which was launched by right clicking on a file in Visual Studio and choosing "Open With"):

A short digression about the EntLib configuration tool...
EntLib is a framework for working with sets of best practices providers for performing common application tasks such as Logging/Tracing, Configuration, DataAccess, etc.  One of the features of EntLib is that it makes heavy use of the AbstractFactory design pattern to choose which provider to instantiate at runtime.  This means that, in my code, when I do application data access, I'm doing it through a data access provider which is retrieved from a factory class; so the implementation can change depending on which provider my factory is returning.

Using this pattern means that you need to write configuration files to tell the factory how to create the correct provider and what properties it needs.  Using the data access provider as an example again, I would need to tell the factory which class I want to use and then tell it what the connection string would look like.

Because of the amount of common configuration file management that is required, the EntLib team decided to build a very generic tool for managing the configuration file maintenance.  Here is a snapshot of the tool in action:

Why this appeals to me
In the course Visual Studio was configured so that we could right click on configuration files and choose to "Open With" the EntLib configuration files.  This gave us the ability to have a rich UI to manage the configuration of the providers without having to leave the IDE... cool!

The new version of ASP.NET also makes heavy use of the provider model to configure things such as: Membership, SessionState, Roles, DataCaching policies, etc.  These providers are very similar to the providers in EntLib in that they are created via an abstract factory which is all managed via configuration settings and currently there is not really a great GUI tool for managing the configuration of them.  So I thought that I'd write some configuration designers for the ASP.NET providers and then I could use the generic tool to manage the configuration process.

The easy bits are the coolest
Of course writing your own configuration designers is a non-trivial excercise so I shall leave that process for another entry.  The thing that I wanted to highlight in this entry was how to make the "Open With" associations so that you can register your own custom tools to appear in the Open With dialog.  It's a registry setting!  Basically you just create the following key:

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\7.1\Default Editors\config\Enterprise Library Configuration]

... and then modify the default value for that key to point to the path of your tool.  In the case of the EntLib config tool I just pointed it to where I had it located:

"C:\Tools\EntLib DooDaddy.exe"

-- Remember to wrap the path in quotes.

Now, if you restarted Visual Studio and right clicked on a "configuration" type of file and chose "Open With", you would be presented with a new option titled "Enterprise Library Configuration".

This weekend I finished the features on ProjectDistributor for the new Visual Studio 2005 Codezone integration and I thought that I'd write this post as a bit of a walk-through of them.  Visual Studio 2005 supports the auto-installation of .vsi files which can be downloaded from the web and automatically placed into a Templates folder on the users computer.  The .vsi file is really just a .zip file (you can actually rename them as .zips and open them to view the contents) which contains a manifest and some content.  The manifest is a .vscontent file and describes the content file and it's location.  Codezone is a program which aggregates information about .vsi's from across the web and exposes them to the Visual Studio 2005 search engine.  This means that you can upload a file via ProjectDistributor and mark it as a Codezone item and, because we participate in the Codezone program, your project will be searchable (and installable) from within Visual Studio 2005!

The user experience

This is a walkthrough of what it feels like to submit Codezone content via ProjectDistributor:

1) There is a new menu item titled "Add Codezone Item" appears on the main menu which allows new users to submit ad-hoc projects as Codezone items.  Existing users will still use their normal process for uploading projects:

2) When the user clicks on the the "Add Codezone Item" link they are taken to a page which allows them to enter some details about their sample and to upload it.  NOTE: If the user is not already authenticated they will be taken to the login page where they must login or, if they don't have an account, sign up for an account and then login:

3) Regular users will still use their group/project page to create and maintain projects.  The release screen will now have an "Add to Codezone" link which will automatically submit your release to Codezone.  I should note that, if you upload your Codezone items from the general page they will get added to a special "Codezone" group:

4) Releases will not immediately be published for the Codezone crawler to see.  They will be subjected to moderation so that I can check that the content is fine and that the description is adequate:

5) When a user attempts to download your project from the ProjectDistributor website they will be informed that it is a Codezone download and informed that they will get a special download experience:

6) Once the user kicks-off the download process, the client-side .vsi handler kicks in and walks them through the install process: