SOA & Integration Services - BizTalk, WCF, WF, AppFabric etc

Why is it drug addicts and computer afficionados are both called users?
posts - 33 , comments - 43 , trackbacks - 74

Sunday, November 18, 2012

PowerShell a constant in a changing world

I've been programming for about 20 years now some of my friends have been at it for over 30. I have read many, many manuals and yes it's not my favourite past time. 

So 10 years ago I made a promise to myself to try and only learn about products which have long life times. I immediately gave up programming GUIs and concentrated on back end development as I decided that these products (Oracle, MQ Series, SQL Server, BizTalk and later WCF, WF) have longer life times and smaller incremental changes than front end products.

10 years ago I had no idea how good a decision that would turn out to be. There have been so many different Microsoft products for the front end in that time; multiple versions of Windows Forms, FrontPage, Html, Javascript,, Silverlight, SharePoint, WPF and now hopefully a stayer Metro.

I remember being at a Microsoft conference in 2006 when Martin Fowler told a crowd of developers (I'm paraphrasing) "If you don't like change then you're in the wrong business!". Well I've been in the business for 20 years and yes I'm a little resistant to change. I like my investment in reading manuals and getting certified to be time well spent!

Over the last 2 years I have been writing A LOT of PowerShell script, I think there is a good chance this product will still be around and be used for new development in 10 years, learning it is a good investment.

Posted On Sunday, November 18, 2012 10:02 AM | Comments (0) |

Sunday, October 28, 2012

Windows Metro: The hardest Hello World example I have ever seen :P 

Contrast that with Hello World in assembler on Windows:


.model flat, stdcall

option casemap :none

extrn MessageBoxA@16 : PROC

extrn ExitProcess@4 : PROC


        HelloWorld db "Hello World!", 0



        lea eax, HelloWorld

        mov ebx, 0

        push ebx

        push eax

        push eax

        push ebx

        call MessageBoxA@16

        push ebx

        call ExitProcess@4

end start

Posted On Sunday, October 28, 2012 11:04 AM | Comments (0) |

Tuesday, February 21, 2012

Developers need to understand interfaces, interfaces, interfaces!

I come across this problem a lot. I often get work to fix problems that largely stem from bad interface design.

Architects know how important interfaces are whether they be WSDL, C# interfaces, stored procedure parameters etc. Developers in my experience should get a grip on this stuff. 

Once you have some requirements spending some quality time working out what your interface is important. Developers need to hammer this into their skulls, it's pritty simple. This isn't an exhaustive post but try and follow these simple steps for good interface design (as I'm doing WSDLs at the moment so the examples will be WSDL\Xsd Schema interfaces). Please feel free to feed back your agreements\disagreements other posts which are better etc...

1) You need a list of requirements to start i.e. data fields, 1 to many relationships, min and max occurrences, business rules and how these fields should be used.

2) Ask what are the important pieces of data here i.e. if you are writing a document delivery system, the document is probably the parent and will be closer to the root of your xsd schema how, where, why, what is delivered will come under that.

3) Group common data elements together. If your document delivery system needs to email and fax documents create a common schema type called DeliveryChannel or something and put ALL the email and fax data elements in there. Maybe look up on line for some example generic\flexible schemas that deal with address information and copy them or get ideas. Good xsd design is a big topic and I won't cover it here.

4) You should now have made a start i.e. a schema that is a best attempt with what you know so far.

5) TEST YOUR INTERFACE: this isn't code testing, your interface might just be on a bit of paper at this stage it doesn't matter. I can think of two tests the interface must pass:

  1. The interface must be able to implement your business requirements, trick them off one at a time.
  2. The interface must work with existing and future interfaces which need to call it and it will need to call find out what the data fields, min and max occurrences, business rules are for the api's, components or services that will use your interface or your interface will use. Create an excel spreadsheet and map this stuff.

6) Modify your interface when it fails these tests until it passes them.

7) Finally check you don’t have redundant fields, the schema is flexible enough for the future BUT NOT RIDICULOUSLY GENERIC and related data is in the same place not littered throughout.

Try to do this or something similar at the very least before you rush in and start coding.

Posted On Tuesday, February 21, 2012 11:36 AM | Comments (1) |

Wednesday, May 26, 2010

Install of AppFabric RC stops AppFabric Monitoring (some traps for young players)


I uninstalled AppFabric Beta 2 and installed AppFabric RC.

The AppFabricEventCollection Service is started (runs under Local Service which is a dbo_owner on the Monitoring Database to prove this wasn’t the issue).

The SQLServerAgent Service is started.

Nothing is being written to the Monitoring DB Staging Table and thus nothing is being written to the Event tables or seen in the AppFabric Dashboard.

Nothing has been written to the following event logs

    - Microsoft-Windows-Application Server-System Services\Admin

    - Microsoft-Windows-Application Server-System Services\Operational

The Microsoft-Windows-Application Server-System Services\Debug event log is not shown in the event viewer.

The WCF configuration appears fine the connection string to the Monitoring DB is correct. Monitoring is set to “Trouble Shooting” and no errors are shown on the “Configure WCF and WF for Application” dialog.

So the problem seems to lie with either AppFabric which writes to the event log or the AppFabricEventCollection Service.

I thought I was flummoxed...

However one of my colleagues said have you checked the etwProviderId? I was using a config which was created under AppFabric  Beta 2 which had a different etwProviderId. So I deleted the following section and all other references to AppFabric monitoring from the web.config and then recreated them using IIS the “Configure WCF and WF for Application” dialog and set the level to TroubleShooting.

        <diagnostics etwProviderId="6b44a7ff-9db4-4723-b8cf-1b584bf1591b">

            <endToEndTracing propagateActivity="true" messageFlowTracing="true" />



I then called a service to create some log entries. Still nothing was written to the Monitoring DB Staging Table...

I checked the Microsoft-Windows-Application Server-System Services\Admin event log. It had the following entry...

Configuration error. Please see the details to correct the problem. \rDetailed information:\r Filename: \\?\C:\Users\xxx\Documents\dotnetdev\Frameworks\SOA\xxx.SOA.Framework\xxx.SOA.Framework.MockServices\SimpleServiceParent\web.config

Error: Cannot read configuration file due to insufficient permissions


 System.UnauthorizedAccessException: Filename: \\?\C:\Users\xxx\Documents\dotnetdev\Frameworks\SOA\xxx.SOA.Framework\xxx.SOA.Framework.MockServices\SimpleServiceParent\web.config

Error: Cannot read configuration file due to insufficient permissions


And guess who the user was... Local Service yes yes I should have used a better User in the AppFabric RC setup to run the AppFabricEventCollection Service under!

So I changed the user to a more appropriate one and removed Local Service as a DBO and hay presto!

Posted On Wednesday, May 26, 2010 9:05 PM | Comments (1) |

I'm back

I haven’t posted much for a while but after 3 years as a Solution Architect I’m back to my old role designing and developing SOA Frameworks and Management Platforms.
So you may be hearing a bit more from me in the future.

Posted On Wednesday, May 26, 2010 9:03 PM | Comments (1) |

Wednesday, June 20, 2007

More Geek Humour - What Da

Simon an architect I work with was talking to a developer this morning. The conversation went as so…


Simon: Are you sure you want to use FTP to transfer this data I’m not sure it will be fast enough

Developer: Well FTP is faster than TCPIP

Simon (to himself): I have now entered the twilight zone


Posted On Wednesday, June 20, 2007 1:08 AM | Comments (0) |

Wednesday, April 25, 2007

Geek Humour - Do you want to call me ... do you really want to call me

We finally got an EJB Web Service today from a Trading Partner the interface looked something like this ...

public Asset ReturnAsset(bool ReturnAsset)

We surmised that due to cronic shortages of quality developers they have finally begun to teach chimpanzees Java, not bad for a chimp.

Also liked this newsgroup thread Simon found yesterday... 

Posted On Wednesday, April 25, 2007 5:18 PM | Comments (0) |

Tuesday, April 17, 2007

Jesus Built My Hotrod

A new online BizTalk mag has come out, a more pleasent read than most whitepapers, especially if you're a motor head or a fan of that Ministry song.

Posted On Tuesday, April 17, 2007 9:36 PM | Comments (0) |

Screen Shots Missing

I just noticed that in GeeksWithBlogs move to SubText some of my old screen shots have been lost, I'll fix these as soon as I get time.

Posted On Tuesday, April 17, 2007 8:08 PM | Comments (3) |

Unable to open Orchestration Designer in Visual Studio

We are developing BizTalk applications with Visual Studio Team Edition and every now and then for some reason the Orchestration Designer is removed from the "Open With" dialog, thus when one double clicks on an orchestration it is opened with the Xml Editor. Although I'm not sure why this happens, I suspected the items in this dialog were configured in the Windows Registry ... and so they were, so to fix:

1) Open RegEdit
2) Go to HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\8.0\Editors and find the BizTalk Orchestration Designer for us it was under Guid {679b7fd6-2104-42b2-8d87-86dd575fc269}
3) Get the package key Guid, for us this was {2f926337-2bfb-46ab-bbc4-a955ce25ff6f}
4) Go to HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\8.0\Packages\{Package Key Guid}\SkipLoading
5) Set the SkipLoading key to 0
6) Restart Visual Studio and the Orchestration Designer should show up in the "Open With" list

Does anyone know why the SkipLoading key is mysterously turned on for the Orchestration Designer?

Posted On Tuesday, April 17, 2007 7:55 PM | Comments (13) |

Tuesday, April 10, 2007

System Testing BizTalk Applications with BAM & Xlsb File Generation

In the past I have noticed that system testers find it difficult to test middleware applications such as those implemented using BizTalk. System testers generally don't want to read BizTalk xml messages dumped out to disk, they like to be able to query for their test results via some form of GUI. Many are adept at writing SQL to return result sets containing the test data they want to analyze.

I came up with the idea of using BAM relationships between different activities within a single view to log before and after snapshots of BizTalk messages. So when messages were transformed by maps, orchestrations or the BRE system testers could test those transforms. You can then give system testers’ access to the BAM Portal Website where they could view and query these before and after snapshots and match the actuals against the expected results to pass or fail their tests.

This solution for system testing only works for certain scenarios’ with certain types of schemas as each message logged cannot contain repeating records if it is to be logged to BAM.

So what am I on about? Take a single record schema which validates an xml message like the one below as I said before this technique won't work with messages which contain multiple records.

   <ns1:SampleTransaction TransactionID="123456" ProductName="ABC" Amount="1234.56" Quantity="24" xmlns:ns1="http://SynbogalooSamples/SampleTransaction/" />

Each time a map, orchestration or business rules were applied to a message I wanted to send messages to BAM so I was basically just logging messages to BAM each time the contents of the message changed. I used BAM relationships to associate the message before it was transformed to the message after it was transformed so the tester could assess whether the transformation worked.

OK this was all easy but very time consuming using the Excel add-in to create the xlsb files for each schema so I wrote a simple xlsb file generator. Note: For this code to work with schemas that have xsd types other than DateTime, Integer, Double and String you will need to add to the switch statement in the CreateActivity method.

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.IO;

namespace BAMXslbGenerator
    class Program
        private const string UCName = "Name";
        private const string LCName = "name";
        private const string View = "View";
        private const string BAMDefinition = "BAMDefinition";
        private const string BAMDefinitionNamespace = "";
        private const string Extension = "Extension";
        private const string OWC = "OWC";
        private const string OWCNamespace = "urn:schemas-microsoft-com:office:excel";
        private const string ActivityView = "ActivityView";
        private const string ActivityRef = "ActivityRef";
        private const string Activity = "Activity";
        private const string Alias = "Alias";
        private const string CheckpointRef = "CheckpointRef";
        private const string Id = "ID";
        private const string Checkpoint = "Checkpoint";
        private const string XpathToAttribute =
        private const string DataType = "DataType";
        private const string DataLength = "DataLength";
        private const string Length = "length";
        private const string MaxLength = "maxLength";
        private const string XpathToLength = "*[local-name()='simpleType']/*[local-name()='restriction']/*[local-name()='length']/@value";
        private const string XpathToMaxLength = "*[local-name()='simpleType']/*[local-name()='restriction']/*[local-name()='maxLength']/@value";
        private const string SqlDateTime = "DATETIME";
        private const string SqlNvarchar = "NVARCHAR";
        private const string SqlInt = "INT";
        private const string SqlFloat = "FLOAT";
        private const string XsdDateTime = "xs:dateTime";
        private const string XsdString = "xs:string";
        private const string XsdInteger = "xs:int";
        private const string XsdDouble = "xs:double";

        private static StringBuilder xlsbText = null;

        static void Main(string[] args)
                //Check for help
                if (args.Length == 0)
                else if ((args[0] == "/?") || (args[0] == "/help"))
                    //Parse out command line arguements
                    string xlsbFileNameArg = GetCommand("/x", args);
                    string viewNameArg = GetCommand("/v", args);
                    string activityNameArg = GetCommand("/a", args).Replace("[","").Replace("]","");
                    string schemaArg = GetCommand("/s", args).Replace("[", "").Replace("]", ""); ;

                    if (xlsbFileNameArg != string.Empty && viewNameArg != string.Empty
                        && activityNameArg != string.Empty && schemaArg != string.Empty)
                        string[] activityNames = activityNameArg.Split(new string[] { "," }, StringSplitOptions.None);
                        string[] schemas = schemaArg.Split(new string[] { "," }, StringSplitOptions.None);

                        GenerateXlsbXml(viewNameArg, activityNames, schemas);
                        //Write out the file
                        File.AppendAllText(xlsbFileNameArg, xlsbText.ToString(), Encoding.Unicode);
                        Console.WriteLine("Incorrect arguements!");
            catch (System.Exception ex)

        private static string GetCommand(string commandToken, string[] args)
            string commandValue = string.Empty;

            for (int i = 0; i < args.Length; i++)
                if (args[i] == commandToken)
                    commandValue = args[i + 1];
            return commandValue;

        private static void ShowUsageMessage()
            Console.WriteLine("Generates an xlsb BAM definition file conaining one view and multiple schema definitions to log to BAM.");
            Console.WriteLine("Syntax: BAMXlsbGenerator");
            Console.WriteLine("\t/x \txlsb file name");
            Console.WriteLine("\t/v \tBAM View definition name");
            Console.WriteLine("\t/a[] \tActivity names enclosed in [] comma separated, there must be one activity name per schema.");
            Console.WriteLine("\t/s[] \tSchema names enclosed in [] comma separated, there must be one schema name per activity.");
            Console.WriteLine("\t/? or /help \tDisplay this usage message");

        private static void GenerateXlsbXml(string viewName, string[] activityNames, string[] schemas)
            //Create an writer for building the xlsb
            xlsbText = new StringBuilder();
            XmlWriterSettings xmlWriterSettings = new XmlWriterSettings();
            xmlWriterSettings.Indent = true;
            XmlWriter xlsbWriter = XmlWriter.Create(xlsbText, xmlWriterSettings);

            //Start building xml
            xlsbWriter.WriteStartElement(BAMDefinition, BAMDefinitionNamespace);

            if (schemas.Length != activityNames.Length)
                throw new ApplicationException("The number of activity names and schemas don't match.");

            Dictionary<string, string>[] checkpointIDsForSchemas = new Dictionary<string, string>[schemas.Length];
            string[] activityIDs = new string[schemas.Length];

            //Create the activities
            for (int i = 0; i < schemas.Length; i++)
                Dictionary<string, string> checkpointIDs;
                string activityID;
                CreateActivity(xlsbWriter, schemas[i], activityNames[i],
                    out checkpointIDs, out activityID);
                checkpointIDsForSchemas[i] = checkpointIDs;
                activityIDs[i] = activityID;

            //Now create the view
            xlsbWriter.WriteAttributeString(UCName, viewName);
            string viewID = Id + Guid.NewGuid().ToString("N");
            xlsbWriter.WriteAttributeString(Id, viewID);

            //Create Activity Views
            for (int i = 0; i < schemas.Length; i++)
                CreateActivityView(xlsbWriter, activityNames[i],
                    checkpointIDsForSchemas[i], activityIDs[i]);

            xlsbWriter.WriteEndElement(); //END View

            xlsbWriter.WriteElementString(OWC, OWCNamespace, "");

            xlsbWriter.WriteEndElement(); //END Extension
            xlsbWriter.WriteEndElement(); //END BAMDefinition

        private static void CreateActivityView(XmlWriter xlsbWriter, string activityName,
            Dictionary<string, string> checkpointIDs, string activityID)
            xlsbWriter.WriteAttributeString(UCName, View + activityName);
            xlsbWriter.WriteAttributeString(Id, Id + Guid.NewGuid().ToString("N"));
            xlsbWriter.WriteAttributeString(ActivityRef, activityID);

            //Loop through the dictionary and create the Aliases
            foreach (string key in checkpointIDs.Keys)
                xlsbWriter.WriteAttributeString(UCName, key);
                xlsbWriter.WriteAttributeString(Id, Id + Guid.NewGuid().ToString("N"));
                xlsbWriter.WriteElementString(CheckpointRef, checkpointIDs[key]);
                xlsbWriter.WriteEndElement(); //END Alias  

            xlsbWriter.WriteEndElement(); //END ActivityView 

        private static void CreateActivity(XmlWriter xlsbWriter, string schema,
            string activityName, out Dictionary<string, string> checkpointIDs,
            out string activityID)
            xlsbWriter.WriteAttributeString(UCName, activityName);
            activityID = Id + Guid.NewGuid().ToString("N");
            xlsbWriter.WriteAttributeString(Id, activityID);

            //Create a dictionary object to store the IDs in
            checkpointIDs = new Dictionary<string, string>();

            //Open the schema
            XmlDocument xmlDocument = new XmlDocument();

            //The rules are

            XmlNodeList xmlNodeList = xmlDocument.SelectNodes(XpathToAttribute);

            //Loop through all the attributes and create the new xslb schema
            foreach (XmlNode xmlNode in xmlNodeList)
                string checkpointID = Id + Guid.NewGuid().ToString("N");
                checkpointIDs.Add(xmlNode.Attributes[LCName].Value, checkpointID);

                //Add the data type attributes
                if (xmlNode.OuterXml.Contains(XsdDateTime))
                    xlsbWriter.WriteAttributeString(UCName, xmlNode.Attributes[LCName].Value);
                    xlsbWriter.WriteAttributeString(Id, checkpointID);

                    xlsbWriter.WriteAttributeString(DataType, SqlDateTime);

                    xlsbWriter.WriteEndElement(); //End Checkpoint
                else if (xmlNode.OuterXml.Contains(XsdInteger))
                    xlsbWriter.WriteAttributeString(UCName, xmlNode.Attributes[LCName].Value);
                    xlsbWriter.WriteAttributeString(Id, checkpointID);

                    xlsbWriter.WriteAttributeString(DataType, SqlInt);

                    xlsbWriter.WriteEndElement(); //End Checkpoint
                else if (xmlNode.OuterXml.Contains(XsdDouble))
                    xlsbWriter.WriteAttributeString(UCName, xmlNode.Attributes[LCName].Value);
                    xlsbWriter.WriteAttributeString(Id, checkpointID);

                    xlsbWriter.WriteAttributeString(DataType, SqlFloat);

                    xlsbWriter.WriteEndElement(); //End Checkpoint
                else if (xmlNode.InnerXml.Contains(XsdString))
                    xlsbWriter.WriteAttributeString(UCName, xmlNode.Attributes[LCName].Value);
                    xlsbWriter.WriteAttributeString(Id, checkpointID);

                    xlsbWriter.WriteAttributeString(DataType, SqlNvarchar);

                    if (xmlNode.InnerXml.Contains(Length))
                    else if (xmlNode.InnerXml.Contains(MaxLength))
                        xlsbWriter.WriteAttributeString(DataLength, "255");

                    xlsbWriter.WriteEndElement(); //End Checkpoint
                else if (xmlNode.OuterXml.Contains(XsdString))
                    xlsbWriter.WriteAttributeString(UCName, xmlNode.Attributes[LCName].Value);
                    xlsbWriter.WriteAttributeString(Id, checkpointID);

                    xlsbWriter.WriteAttributeString(DataType, SqlNvarchar);
                    xlsbWriter.WriteAttributeString(DataLength, "255");

                    xlsbWriter.WriteEndElement(); //End Checkpoint
                    Console.Write("Unknown xsd datatype: " + xmlNode.OuterXml + " continuing to create xlsb.");
            xlsbWriter.WriteEndElement(); //End Activity

How to call example: BAMXlsbGenerator /x "C:\Projects\SynboogalooSamples\Test.xlsb" /v TestView /a [TestActivityA,TestActivityB] /s [C:\Projects\SynboogalooSamples\SampleTransactionA.xsd,C:\Projects\SynboogalooSamples\SampleTransactionB.xsd]

This saved me a lot of time all I needed to do now to finish setting up BAM was deploy the xlsb BAM definition using the BM.exe tool and use the Tracking Profile Editor to associate the message payload with the BAM activity definition about 5 minutes of drag n' drop work. In the BAM Portal system testers can query transformed data and can even query the BAM tables if they want. They are able through BAM relationships to view the message contents each time it has been modified and logged to BAM.

I found this technique was a quick and simple way of empowering system testers to test BizTalk applications.

I should note another option for System Testing EAI solutions is to go with an implementation of FIT (Framework For Integration Testing)  here is a blog with more info.


Posted On Tuesday, April 10, 2007 6:37 AM | Comments (0) |

Wednesday, April 4, 2007

Passed the BizTalk 2006 Exam

84% it wasn't that bad, but it should have been easy after 3+ years working with BizTalk, lots on BAM and the BRE. I think I lost marks on the drag and drop questions and there was a tricky question on coding a BRE fact retriever. I used Saravana's exam preparation diary.

Posted On Wednesday, April 4, 2007 9:25 PM | Comments (4) |

Tuesday, April 3, 2007

BizTalk R2 Beta 2 is supposed to be a public release, so non-TAPpers can get their hands on it

BizTalk Server 2006 R2 Beta 2 released - Thanks Mike ;)

Posted On Tuesday, April 3, 2007 8:26 PM | Comments (1) |

Monday, March 26, 2007

in a sequential convoy the ports must be identical

A client was getting this compile error "in a sequential convoy the ports must be identical" when compiling an orchestration which had two consecutive receive ports in a sequential convoy. The answer was simple they needed to use a concurrent convoy not a sequential convoy.

Posted On Monday, March 26, 2007 8:46 PM | Comments (1) |

Thursday, March 29, 2007

A must have for WCF development

If you haven't already download and install the Service Factory it really reduces the time taken to build and publish your WCF services to IIS letting you concentrate on implementing business logic. Unfortunately it doesn't come with any refactoring functionality.


Posted On Thursday, March 29, 2007 4:30 AM | Comments (0) |

Powered by: