Thursday, July 19, 2007
In the BizTalk Hotrod quarterly ezine. You can find a condensed version of the article at http://www.biztalkhotrod.com
The full version (huge!) can be found at http://biztalkhotrod.com/Documents/BizTalk%20Smack%20Down%20Full.pdf
The article was a ton of work but was well worth the effort. The intent was to have some fun comparing BizTalk and WF/WCF. Let me know if you have any feedback!
Sunday, May 06, 2007
Thanks to all who attended my session on BAM last weekend.
You can find the presentation and demo source code here. Note that I did not get the WCF Interception running so the XML file for that is not included - I'll discuss that in a follow up post in the next week or two.
To run the demo artifacts you'll need BizTalk Server 2006 R2 Beta 2, .NET 3.0 installed and Excel 2007 installed.
Monday, April 23, 2007
I'm a little late in posting about this but there is a new BizTalk-specific "magazine" called BizTalk Hotrod: http://biztalkhotrod.com/default.aspx
The first issue was published just a few days ago.
Enjoy!
The second Twin Cities Code Camp will occur this Saturday, April 28th. Here's the website, which includes a schedule and list of sessions: http://www.twincitiescodecamp.com/TCCC/Default.aspx
I'll be presenting on utilizing BizTalk's BAM features from non-BizTalk applications.
Wednesday, December 06, 2006
This post disappeared from a previous blog of mine so I'm reposting it.
---
'Thank you' to everyone who attended the Debugging Tips and Tricks webcast that I presented today for the BizTalkRocks community. You can find the presentation at http://www.biztalknuggets.com/presentations/BizTalk Debugging Tips and Tricks.zip
I recorded the webcast. I will repost this article and include a link to the recorded webcast when it is posted.
If you have any questions on the presentation or the webcast please contact me through the Contact link on this blog.
Update (12/9/05): I think I skipped a slide yesterday. If you download the presentation, search for 'BizUnit' to find another 'Other Tools...' slide. Sorry about that.
Update (3/26/07): Updated the link to the presentation.
Tuesday, December 05, 2006
If you’ve worked with BAM and the BAM Portal in BizTalk 2006 you’ve noticed that the PivotTables in the My Views - Aggegations section shows up with ugly names like PivotTable13, PivotTable14, etc. This is fine when you are doing POCs with a small number of Views/PivotTables but it can quickly become confusing – and more importantly these names aren’t appropriate when you’re deploying a real BAM solution to end users.
If you want to change these names before you deploy your BAM Definition you need to:
- Export the BAM Definition to XML using the BAM menu in Excel.
- Open the exported BAM Definition
- Update each /BAMDefinition/Extension/OWC/PivotTableView/PivotTable/PivotView/Label/Caption element’s text to use a more appropriate name; note that this applies to both Real-Time aggregations and scheduled aggregations.
[Update 12/7/06 - You can also right click the PivotTable in Excel, select Table Options and then change the Name.]
Now when deploying the BAM Definition, use the .xml version of it and then you will see your version of the PivotTable names in the My Views section of the BAM Portal.
Wednesday, December 06, 2006
The Twin Cities BizTalk User Group now has a website, you can find it here: http://www.mnbiztalk.com/
If you are a BizTalker, and live in or around the Twin Cities, and you haven't attended a session yet - I strongly encourage you to attend. The sessions are great and the environment isn't as "stiff" as some of the other users groups out there :)
It took a while for the online version to appear but you can find the article, "Introduction to BizTalk Server's Features and Benefits" ,
here.
This post disappeared from a previous blog of mine so I'm reposting it.
---
A couple of weeks ago I was having difficulty turning on Windows Integrated security for a couple of virtual directories that were interfaces to the SOAP and HTTP adapter. I initially had Anonymous and Integrated Windows turned on. As soon as I turned off anonymous and left only Integrated Windows security turned on I would get prompted to enter my credentials. After three failed attempts to login I would get the following error message: ‘HTTP Error 401.2 - Unauthorized: Access is denied due to server configuration.’
I assumed that I didn’t have the ACLs properly configured on the directories and files that the virtual directories managed but after reviewing their configuration everything checked out.
I stumbled across a reference to AuthDiag, a new tool from MS that helps you troubleshoot authentication and authorization issues with IIS 5.x and 6.0. I installed the tool, ran the ‘Check Authentication’ test on the virtual directories and quickly discovered that my issues had something to do with the fact the identity for the application pools that my virtual directories were running in wasn’t configured properly. Auth Diag kept giving me a failure with the message ‘Service principal name (SPN) for user xx\yy not found in Active Directory’ where xx was the domain and yy was the user name for the account I was using as the application pool identities. (As you probably know, if you have multiple BizTalk Servers in your BizTalk Group then you must use domain accounts for you host instance credentials and I was using the same account for the application pool identities.)
After a suggestion from a colleague, I switched the identities for the application pools to a local account (Network Service) and the prompt and 401 immediately went away. Of course the requests couldn’t make it into BizTalk because the local Network Service Account didn’t have the permissions to reach the BizTalk MessageBox database.
Another colleague suggested using wfetch.exe, part of the IIS 6.0 Resource Kit to help debug my issue. I ran wfetch.exe using NTLM (for the ‘Auth’ value in the ‘Authentication’ group box) and could access the virtual directories without problems. However, when using either Kerberos or Negotiate I saw the following error message in the response from the IIS: ‘0x80090322 (The target principal name is incorrect.): Unable to InitializeSecurityContext’. This article describes the differences between these three options although it describes these in the context of the IIS authentication and not wfectch.exe
I was stumped until I found the Configuration Application Pool Identity in the IIS 6.0 documentation. According to that page, there are issues with using Kerberos (which may be used when you use Integrated Windows Security.) The solution to my problem was that I needed to run the setspn.exe tool in order to set a Service Principal Name (SPN) on the account configured as the Identity for the application pools. We ran setspn.exe using the machine name and the machine’s fully qualified domain name, and the issue immediately went away:
setspn -A HTTP/servername domain\account
setspn -A HTTP/servername.fullyqualifieddomainname domain\account
You can find the setspn.exe tool on the Windows 2003 CD-ROM or .ISO at \support\tools\suptools.msi
One thing to note about setspn.exe is that you must be a domain administrator, of the domain that the account is in, that you use as a parameter to setnspn.exe (which is also the account you are using for the application pool identity. There is a note here (click on the ‘Troubleshooting Setspn’ link) that says the permission necessary for running the setspn.exe can be delegated.
In summary, you must execute setspn.exe if you are using domain accounts for your application pool identities if you are using Integrated Windows Security.
This post disappeared from a previous blog of mine so I'm reposting it.
---
I've installed BizTalk 2004 on a developer workstation several times at this point. For a developer workstation I recommend the following:
1) If you do 'regular' ASP.NET development on your workstation and you will continue to do so after installing/configuring BizTalk then when you are configuring Windows SharePoint Services (WSS) do not configure it to use your default website. Create a second website in IIS and point to this site during WSS configuration (and later during BizTalk configuration point to this site instead of your default web site for the BAS site location.) If you don't do this you will have to put your ASP.NET applications on a second website that doesn't run on port 80 or you will have to configure WSS to not 'apply' to each Virtual Directory that you want to use for ASP.NET applications.
2) I have never succesfully been able to create the WSS configuration db from the WSS configuration wizard. I always get an access denied error when it tries to create the configuration database, complaining about NT Authority\Network Service not having appropriate access to the database (or something like that.) Instead, I create the WSS configuration database prior to configuring WSS using the Using the Command Line to Create or Connect to a Configuration Database section of the Administrator's Guide for SharePoint Services. This section describes how you can use the following command to create the config database:
stsadm -o setconfigdb -ds -dn
You can then point to the database you created with stsadm during the WSS configuration.
This post disappeared from a previous blog of mine so I'm reposting it.
---
I've been doing some research on custom adapter development and one of the issues that came up was figuring out how to do the schema generation. BizTalk Server 2004 includes a wizard that will generate schemas from Xml instance documents. I decided to figure out how it worked - and if it could be reused.
It is possible to reuse this functionality. Here's how you do it:
- Create a console project.
- Reference the Microsoft.BizTalk.WFXToXSDGenerator.dll and XSDInfer.dll assemblies in the [installDir]\Microsoft BizTalk Server 2004\SDK\Utilities\Schema Generator directory.
- Reference the Microsoft.BizTalk.SchemaEditor.Extensibility.dll assembly in the [installDir]\Program Files\Microsoft BizTalk Server 2004\Developer Tools directory.
- Pull in the Microsoft.BizTalk.WfxToXSDGenerator namespace.
- Add the following code to your Main method:
WfxToXSDGenerator gen = new WfxToXSDGenerator();
gen.GenerateSchema(args[0], args[1]);
object[] warnings = gen.Warnings();
object[] errors = gen.Errors();
object[] schemas = gen.ReferencedSchemas();
if(warnings.Length != 0)
{
Console.WriteLine("Warnings were found:");
for(int i = 0; i < warnings.Length; i++)
{
Console.WriteLine(warnings[i].ToString());
}
Console.ReadLine();
return;
}
if(errors.Length != 0)
{
Console.WriteLine("Errors were found:");
for(int i = 0; i < warnings.Length; i++)
{
Console.WriteLine(errors[i].ToString());
}
Console.ReadLine();
return;
}
if(schemas.Length !=0)
{
Console.WriteLine("The following schemas were generated:");
for(int i = 0; i < schemas.Length; i++)
{
Console.WriteLine(schemas[i].ToString());
}
Console.ReadLine();
return;
}
Notes:
- The first argument to the GenerateSchema method is the location of the Xml instance document. The second argument is the output directory for the generated schemas.
- A Warning and Errors array is present; oddly you access these through methods, not properties. I didn't run into any errors or warnings so I didn't investigate their Types; my assumption is that they are string but it would be easy for you to figure this out if you run into issues.
- GenerateSchema() doesn't overwrite schemas you already created with it; it adds a n (a number starting with 1) to the end of the filename (i.e. sample1.xml)
- There does appear to be another MS Xsd inference assembly out there (http://www.fawcette.com/xmlmag/2002_11/online/xml_rjennings_11_11_02/); perhaps the assembly is later version of this assembly.Should be fairly robust given this is what the Wizard uses (inferred from the directories.)
- I did not investigate the DTD version of the generator.
Why would you use reuse this library? A couple of thoughts come to mind: when automating batch schema generation and when writing custom application adapters. Obviously, there are other resources available on the web (and within VS.NET) that can generate schemas from Xml instance documents but I assume that this library has been put through a rigorous QA process (although it is a sample according to this post http://groups-beta.google.com/group/microsoft.public.biztalk.sdk/browse_thread/thread/913c7fa57fc127a/2ac3418b674f5da8?q=ISchemaGenerator#2ac3418b674f5da8 [I wonder why it is in the utilities directory if it is a sample?]) given its usage in BizTalk Server 2004.
This post disappeared from a previous blog of mine so I'm reposting it.
---
Are you read to take your BizTalk Server 2004 application to production?
Integration applications are usually very complex; they have a lot of moving pieces, they use many different technologies and they are often built to automate processes that are not understand nearly as well as the business units involved think they are understood. In addition to the technical problems related to integration applications, integration problems have a corresponding set of issues centered around communication difficulties. Often these communication difficulties arise because the different teams involved in the project do not understand ant technologies other than their own, because people of various technical levels are involved in project and because project teams do not reside at the same physical location.
Because of these complexities it is easy to overlook items that should be handled prior to your BizTalk Server 2004 application going live. The article contains suggestions for items you should resolve prior to going live.
Deployment
- Is your deployment automated? Fully automated?
- Have you defined your rollback procedures?
Application Architecture Review
- Have you covered all of your error handling scenarios from you requirements?
- Will HAT provide all of the necessary debugging, process analysis and replay requirements for your application? If not you may need to
- Database recovery, backups, failover, etc.
- Logging - where, how, when?
- How are you going to monitor you application and environment? What happens after hours? Have you considered Microsoft Operations Manager?
- How are you going to implement notification? Who gets notified? When? Will you filter so that you don't send out notifications for each and every BizTalk related event log entry?
- Will you have enough information to debug your problems (they will occur!)?
- Can you really guarantee delivery? Did you design your orchestrations correctly?
- Have you reviewed the Performance and High-availability whitepapers?
Hardware Architecture Review
- Are your environments the same from Development to Production?
- If necessary, have you really eliminated single-points of failure? What about the resources BizTalk will depend on such as MQSeries server, databases, etc. You are as weak as you weakest point.
- Can you really meet you availability requirements?
- If necessary, do you have an environment that you can perform load and performance testing in? Do you have access to it? You will need access to it so that you can change tuning settings.
Operations Staff
- Do you know how to work with HAT?
- Will you have access to HAT in you production environment? If not, how will you work with your operations staff during issues?
- Have you trained your operations staff on HAT?
- Does your operations staff know how to use your deployment tool?
- Does your operations staff know how to use BizTalk Administration?
- Have you planned your situation handling procedures and documented them?
- Does operations understand all of your application scenarios?
- Does operations understand the service-level agreement for you application? For your platform?
Testing
- Have you tested failover scenarios?
- Can you really run your business scenarios in your pre-production environment? If not, be prepared for issues in production soon after you deploy there.
- Are your environments the same from Development to Production?
- For applications that will be under heavy load, when you PoC'ed did you test the PoC under a full load? You might be surprised at what happens if you only tested with a percentage of your full load and extrapolated.
- For applications that will be under heavy load, have you tested all of the adapters separately? They behave very differently under load and have different tuning settings.
- Have you tested all of you unhappy paths?
- Does everyone know what end-to-end testing means? Are you sure?
- Do you have testing schedule defined? This can be very difficult to plan given that integration applications often have many systems and departments involved.
Business
- Have you reminded the business units about the new process changes?
- Have you defined the communication paths between the involved business unit/parties when problems occur? This is also something that your operations staff will have to know about.
- Does the business know how/when they will be notified of issues; this includes automated notifications and SLA requirements for how the operations staff will notify customers about issue resolution?
This list of items is not exhaustive and there are likely many other things that you should consider for your particular situation prior to taking your BizTalk Server 2004 application to production; however, I have provided a sample of items that you may have not thought of and that will hopefully provide some motivation to discover other items for you particular situation.
If you have other suggestions for this list, let me know.
This post disappeared from a previous blog of mine so I'm reposting it.
---
A couple of weeks ago I wrote a short article about an ‘easy’ method of determining the correct BTSNTSvc.exe process to attach to in VS.NET in order to debug orchestration utility components. Scott Colestock described a better way to determine this here.
I liked his method much better than mine but I didn’t want to have Perfmon open whenever I needed to find the correct process id. I took his suggestion one step further and created a simple console application that writes out the process ids of all started, in-process host instances. Here are the steps to create (using C#) and use this simple application:
- Create a new .NET console application.
- Add a reference to System.Management.dll
- In the class that contains your main() method add using statements for:
- System.Diagnostics
- System.Management
- Add the following code to the main() method:
//Enumerate all HostInstances of "InProcess" type
try
{
//Create EnumerationOptions
EnumerationOptions enumOptions = new EnumerationOptions();
enumOptions.ReturnImmediately = false;
//Search for all HostInstances of 'InProcess' type in the Biztalk namespace scope
ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer","Select * from MSBTS_HostInstance where HostType=1 and ServiceState=4",enumOptions);
//Enumerate through the ManagementObjects and write out the Host Instance Process Ids.
foreach(ManagementObject inst in searchObject.Get())
{
string hostInstanceName = (string)inst["HostName"];
PerformanceCounter pc = new PerformanceCounter("BizTalk:Messaging", "ID Process", true);
pc.InstanceName = hostInstanceName;
float instanceId = pc.NextValue();
Console.WriteLine(hostInstanceName + ": " + instanceId);
}
}
catch(Exception ex)
{
Console.WriteLine(ex);
}
- Compile the project.
- Add a new External Tool to VS.NET
- Open VS.NET, go to Tools, External Tools.
- Click Add
- Give this tool a Title, I called it ‘BTSHostInstanceProcessIds’
- For the Command, browse to the .exe that you just compiled, above.
- Check the Use Output Window checkbox.
That’s it. Now when you need to know the process ids for your in-process host instances you can use this tool from the VS.NET Tools window. Execute it just prior to attaching the debugger. It will write out the host instance name (actually it is the host instance’s host name) and their process ids to the VS.NET Output window like this:
BizTalkServerApplication: 2856
TrackingHost: 4540
This post disappeared from a previous blog of mine so I'm reposting it.
---
One may get the impression, based on the example configuration listing for 'AppDomain configuration' in the BTS docs (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/ebiz_prog_orch_jzex.asp), that the Orchestration Engine's default values for SecondsIdleBeforeShutdown and SecondsEmptyBeforeShutdown are 1200 and 1800, respectively. The example configurations are so plagued with errors that I investigated a bit further.
In the Microsoft.XLANGs.BizTalk.CrossProcess.AppDomains constructor (in the Microsoft.XLANGs.BizTalk.CrossProcess assembly),
the default values are set to the reverse:
SecondsIdleBeforeShutdown = 1800 (30 minutes)
SecondsEmptyBeforeShutdown = 1200 (20 minutes)
Obviously these values could be reset to something other than their default values by code that executed after the constructor but it does appear that the default values are reversed in relation to the example's configuration.
In case you are not familiar with the settings here is a snip from the referenced page in the BizTalk docs:
"In this section the user may specify default configuration for any app domain created that does not have a named configuration associated with it (see AppDomainSpecs below) SecondsEmptyBeforeShutdown is the number of seconds that an app domain is empty (that is, it does not contain any orchestrations) before being unloaded. Specify -1 to signal that an app domain should never unload, even when empty. Similarly, SecondsIdleBeforeShutdown is the number of seconds that an app domain is idle (that is, it contains only dehydratable orchestrations) before being unloaded. Specify -1 to signal that an app domain should never unload when idle but not empty. When an idle but non-empty domain is shut down, all of the contained instances are dehydrated first."
I encourage you to review all of the Orchestration Engine settings on the referenced page; many of them are very useful.
This post disappeared from a previous blog of mine so I'm reposting it.
---
When using the xpath() function in the orchestration designer you may need to obtain the value from a single node (I'm distinguishing between trying to obtain the node vs. the node's value.) If you are attempting to do this, please don't forget to wrap your xpath statement with the xpath string function, for instance:
myStringValue = xpath(myMessage, "string(/*[local-name()='MyElement' and namespace-uri()='']/@*[local-name()='MyAttribute' and namespace-uri()=''])");
If you forget to do this your variable (in this case myStringValue) will have a value of null and depending on your usage of that variable later on in your orchestration you may end up with an exception.