Error configuring BizTalk Server 2006

Today I had quite some troubles installing BizTalk Server 2006. As I have previously installed BizTalk many times without being stuck on a problem for so long, I thought I would blog about it. Who knows, it might help some other soul encountering the same issue.

My configuration was the following:

• 1 Windows Server 2003 64-bit running SQL Server 2005.
• 1 Windows Server 2003 64-bit with BizTalk Server 2006 installed but not yet configured.

Both machines had MSDTC and COM+ installed and running, a pre-requisite to install BizTalk on a multiple-server environment.
All the BizTalk Users as well as the Windows Account used to install BizTalk had the necessary rights and belonged to the necessary Groups. See a previous post of mine for more information; Biztalk 2006 multi-server installation guide.

So, after having installed BizTalk Server 2006, the BizTalk Server needs to be configured by using the BizTalk Server Configuration tool and there the weirdest thing happened: it failed, but not completely! Enterprise SSO (Enterprise Single Sign-On) and Business Rules Engine (BRE) components were configured successfully while Group and BizTalk Runtime failed.
Thinking I made some human mistake in the configuration, I retried 2 times but the same problem happened. On a side note, before retrying to configure BizTalk, I had to manually delete the BizTalkMgmtDb as it was created even as the configuration step creating that DB failed.

I then read line by line the log file produced by the configuration tool and searched for the source of the problem. I will paste here the relevant parts:

[1:57:09 PM Error ConfigHelper] [DBNETLIB][ConnectionOpen (Connect()).]SQL Server does not exist or access denied.
[1:57:09 PM Error ConfigHelper] SQL error: 08001 Native error code: 17
[1:57:09 PM Error ConfigHelper] c:\depotsetupv2\private\common\configwizard\confighelper\sqlhelper.cpp(1176): FAILED hr = 80004005
[1:57:09 PM Error ConfigHelper] c:\depotsetupv2\private\common\configwizard\confighelper\sqlhelper.cpp(918): FAILED hr = 80004005
[1:57:09 PM Info ConfigHelper] Did not find existing database: BizTalkMgmtDb on computer:
[1:57:09 PM Info RuleEngineConfig] Management database (server = , database = BizTalkMgmtDb) doesn’t exist

With this information in hand, I suspected something was wrong with the credentials and I checked all the Users and Groups in Active Directory and also checked that I was admin of the SQL Server machine but everything looked ok. I nevertheless re-tried the configuration a few times, being extra careful with every setting but unsuccessfully. I then moved on to next problem in the log file.

[2:08:41 PM Error ConfigHelper] c:\depotsetupv2\private\common\configwizard\confighelper\service.cpp(729): FAILED hr = 80070421
[2:08:41 PM Warning ConfigHelper] The account name is invalid or does not exist, or the password is invalid for the account name specified.
[2:08:41 PM Warning ConfigHelper] Failed to validate service credentials for account: %1
[2:08:41 PM Info BtsCfg] Check Your NT Account Specification

Here the configuration log complains about Windows Credential issues again. Everything was pointing out to Active Directory, this would reveal to actually point me in the wrong direction. So, as I could still not find any problem in AD, I moved on to the next problem I could find in the log:

2008-09-12 14:13:16:0075 [INFO] WMI Deploying ‘C:\Program Files (x86)\Microsoft BizTalk Server 2006\Microsoft.BizTalk.GlobalPropertySchemas.dll’
2008-09-12 14:13:22:0184 [WARN] AdminLib GetBTSMessage: hrErr=80070002; Msg=The system cannot find the file specified.;
2008-09-12 14:13:22:0184 [WARN] AdminLib GetBTSMessage: hrErr=c0c02560; Msg=Failed to read “KeepDbDebugKey” from the registry.
The system cannot find the file specified.;
2008-09-12 14:13:22:0184 [INFO] WMI Error occurred during database creation; attempt to rollback and delete the partially created database’ny-agbdb-301\BizTalkMsgBoxDb’

Two points here:

• The file “T:\Program Files (x86)\Microsoft BizTalk Server 2006\Microsoft.BizTalk.GlobalPropertySchemas.dll” was present on the file system so it didn’t seem there was anything I could do about it.
• The key “KeepDbDebugKey” did not exist in the registry of running healthy BizTalk systems so I supposed this registry key exists only temporarily during the BizTalk configuration procedure.

I then moved on the next potential problem in the log file:

2008-09-12 14:14:02:0465 [ERR] WMI Failed in pAdmInst->Create() in CWMIInstProv::PutInstance(). HR=c0c025b3
2008-09-12 14:14:02:0465 [ERR] WMI WMI error description is generated: Exception of type ‘System.EnterpriseServices.TransactionProxyException’ was thrown.

Here the configuration tool log entry complains about a Transaction from the System.EnterpriseServices library. As this package refers to Distributed Transactions (MS DTC), I doubled checked that MS DTC was running well on both machines, checked firewall configurations but all the necessary ports were open.
I then started to try to troubleshoot DTC following the MSDTC troubleshooting guide from Microsoft and found a utility called dtctester that can be downloaded from the KB 293799.

The dtctester test failed and so I was now sure that the problem was related with DTC. I re-checked the firewall settings, checked the dual network adapter and the DNS as advised by Microsoft and found out that the problem lied with the DNS having some wrong reverse mappings (IP address -> hostname mappings).

As a corollary, notice that the only log information that proved to point me to the actual source of the problem were log entries with the string [ERR] in their header. All others had either [INFO] or [WARN].

To conclude, I would advise to:

• Use dtctester before operating a BizTalk multi-server installation.
• When running into problems, look first at the log entries having either the [ERR] or Error string in their header – It indeed seems there are two different formats of log entry header in the log file produced by the Configuration tool. While looking first for the Error type of log entries sounds obvious, it seems that not many people are aware of the log entry header formats and the log entry types generated in the log file.