BizTalk WCF Metadata Only MEX Endpoint Error: Root element is missing

In a BizTalk 2016 application, I have a receive location using the WCF-NetTcp adapter. The receive location is using an in-process Receive Handler and so I used the BizTalk WCF Service Publishing Wizard to publish a Service Metadata Endpoint (MEX) hosted in IIS so that clients can retrieve the WSDL of the web service exposed.
Note that typically, a BizTalk isolated host is an IIS Application Pool instance: w3wp.exe.

Endpoint error: Root element is missing

Once the MEX endpoint is deployed in an IIS application, I saw the following error when browsing to the endpoint:

Server Error in ‘xxx’ Application.
Root element is missing. 
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 
Exception Details: System.Xml.XmlException: Root element is missing.
Source Error: 
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Windows Event Log Analysis

When looking at my Windows Event Log, I saw the following error:

Server Error in ‘xxx’ Application.
WebHost failed to process a request.
Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/4032828
Exception: System.ServiceModel.ServiceActivationException: The service ‘xxx.svc’ cannot be activated due to an exception during compilation. The exception message is: Root element is missing.. —> System.Xml.XmlException: Root element is missing.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
….
Process Name: w3wp

I looked further down in my event log and saw a series of warnings from the Enterprise SSO service with messages of the likes:

SSO AUDIT
Function: GetConfigInfo ({CB480FD2-902B-4F1E-A2DB-43B3954A341B})
Tracking ID: 1c44a765-61a3-4679-901d-f1853fb2f497
Client Computer: BTS2016 (wmiprvse.exe:8432)
Client User: IIS APPPOOL\BizTalkIsolatedHostAppPool
Application Name: {315B6926-BF0C-462D-A8FD-5512F5E41456}
Error Code: 0x80070005, Access is denied.

And:

Access denied. The client user must be a member of one of the following accounts to perform this function.
SSO Administrators: SSO Administrators
SSO Affiliate Administrators: SSO Affiliate Administrators
Application Administrators: BizTalk Server Administrators
Application Users: –
Additional Data: IIS APPPOOL\BizTalkIsolatedHostAppPool {315B6926-BF0C-462D-A8FD-5512F5E41456} WCF-NetTcp_RL_BizTalkServerApplication_{315B6926-BF0C-462D-A8FD-5512F5E41456}

The first thing I noticed in the SSO warnings here above is that they refer to the user “IIS APPPOOL\BizTalkIsolatedHostAppPool” which in my case is the identity of the Application Pool running my WCF Service metadata endpoint.
In IIS 7.5 and above, each Application Pool is by default assigned it’s own virtual account and it’s named using the following pattern: “IIS AppPool\<ApplicationPoolName>” (i.e. this is the name of the virtual account in Windows). In the Application Pool setting, the virtual account name is simply referenced to by setting the identity property to “ApplicationPoolIdentity“.

I already had this account part of the BizTalk Isolated Host Users Windows Group but it seems to not be enough as Enterprise SSO is complaining.

At this stage I did some research and found a clue here but, for security reason, I was not satisfied with having the App Pool Service Account being part of BizTalk Server Administrators Windows Group.
I dug a little deeper and found this interesting blog post. This made me check the BizTalk documentation and it is indeed now documented that BizTalk Host Instance Accounts and BizTalk Isolated Host Instance Accounts must be part of the SSO Affiliate Administrators Windows Group.

Final Solution

Finally, what I did to solve the issue was to add the Application Pool identity to the SSO Affiliate Administrators Windows Group (as it is actually instructed in the BizTalk’s documentation!) and rebooted my machine.

Nevertheless, my personal opinion is that while it is the supported solution documented by Microsoft, the root of the problem lies in the implementation of ISSOConfigStore::GetConfigInfo and that Microsoft should fix that or change the way MEX endpoints retrieve data. Indeed, to me, it does not really make sense that an IIS App Pool should run under an account being an SSO Afiliate Administrator.

Note: I did not have to modify the web.config of the WCF Service MEX endpoint web service generated by the BizTalk WCF Service Publishing Wizard as was mentioned in one of the blog I referenced to:
<system.web>
    <trust level=”Full” originUrl=”” />
</system.web>

Failure Importing WCF Extension configuration to BizTalk WCF-Custom Handler

One of the BizTalk application that I am working on has to access web services from a third-party which does not play nicely with WCF. Nothing too uncommon if the third-party is a little exotic.
The solution was that a Custom Message Encoder was written to be able to read the messages coming from the third-party. To configure the WCF runtime stack, a Custom Message Encoding Binding Element was also created. As we want this custom encoder to only be used by that BizTalk application, we created a specific WCF-Custom Adapter Handler for the application and only that handler is configured with the binding extension configuration (instead of doing a change in machine.config or maybe the BTSNTSvc.exe.config).

As I was recreating a new development environment, I tried to import the WCF binding extension configuration file (a .config file containing a bindingElementExtensions element) to the WCF-Custom Adapter handler and got the following error: “Unable to import configuration from file “xxx.config”. (System.UnauthorizedAccessException) Access to path ‘xxx’ is denied.

WCF Extension configuration fails BizTalk Handler

I found out that this was simply because the configuration file was read-only (as it was coming from TFS source control) and that the BizTalk Admin Console first copies the config file to a temp folder and then tries to delete it. As the file is read-only, it cannot be deleted and importing the binding configuration fails!

The solution was simple, I just had to make the file writeable.
I chose to check-out the file temporarily, import the WCF binding extension configuration file and undo the check-out once the configuration was imported through the BizTalk Admin Console.

Notes:

If you are unfamiliar with WCF extensibility, a sample of a Custom Message Encoder sample is available on MSDN at Custom Message Encoder: Custom Text Encoder and the source code of the sample (and other samples) can be downloaded at Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF) Samples for .NET Framework 4.

To import WCF Extension to a WCF-Custom Adapter Handler, open the BizTalk Admin Console, navigate to BizTalk Group\Platforms Settings\Adapters\WCF-Custom and right click the Handler of choice. Click Properties and then click Import.

Import WCF Extension bindings for BizTalk WCF-Custom Adapter Handler