Accessing 64-bit Native COM Server From a Managed Client

Not long ago I ran into a COM interop problem that was a bit tricky to fix. So I’m sharing the problem and the solution here in case others encounter the same problem.

I had this native in-proc COM server that initially was built only for x86. It was used in a native MFC application as well as a C# Windows Forms application, where it was added as a COM reference. Both worked nicely. But then I needed to port the MFC app to the x64 platform, so I had to do the same with the in-proc COM server. They both worked correctly, but the managed app, which had to be available also both as 64-bit and 32-bit (even on 64-bit machines), was broken. Eventually I traced the problem to some COM method calls which were a bit atypical, because the arguments to the methods were not COM “native” types, but custom structures.

These structures looked like this:

[uuid(6F13C84D-0E01-48cd-BFD4-F7071A32B49F)] struct S
{
      long a;
      BSTR b;
      long c;
      BSTR d;
      long e;
      BSTR f;
      BSTR g;
      BSTR h;
      BSTR i;
      long j;
      BSTR k;
      long l;
      BSTR m;
      long n;
};

and the COM method signature:

[id(42)] HRESULT GetListOfStructs(SAFEARRAY(struct S)* arrRes);

When you add a COM reference to a .NET assembly, Visual Studio runs tlbimp.exe to generate a .NET assembly with equivalent definitions to the type definitions found in the COM type library. This interop assembly is used to access the COM server. It contains Runtime Callable Wrappers that bridge the two worlds together. By default, the interop assembly, generated in the project’s output folder, is called Interop.<comname>Lib.dll. For instance if the COM server is called NativeCOMServer, the interop assembly is called Interop.NativeCOMServerLib.dll.

The wrapper that was generated in the interop assembly had the following signature for the aforementioned method:

[DispId(42)]
void GetListOfStructs(ref Array arrRes);

and therefore used like this:

System.Array result = null;
obj.GetListOfStructs(ref result);

The call was performed correctly, the native code was executing, but as soon as it was returning an access violation exception was occurring. Thanks to Hans Passant I figured the problem was rooted in the way Visual Studio generates the interop assembly. The generated RCW didn’t know how to handle the custom structure correctly. Probably different padding on the two sides generated the access violation.

The trick was to generate the interop assembly directly, as a custom build step in the COM server project, and include it as an assembly reference in the managed project. Here are the commands for the custom build (must to make sure you have the correct path to the 32-bit and the 64-bit version of tlbimp.exe):

\TlbImp.exe $(TargetPath) /out:$(TargetDir)\NativeCOMLib.Interop.dll /primary  /keyfile:mykey.snk /machine:x86

\TlbImp.exe $(TargetPath) /out:$(TargetDir)\NativeCOMLib.Interop.dll /primary  /keyfile:mykey.snk /machine:x64

The result was a wrapper with methods like this:

[DispId(42)]
void GetListOfStructs(ref S[] arrRes);

which of course had to be called like this:

S[] result = null;
obj.GetListOfStructs(ref result);

To include either one or the other in the C# project I had to manually change the project file, since Visual Studio does not know conditional references, a feature available in MSBuild. I didn’t build the managed app for Any CPU because it had to be able to run as 32-bit on 64-bit machines, so I had two configurations, one for x86 and one fore x64.

<Reference Condition=" '$(Platform)' == 'x86' " Include="NativeCOMServer.Interop, 
               Version=1.0.0.0, Culture=neutral, PublicKeyToken=f5b9312191a42d52, processorArchitecture=x86">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\Bin\NativeCOMServer.Interop.dll</HintPath>
    </Reference>
    <Reference Condition=" '$(Platform)' == 'x64' " Include="NativeCOMServer.Interop, 
               Version=1.0.0.0, Culture=neutral, PublicKeyToken=f5b9312191a42d52, processorArchitecture=x64">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\Bin64\NativeCOMServer.Interop.dll</HintPath>
    </Reference>

But this time the wrappers were able to bridge the call and everything worked smoothly.

The lesson learned is that when you have COM custom structures you should not rely on the way Visual Studio generates the interop assembly. You should build the interop explicitly (maybe as a custom build step, like I did) and include it as an assembly reference to your managed project.

CodeProject