One of my colleagues asked me to assist in troubleshooting an application called SmartWare FM running on Citrix XenApp.

This application reads data from an external device called SECUTEST.

The device is connected to a COM port which is redirected to the XenApp session. In contrast to Microsoft Remote Desktop Services COM ports are not automatically redirected in XenApp but need to be mapped via eg a logonscript (NET USE COM1: \\Client\COM1:) or using UEM.

In my case the COM port was mapped with RES Workspace Manager:

I verified using net use that the com port is actually mapped:

The import in the application fails however with a generic error (Er heeft zich een fout voorgedaan):

I disconnected the XenApp session and ran Putty locally connected to COM1 9600-N-8-1.

Pressing enter echoes $24 (apparently some kind of protocol) so the communication between PC and device seems to work:

I then tried to connect with Putty (a lightweight terminal client)from within the XenApp session but got an immediate connection failure:

This message normally appears when a COM port is already in use so I opened Process Explorer and used Find | Find Handle or DLL and searched for COM1:

CdmRedirector is the Client Device Mapping redirector from Citrix so it appears that our COM port is already connected to the psp6164a.exe process.

psp6164a.exe belongs to Philips G2 Speech and seems to be responsible for communication to serial G2 Speech devices. The executable is launched from the HKLM run key:

I knew that in this environment only USB connected G2 Speech devices are used but I wanted to check if there was some kind of commandline argument I could set to skip COM1.

I opened psp6164a.exe in Ida Pro and searched for COM1:

Using Ctrl-X (references) I ended up in this code:

Using the registry (HKLM\SOFTWARE\Philips Speech\Control Panel\6164) we can configure the active port and Port Search Order:

After reconfiguring I retested with Putty and the communication seemed to work from within the XenApp session:

However the application failed with the same error:

Knowing that the connection and protocol were working I monitored with Process Monitor. I usually focus on ACCESS DENIED errors by using the Filter | Highlight option to easily identify them:

There was just one:

Looks like the application tries to write a file to the Program Files folder (why do developers insist on doing this?).

We can of course give the users write permissions on this folder or perhaps even this file but what happens when multiple users try to do this at the same time?

A better solution is to use an Application Compatibility shim to redirect this file to a user specific and user writable location. I choose to redirect it to H:\Windows\Temp (H:\ is the users homefolder).

But we need to verify if there are any other files we need to redirect either by trial and error or search for related files.

I double-clicked the line that resulted in ACCESS DENIED in Process Monitor and switched to the Stack tab. There was only one file from the Applications folder in there called SW2_FM_SEC.dll:

I opened Process Explorer, clicked the Applications executable and set the Lower Pane View to DLLs. Then I selected SW2_FM_SEC.dll from the list and listed it’s properties, on the Strings tab we can see that besides IMPORT.SEC there is probably a file called ERROR.SEC:

So let’s create the redirect shim (if you are new to Application Compatibility read this article first).

Create a new database and the executable (SW2000.exe in my case):

Select the CorrectFilePath Shim and click Parameters:

Note that MSVBVM60.DLL is on stack (in the Process Monitor screenshot above) so we need to add MSVBVM60.DLL to the include modules (see here for an explanation).

The Command line is:

Save and install the database and now everything works correctly:

Of course you need to distribute and install the Shim to your production environment, more about that here.