The case of the COM Port Redirection

SecutestOne of my colleagues asked me to assist in troubleshooting an application called SmartWare FM running on Citrix XenApp.

This application reads data from an external device called SECUTEST.

The device is connected to a COM port which is redirected to the XenApp session. In contrast to Microsoft Remote Desktop Services COM ports are not automatically redirected in XenApp but need to be mapped via eg a logonscript (NET USE COM1: \\Client\COM1:) or using UEM.

In my case the COM port was mapped with RES Workspace Manager:

image

I verified using net use that the com port is actually mapped:

image

The import in the application fails however with a generic error (Er heeft zich een fout voorgedaan):

image

I disconnected the XenApp session and ran Putty locally connected to COM1 9600-N-8-1.

image

Pressing enter echoes $24 (apparently some kind of protocol) so the communication between PC and device seems to work:

image

I then tried to connect with Putty (a lightweight terminal client)from within the XenApp session but got an immediate connection failure:

image

This message normally appears when a COM port is already in use so I opened Process Explorer and used Find | Find Handle or DLL and searched for COM1:

image

CdmRedirector is the Client Device Mapping redirector from Citrix so it appears that our COM port is already connected to the psp6164a.exe process.

psp6164a.exe belongs to Philips G2 Speech and seems to be responsible for communication to serial G2 Speech devices. The executable is launched from the HKLM run key:

SNAGHTML1ac2ce04

I knew that in this environment only USB connected G2 Speech devices are used but I wanted to check if there was some kind of commandline argument I could set to skip COM1.

I opened psp6164a.exe in Ida Pro and searched for COM1:

image

Using Ctrl-X (references) I ended up in this code:

image

Using the registry (HKLM\SOFTWARE\Philips Speech\Control Panel\6164) we can configure the active port and Port Search Order:

image

After reconfiguring I retested with Putty and the communication seemed to work from within the XenApp session:

image

However the application failed with the same error:

image

Knowing that the connection and protocol were working I monitored with Process Monitor. I usually focus on ACCESS DENIED errors by using the Filter | Highlight option to easily identify them:

image

There was just one:

image

Looks like the application tries to write a file to the Program Files folder (why do developers insist on doing this?).

We can of course give the users write permissions on this folder or perhaps even this file but what happens when multiple users try to do this at the same time?

A better solution is to use an Application Compatibility shim to redirect this file to a user specific and user writable location. I choose to redirect it to H:\Windows\Temp (H:\ is the users homefolder).

But we need to verify if there are any other files we need to redirect either by trial and error or search for related files.

I double-clicked the line that resulted in ACCESS DENIED in Process Monitor and switched to the Stack tab. There was only one file from the Applications folder in there called SW2_FM_SEC.dll:

image

I opened Process Explorer, clicked the Applications executable and set the Lower Pane View to DLLs. Then I selected SW2_FM_SEC.dll from the list and listed it’s properties, on the Strings tab we can see that besides IMPORT.SEC there is probably a file called ERROR.SEC:

image

So let’s create the redirect shim (if you are new to Application Compatibility read this article first).

Create a new database and the executable (SW2000.exe in my case):

image

Select the CorrectFilePath Shim and click Parameters:

image

Note that MSVBVM60.DLL is on stack (in the Process Monitor screenshot above) so we need to add MSVBVM60.DLL to the include modules (see here for an explanation).

The Command line is:

image

Save and install the database and now everything works correctly:

image

Of course you need to distribute and install the Shim to your production environment, more about that here.

Leave a Reply

  1. Kinda awesome. However, does also leave me with bewildered. I mean, it’s nice that you _can_ solve this.

    But it is really crappy that you have to, in the first place. Why don’t we switch to quality software that was actually written this century?

    Either that, or, why don’t we keep these applications on their intended platform? I mean, you are basically sewing together a backwards compatible virtual machine from various components (XenApp/RDP, RES manager, Application virtualization, (UAC?) virtualization, compatibility shims, what not).

    FrankenPC!

    Do we ever wonder why our software delivery fails to get robust?

    How many paid products were used in trying to make other paid products work as advertised?

    (Also, I wager that there would have been a decent chance of fixing the file access problems by having SW2000.EXE start from a safe working directory (there is a reasonable chance that it just accesses these files by relative paths).)

    Still awesome war story. Thanks for sharing, seriously. It does inspire me at least as much as it saddens me :)

  2. @sehe: thanks for your comment, it’s good to have a discussion about it so thanks for that!

    My reply below is not specifically about this application but more in general.

    I often call badly behaving applications “Crapplications” just to emphasize how badly written they are. I would like nothing more than to keep Crapplications out and replace them with better written software (from a technical view).

    As an external consultant I usually don’t have such powers, I am just brought in to make it work. In fact often the IT department doesn’t have such power since the business simply dictates IT what to do. And from the business perspective the Crapplication often perfectly supports their business process.

    Medical applications often required FDA approval (may take years) which is a big hurdle to take when deciding to rewrite the software.

    As for this application specific:
    I agree that it’s likely we can install this application into a different path, question is if that will really fix our problem. On a multi user system it still means that multiple users might write into this file at the same time!
    In a migration scenario it’s often not even possible to change paths since the configuration files (think ini’s on a share) are usually shared.

    Still I agree with you: I would really like to see the day where all of these skills are not required anymore :D

  3. Pingback: MSCOMM32.OCX returns error 80040112 - Remko Weijnen's Blog (Remko's Blog)

  4. Pingback: Application compatibility.. No longer an issue? • My Virtual Vision