Everything has a reason

A time ago, I wrote about how you could only read GPS data from a USB GPS device from one COM port and how I managed to handle GPS data on different virtual COM ports so that the GPS data could be used in different applications.

One of these applications is one in Java that I develop myself. However, the RxTx driver I’m using is a little unstable, especially when the connection is lost and you try to re-establish the connection to the Serial Port. So I tried to find a solution and hub4com actually offers one as well. You can actually route the GPS data, read from a physical COM port, to a tcp port so that you can create a network connection to the machine and fetch the GPS data from there.

You can enable filters in hub4com and one of these filters is the telnet filter over a tcp driver. I modified the command so that it looks like this:

hub4com
    --baud=9600
    --route=0:All \\.\COM6
    --baud=9600 \\.\CNCA0
    --baud=9600 \\.\CNCA1
    --baud=9600 \\.\CNCA2
    --create-filter=telnet
    --add-filters=0:telnet
    --use-driver=tcp *23 *23 *23 *23"

This now means the following:
Start hub4com,
--baud=9600
read at speed 9600 baud,

--route=0:all
route the first specified COM port to all the others

\\.\COM6
the first specified COM port, which is the one that we’ll be reading from

--baud=9600 \\.\CNCAx
write to these virtual COM ports with speed 9600 (the –baud parameter on these three lines might be omitted since you have specified hub4com to read at that speed)

--create-filter=telnet
Create a telnet-filter so that hub4com can answer telnet-like

--add-filters=0:telnet
redirect the input from the first specified COM port (COM6 in our case) to the telnet session

--use-driver tcp *23 *23 *23 *23
this will use tcp for communication and will listen on port 23. Four concurrent telnet sessions on port 23 are possible. If you only specify one *23, only one connection at the same time will be allowed.

This now allows me to create a tcp connection on port 23 and read the NMEA data from there. Much more stable.
Now, to wrap this all up into a windows service, I run the following bat file (everything on one line) to install that service:

@RunAsSvc.exe
    --install
    --displayname "hub4com"
    --description "Routes the GPS data from COM6 to virtual COM14, COM15 and COM16 and allows 4 telnet connections to the raw NMEA data"
    --exe "C:\com0com\hub4com.exe"
    --params "--baud=9600 --route=0:All \\.\COM6 --baud=9600 \\.\CNCA0 --baud=9600 \\.\CNCA1 --baud=9600 \\.\CNCA2 --create-filter=telnet --add-filters=0:telnet --use-driver=tcp *23 *23 *23 *23"
    --workingdir "C:\com0com"

The service now launches on system startup and I can use multiple COM ports as well as 4 concurrent telnet sessions to the host on which the GPS is connected. telnet localhost is more stable to use in Java than RxTx (at the moment of writing)

Het was een winderige zondagnamiddag, zo één van die lazy afternoons waarbij je gewoon wat achter je computer zit te zitten en doet alsof je hard aan het werken bent, maar waarbij je je eigenlijk dood verveelt.

Dat is tot ik op het lumineuze idee kwam om eens een eerste keer te gaan geocachen. Zowel op het werk als thuis had ik er pas over gesproken en we vonden het wel een leuk idee. Dus, we namen onze jassen en dikke truien, muts op en gewapend met pen en Desire HD gingen we op pad. Om te geocachen heb ik twee programma’s gebruikt: c:geo en locus.

c:geo is een geocaching programma dat via de website geocaching.com werkt. Met de Live map functie zocht ik naar caches die in de buurt liggen en blijkbaar zijn het er nogal wat. Uit de vele caches zocht ik er één uit die snel bereikbaar was te voet en gemakkelijk te vinden zou zijn. Het is nog steeds onze eerste cache en omdat het een wild idee was op een namiddag, hebben we geen hele dag meer om te gaan wandelen. Ik koos Cache boost 9, De vaart op minder dan twee kilometer en downloadde het naar de telefoon zodat ik onderweg aan de informatie kon. Een andere functie die het programma heeft, is de cache te sturen naar een extern programma voor maps en dat is locus.

Locus is op dit moment mijn favoriete offline maps app. Het kan online kaarten in cache houden zodat je ze offline kunt bekijken, bijvoorbeeld tijdens het geocachen. Ook kun je het gebruiken om je trip op te slaan en heeft vele andere functies aan boord zoals het markeren van de locatie waar je je auto hebt geparkeerd en toont vervolgens hoe lang je die daar al hebt staan en wanneer je die terug moet oppikken omdat de parkeermeter verloopt. Die laatste functie heb ik nu niet gebruikt, omdat we de hele tocht te voet hebben afgelegd, dus ik heb het enkel gebruikt om de cache te localiseren en om onze toch op te nemen.

Toon Eerste Geocache in een groter formaat.

Koud en winderig, dat was het wel, maar doordat de regen niet van de partij was, is het een heerlijke tocht geworden. Wandelen is iets dat we echt te weinig doen. De cache hebben we snel gevonden, maar het was best tricky om onze naam op het papiertje te schrijven, gezien dat vochtig was en de wind zo erg blies dat het bijna uit mijn handen en in het water werd gesleurd.

Onze eerste cache is gelogd. En smaakt naar meer…

Given this situation: You have a GPS device that is connected to your computer (not handheld) via physical COM1. Now you want to use this data in two different applications, say: a Route Navigator and a GPS logger. You launch the Route Navigator, it’ll connect to the COM1 port and it’ll show you your position. Now, launch your GPS logger and try to connect on COM1. It’s not happening: that COM port is in use. You can connect to a COM port with only one application.

On a portable device, this is solved by installing a virtual COM port, which can be read by multiple applications simultaneously. Often, this is COM8 (so it is on my Windows Mobile 5 system). But what to do when you want this functionality on a normal pc?

XPort

Well, there is a very cool application that actually makes this possible. It’s called XPort. Just launch the application, hit “Find GPS”, select a virtual COM port in the right column and hit “Enable ports”. Now, you can connect with both applications to that selected virtual COM port. Or, you can select two virtual COM ports and have both applications connected on a different port.

There’s a little drawback. The application needs Administrative rights, so on your own computer at home it’ll work, but if you have to set this thing up on a terminal build in a truck and running Windows XP on which the GPS logger starts up silently and the Route Navigator launches in full screen, all automatically while a very limited user is logged on, XPort will be denied to launch the virtual COM ports since it should be run with administrative rights.

com0com

So we need an application that makes it possible to install the virtual COM ports as an administrator, then launch the mapping as a windows service.
I found com0com.
Using the application com0com, you can create virtual COM port pairs, each representing a virtual NULL Modem Cable. You can use the graphical interface, setupg or the commandline interface setupc. This last one can be used in scripts. I created three pairs, one for the Route Navigator, one for my software and one for later usage supposing that I might want to debug one day while the other apps are running. So my script looks like this:

setupc install 0 - PortName=COM14,EmuBR=yes
setupc change CNCA0 EmuBR=yes
setupc install 1 - PortName=COM15,EmuBR=yes
setupc change CNCA1 EmuBR=yes
setupc install 2 - PortName=COM16,EmuBR=yes
setupc change CNCA2 EmuBR=yes

This will create the following pairs:

CNCA0 <-> COM14
CNCA1 <-> COM15
CNCA2 <-> COM16

hub4com

Part of the project is hub4com that makes it possible to get the input from one COM port an route it to several others. The first example they give, is exactly that what I need.

You have a GPS device that connected to your computer via a phisical COM1 port and you’d like to handle its data by two GPS applications. You can do it this way:[..]

Following the rest of the example, while using COM6 as GPS COM port, CNCAx as virtual input, COM1x as virtual output, it showed me this on the commandline:

C:\com0com\>hub4com \\.\COM6 \\.\CNCA0 \\.\CNCA1 \\.\CNCA2
Open("\\.\COM6", baud=19200, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA0", baud=19200, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA1", baud=19200, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA2", baud=19200, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Route data COM6(0) --> CNCA0(1) CNCA1(2) CNCA2(2)
Route data CNCA0(1) --> COM6(0)
Route flow control COM6(0) --> CNCA0(1)
Route flow control CNCA0(1) --> COM6(0)
Started COM7(0)
Started CNCA0(1)
Started CNCA1(2)

When I now connect the GPS logger on COM5, I see a lot of trash appearing. The reason is that the baud rate doesn’t match. It’s very important to read from the COM port at the same rate as the device produces it’s data. The GPS receiver in my case has a speed of 9600 Bd, but as you can see in the console log, hub4com is reading it at 19200 Bd. Too fast, thus the trash.

Of course, hub4com provides a way to set the baud rate, option --baud=9600 is bringing it down and the output is now

C:\com0com>hub4com --baud=9600 \\.\COM6 \\.\CNCA0 \\.\CNCA1 \\.\CNCA2
Open("\\.\COM6", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA0", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA1", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA2", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Route data COM6(0) --> CNCA0(1) CNCA1(2) CNCA2(3)
Route data CNCA0(1) --> COM6(0)
Route flow control COM6(0) --> CNCA0(1)
Route flow control CNCA0(1) --> COM6(0)
Started COM6(0)
Started CNCA0(1)
Started CNCA1(2)
Started CNCA2(3)

Maybe it’s just me, but it took me a full day to understand what is happening now after reading and re-reading all information I could find. Launching the Route Navigator on COM14 will give you a GPS signal and launching the GPS logger on COM15 or COM16 will give you a signal as well. So far so good. Until you close the application on COM14. It doesn’t matter which one that is, Route Navigator or GPS logger, it all gives the same result. When the application on COM14 is closed, the signal is lost on COM14 and COM15. Although the configuration of all 3 ports are identically, the hub4com ReadMe states (adjusted to fit my example) that

It will send data received from COM6 port to CNCA0, CNCA1 and CNCA2 ports
and it will send data received from CNCA0 port to COM6 port.

As seen on the commandline:

Route data COM6(0) --> CNCA0(1) CNCA1(2) CNCA2(3)
Route data CNCA0(1) --> COM6(0)
Route flow control COM6(0) --> CNCA0(1)
Route flow control CNCA0(1) --> COM6(0)

The data is routed to all three COM ports, the “flow control” is applied from COM6 to CNCA0 and CNCA0 to COM6 (with CNCA0 the input for COM14). No flow control for the others is configured. So my only guess was to create a flow control to all others as well. One of the extended examples (got by issuing hub4com --help) stated this

  hub4com --route=All:All \\.\CNCB0 \\.\CNCB1 \\.\CNCB2
    - receive data from CNCB0 and send it to CNCB1 and CNCB2,
      receive data from CNCB1 and send it to CNCB0 and CNCB2,
      receive data from CNCB2 and send it to CNCB0 and CNCB1.

So that’s almost what I wanted to do. Assuming the syntax is consistent with, say, connect:

  --connect=<lstr>:<lstl>
      handle data sent to any port listed in <LstR>
      as data received by all ports listed in <LstL>.

(where I would have switched R and L by the way) and where --route=All:All says that "ports can talk to each other", I added the parameter --route=0:All to route all data always from 0 to the others, so from COM6 to COM14, COM15 and COM16:

C:\com0com>hub4com --baud=9600 --route=0:All \\.\COM6 \\.\CNCA0 \\.\CNCA1 \\.\CNCA2
Open("\\.\COM6", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA0", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA1", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Open("\\.\CNCA2", baud=9600, data=8, parity=no, stop=1, octs=on, odsr=off, ox=off, ix=off, idsr=off, ito=0) - OK
Route data COM6(0) --> CNCA0(1) CNCA1(2) CNCA2(3)
No route for flow control
Started COM6(0)
Started CNCA0(1)
Started CNCA1(2)
Started CNCA2(3)

With this setup the data always flows from COM6 to the others no matter which port is read or left unread.
It took me about four days to find out how com0com and hub4com work, mainly because there’s not much information about it. There are quite a few examples, but none of them are explained in detail.

I sure hope the developers are still maintaining this application, since it really does what it has to and works like a charm. I hope the lack of updates since December 2009 is just because the application works perfectly

My final command is this:

hub4com
   --baud=9600 --route=0:All \\.\COM11
   --baud=9600 \\.\CNCA0
   --baud=9600 \\.\CNCA1
   --baud=9600 \\.\CNCA2

I added the --baud=9600 to all other Null cable entrances, but that’s more a leftover from my tests. I think it’s sufficient to add the parameter before the list of com ports.

Install as Windows Service

Once configured, the Virtual COM ports stay active until you remove them. However, you have to launch hub4com every time you have rebooted the computer. To solve that, I’ve used runassvc, which allows a silent install as well. Without parameters, the application launches a GUI in which you can enter the required values. With parameters, like opened through a bat file, it launches and enters the parameters you have defined. Then, when you use --quiet as parameter, the GUI doesn’t launch, but the service is installed and started. My command looks like this

RunAsSvc.exe
   --install
   --displayname "hub4com"
   --description "Routes the GPS data from COM6 to virtual COM14, COM15 and COM16"
   --exe "C:\com0com\hub4com.exe"
   --params "--baud=9600 --route=0:All \\.\COM6 --baud=9600 \\.\CNCA0 --baud=9600 \\.\CNCA1 --baud=9600 \\.\CNCA2"
   --workingdir "C:\com0com"
   --quiet

Now, even when the computer is rebooted, as soon as it has started and before any user has logged on, the system is running so every application is able to read one GPS device even if another is already accessing it. It was tested successfully on a Windows XP and even on a Windows 7 machine, although on the last one you have to boot the machine in Test configuration (bcdedit -set TESTSIGNING ON) otherwise the com0com drivers won’t load.

Used applications

The applications I used to set up this environment, are

• com0com (local x86 copy or local x64 copy)
• hub4com (local copy)
• runassvc (local copy)

Further reading

It is also possible to reroute the GPS data to a tcp port. Read Send GPS data over TCP how to do this.

Terwijl ik dit schrijf, vraag ik me af… Ik ben mezelf aan het voorbereiden op een nieuw restaurant – of iets dat daarvoor moet doorgaan – en ik wilde er te voet naartoe gaan. NXNW (North by NorthWest) is hier vlakbij. Maar tot mijn grote verbazing blijkt dat er gewoon geen voetpaden zijn?! Er lopen hier trouwens ook gen voetgangers rond. Dan zal ik dus toch maar gewoon de wagen nemen zeker? Grmbl…

…

Bij NXNW was er geen plaats meer. Dus had ik nog een andere optie: naar The Domain gaan, daar hebben ze een hele boel restaurants. Naar het schijnt. Ik heb ze niet gevonden. Ik ben dan maar wat rond beginnen rijden en nog wat rond beginnen rijden en… uiteindelijk ben ik bij een Mc Donalds terecht gekomen en heb gauw iets naar binnen gespeeld. Grmbl. Ik denk dat wanneer ik iets deftig heb gevonden, niet te ver weg, dat ik daar een aantal keren naartoe zal gaan.

Eerlijk? Het was een goed idee om gewoon op te nemen waar ik aan het rijden was. Blijkbaar ben ik gewoon ergens verkeerd ingedraaid – zoals op de satellietbeelden te zien is. Ik moest langs de andere kant van de weg zijn. Wel, dat weet ik dan weer voor morgen. Zal ik nog eens een poging doen om ofwel bij NXNW te gaan of anders door te rijden naar een van de restaurants op The Domain… Hopelijk vind ik dan wel iets. Hopelijk rij ik dan wel goed.

Vandaag Luc naar de luchthaven gedaan. Het verkeer viel heel goed mee, waardoor ik al om half negen bij Dell aangekomen ben. Mijn vliegtuigticket is verlengd, de termijn van de huurauto is verlengd, het verblijf in het hotel is verlengd… Ik blijf nog even.

Having a colleague who has bought a geotagging device, and one who is thinking about doing the same, is the best way to get interested too. Asking colleagues how they geotag their imagese, is not quite a success, since they can only tell which programs to use on a W*ndows PC. Since I don’t use crap, I have to search a way to do this on a Mac. My Equipment: an HP iPaq 2490, a tomtom bluetooth GPS receiver, Visual GPSce to record my track in NMEA data and a camera from which the clock is in sync with my PDA…

In this post, I am explaining how to record the data, I assume you have your RAW NMEA data file available.

NMEA or GPX?

Although the NMEA gps data is a standard, not all programs support that format. GPX is the format which is used by most programs and thus recommended. Now how do you convert your raw data to GPX? There is only one program I know with which you can do this, GPS Babel. This is a command line tool and thus not really user friendly for most users but luckily there are enough GUI’s written on top of this tool.

1. GPS Visualizer, a website which supports multiple formats
2. Houdah GPS, a Mac OS X frontend to GPS Babel.

Houdah GPS

I’m using Houdah GPS because I might not be online while I geotag my photo’s and this way, I don’t have to upload my geo data to the Internet. Using Houdah GPS is quite simple:

1. Select your source Because I have an NMEA file available (recorded on my PDA), I select “File” as source and NMEA as format. I had to rename the extension that VisualGPSce has created: the extension is .txt but HoudahGPS only takes .nmea files but an .nmea file is nothing more than a plain text file, thus renaming the .txt to a .nmea file is sufficient.
2. Select your data I just leave the default: Both tracks and Waypoints.
3. Select your Destination data NMEA is the one we already have, so we don’t have to select that as destination. KML can be used in Google Earth, and as I said before: GPX is a standardized file format, so we’re going to create a GPX file. This can be used in google earth as well, by the way.
4. Convert Now that all the preferences are set, click the Convert button in the left upper corner. A File Selection dialog will appear for a first time. Here, you have to select your .nmea file (remember, it’s just the .txt file that you have renamed). Click "Open". A second File Selection dialog will appear in which you can define the name of your gpx file. Choose a name and click "Save"

Geotag

With our gpx file ready, we now have to put the geo data in our photos. There are multiple programs available to do this.

1. houdahGEO, from the makers of houdahGPS. This is share ware and costs $25.
2. Photo GPS Editor, donnationware.
3. GPS Photo Linker, freeware and my favourite.

The two last applications doesn’t seem to be maintained anymore, but they do what they have to do, so right now, I don’t pay attention to that.

GPS Photo Linker

In GPS Photo Linker, click the button "Load Tracks" in the left upper corner and select your GPX file. This may take some time, depending on how many data you have recorded. Next, click "Load Photos" in the upper right corner and select the photos you want to geotag. Not only JPEG is supported, but RAW images are supported as well. When all the images are loaded, go to the Batch tab in the left pane (not Standard or Manual) and click "Batch save to photos". This may take some time too, depending on how many data you have recorded an how many images you have selected.

And that’s basically it. Now your photos are geotagged and Flickr or Picasaweb pin your picture on the earth! But remember: make sure your camera and your recording device have their clocks synced, or else your images won’t be pinned correctly! With GPS Photo Linker, you can shift the time of the photos in order to have them geotagged correctly anyway.