Location-aware video recording (Quicktime, 9.5 Mb)

Trans-Cultural Mapping workshop, Helsinki

This project has made a fusion of a global positioning system and a recorded moving camera video image.

Any recorded sequence creates a simultaneous and parallel record of the exact GPS location where it was shot, which is fixed on the screen.

Pall Thayer Sara Kolster Pete Gomes	Links: Pete Gomes on Geo-Cinema Pure Data CPAN: Perl+Modules PDP Locative Media Workshops
	Files: geocinema.tar.gz

This project was formulated and realized in less than a week so it's basically at a proof-of-concept stage.
UPDATE: we did some more work on this at the Iceland inside and out workshop (summer 2004) and have updated the downloadable files above. We now use NMEA instead of GRMN, which works fine on all platforms and have included a colored strip for the text to appear on, making it more readable. So at this point, it works well on Mac OS X but I'm unsure about video capture on Linux (see below).

Initiation:

On the first day of the workshop, we had a session where each of the participants introduced themselves. Pete Gomes, artist/filmmaker, mentioned that he would be interested in being able to present GPS coordinate information directly on a video image. [1] One of my interests in the realm of locative/located media, is developing innovative, (relatively) easy-to-use tools for artists so this hit me as a sort of challenge requiring a response.

Approach:

I already had a Perl script reading data from my GPS device, so my initial approach was to modify this script so that it would also create a sequence of images simply containing a blue background (for chroma-keying) and the GPS coordinates. These could then be super-imposed on to the video afterwards. Quite time consuming and not necessarily very accurate. I realized that for true accuracy, the super-imposing of coordinates would have to happen in real-time and on-the-fly. I happened to know that Sara Kolster, artist and another workshop participant, has been experimenting with the pdp video-capture objects in Pure Data. I approached Sara and asked her if there was an object in pdp for super-imposing text on to the captured video and the answer was “Yes.” One of the really nice things about Pure Data is the possibility of implementing socket communications very easily using the netsend and netreceive objects. Such implementations are of course also quite simple in Perl so now my new approach was to use Perl to read the GPS data and send it via a socket connection to a Pure Data patch, super-impose the text onto the video and capture this in real-time to a laptop’s harddrive using the pdp objects.

Hurdles:

The only real problem we encountered at this stage was that the pdp_text object didn’t appear to know what to do with the messages it was receiving from the netreceive object. We emailed Yves Degoyon, the maintainer of pidip for pdp and asked for suggestions. He promptly replied and explained that we had to use the list2symbol object from the Zexy library to convert the collection of symbols into one. Another problem we ran into was that the text appeared simply as rectangles instead of letters which indicated a font problem. This was fixed by simply selecting a different font.

The last problem was most likely due to a faulty Perl installation on Mac OS X on Sara’s powerbook. We were unable to get the script that read data from the GPS device working properly. It appeared to work but the coordinates it was showing didn’t make any sense at all and were not the same as what the actual device showed. This was not a problem in Linux. If we would have had more time, I’m sure that we would have figured out the problem and been able to do the test-run on a single laptop.

um... just found out that pdp on Linux doesn't support firewire DV capture (there's a workaround using GEM), however I will be working on getting my Mac to read the GPS stuff correctly soon. Sure would be nice though to do the whole thing on a Linux machine.

About the video sample (requires Quicktime 3.0+):

(Quality has been greatly reduced for the web. I guess we should have had a darkened field behind the text. We'll remember that next time.)

The test movie simply shows us walking around the area outside the MUU Gallery in Helsinki, using the software. However, the video makes things look a lot more complicated than they actually are. In the video we are walking around holding two open laptops with an ethernet cable connecting them. The main reason for this is that my home-made Garmin cable was a total hack, put together in about an hour with lots of duct tape and no soldering. For the cable to work properly I had to apply physical pressure to it and if at any point the connection was lost (due to lack of pressure) the Perl script would stop running. So the only reason the laptops were open was so that we could monitor whether or not the script was running (there is one spot where it stops briefly, see if you can spot it). We also didn’t have enough time to figure out how we could close a Powerbook without it going into sleep mode. We used two laptops simply because of the problem with the script on OS X. We had to run the script on my laptop in Linux but couldn’t do the video capture on my laptop because I don’t have firewire. I think the ideal way of doing this would be to run it on a laptop with firewire running Linux. However, if you can get the Perl script to work correctly on OS X you can just as well use a Mac. As far as Windows goes, I haven’t been able to get Perl to read from my GPS device at all on Windows. It appears to send messages to the device but the device won’t respond, also there is no pidip for Windows.

What you need:

You need either a Linux or OS X system with firewire and Perl including the perl-gps module [2] , Pure Data including the pdp and zexy libraries, a DV camera, a GPS device with Garmin protocol, Garmin serial cable and if your laptop doesn’t have serialports you’ll need a USB-to-Serial (RS232) converter. It may sound a bit complicated but it’s not really. Once you have everything installed and configured correctly it’s very simple to use.

Uses:

I haven’t used the video objects for Pure Data much but I would think that GEM could be used to super-impose actual graphics. You could for instance generate an actual map of your travels and super-impose them. If the laptop has a network connection you could even query a database of known GPS coordinates in the vicinity and actually print info to the video image about the location. If you’re not on a network, the database could of course be on the laptop itself.
One of the interesting application I can see coming out of this, could benefit the tourist industry. Just think if it were possible to send tourists out videotaping a region and when they come back and watch their recording, they not only see the images they recorded but also detailed information about the places. With what we have right now, this would be a very simple feature to add. Pall Thayer

See Pete's article for more.