lintblog – Random Projects

Late

Late is a real-time public transit application for the Nokia N9 and N950 Harmattan phones. It works with any transit agency that uses NextBus and provides public access to their information.

Basic Usage Information

When starting the application, a view similar to above is shown. You can select Agency, Route, Destination and Stop to change them. Each item will provide a list that is relative to the list entry above it, and will allow quick navigation.

The top of the application has two buttons which are always visible. Pressing the title, “Late”, will always return you to the home screen. The three vertical dashes on the right side of the titlebar is a menu button. The menu will provide basic navigation and additional version information about the application. The “About Page” will also include a button to remove all content Late has cached on the phone.

The bottom shows a toolbar that provides quick navigation to most features in the application. The left button, an arrow pointing left, will return to the previous page or quit the application if already on the homescreen. The circular arrows button will refresh the currently shown prediction, or if on the homescreen, will load a prediction for the currently shown Agency/Route/Destination/Stop. The button with the circular arrow and checkbox will provide a history of recently viewed stops, including across multiple agencies. The star will provide a list of bookmarked stops, if any.

Inside of the History and Bookmarks screens, there will be a trashcan button that will empty out the history/bookmarks. Each entry can be clicked on to immediately load a prediction.

The Prediction screen will show the current arrival estimates for the selected stop. The top will indicate the Agency, Route and Stop currently shown. Each individual prediction will indicate the destination/direction of the vehicle. The star to the right of this text will toggle it as a bookmark.

Clicking on a prediction will expand it to show the vehicle id. At the bottom of the screen, a message window may be present to show any current broadcast messages from the transit agency.

To report any bugs, please send a coherent description with all relevant information in English to late at tehinterweb.com.

MagRead on Symbian^3

First off, let me start by providing a link to a Symbian^3 binary for MagRead. It can be found here. See the bottom of this post for some more details on this.

I’ve previously shown screenshots of MagRead running in Maemo on the N900 device. This post will feature some pictures of MagRead on Symbian^3 using a Nokia C7 device. At the bottom is a link to the program for Symbian, so those of you with a dongle can try it yourself.

The application has now been tested on the Nokia C7 and Nokia N8 phones.

First up is the intro screen:

The home screen for MagRead on Symbian^3

It’s rather simple at the moment, but intended to not overly clutter the interface. More than just screenshots, here’s a picture of this on the Nokia C7:

MagRead on Nokia C7 — The Nokia C7 running MagRead

Now with the program actually running. Here’s a screenshot of it having read an American Express card, followed by a picture of the C7 and the card next to it:

Amex card in MagRead — An Amex card as read by MagRead

Nokia C7 with an Amex card in MagRead — The Nokia C7 with the Amex card it just read

The picture shows the application reoriented into landscape mode, where it changes the layout slightly to better fit the new screen resolution.

When a card that is expired is swiped, MagRead also notes it by printing the expiration date in red:

In addition to credit cards, MagRead is also able to parse cards that comply with AAMVA standard — that is, identification/driver licence cards from the US and Canada that have magnetic stripes. Here is my California driver licence:

CA DL in MagRead — A California DL in MagRead

For anything other than a credit card or AAMVA-compliant card, MagRead will simply show the data as it is written on the card. Here’s an example using a membership card:

Membership card in MagRead — A membership card read by MagRead

The last big feature of MagRead in its current state is the ability to show partial reads. This is when a part of the stream can be decoded, but it fails one or more checksums. Here’s the Visa card from above with a failed swipe, showing most of the data stored on the stripe:

Partial Swipe in MagRead — A partial swipe in MagRead

The red character indicates that a checksum failed there. Any characters to the right of a failed checksum may also be invalid. Nonetheless, this swipe shows the complete card number and its expiration date (in YYMM format after the field separator).

Note that all credit card numbers shown are in fact invalid. The cards are old ones that had been compromised, and the numbers reissued.

I am now able to offer a self-signed package for MagRead. It can be found here. The package is version 0.1.3 (or 0.1(3), as shown in Symbian), and is the one demonstrated above.

There will be a security warning when installing, noting that it is not from a trusted source. Additionally, you’ll be asked to authorize the application to access the “camera or microphone”. This is required in order to record from the audio dongle (the camera is not touched). Those who don’t wish to trust me can use the Nokia Qt SDK to build the application from source themselves, and verify that what they’re installing does nothing malicious.

No information read by MagRead is ever saved onto the device or transferred over the network. Once it disappears from the screen, it’s gone. I will not be offering any application that doesn’t follow this principle.

Tips on using MagRead

As the user base has increased (ie, became existent), I’ve realized that it would be best to give a few usage tips for MagRead.

For Maemo 5, you can download the program from extras-testing. It’s entered as “magread”, and the package overview can be found here. The source code for the program, as compiled by Nokia’s Maemo buildhost, can be found from there as well.
For Symbian, Windows, OS X and Linux, there are presently no binary packages provided. I intend to provide some later, but at present you need to compile them yourself.
The code for the project, along with the bugtracker, is hosted by github. The github project page is here.
The program is capable of reading swipes in either direction. However, the success rate of swipes will get even better if you swipe the card in the same direction as the data is written. This would be from right to left, with the card oriented such that the magnetic stripe is on top and facing you.
MagRead in no way saves or transfers any information that it records. The open source nature of the program allows anyone to verify this, which I welcome anyone to do.
The Square dongle is only compatible with audio jacks wired for headsets. That is, audio jacks that support mono input and stereo output, as found on mobile phones. I’ve run into some Mac Book Pros that have compatible audio jacks, and some Thinkpads.
The Square dongle is configured to read track 2 only. For more reference on what this means, see this.
The Square dongle can read track 1, however you must offset the card from the read head by an additional 2.79mm (.110in). This can be accomplished by simple adding a wedge into the track. People have also had success in just holding the card slightly off the base while swiping.
If you swipe a card, and it doesn’t even give a “bad swipe” error, then the track you’re reading is blank. I’ve experienced this with several store cards, and hotel keys. If it repeatedly gives a bad swipe error, try reversing the direction and varying the speed. It may be damaged, or it may be in an unsupported encoding format.
mslib, which provides the decoding support for MagRead, only supports two encoding schemes (ABA and IATA). While they are by far the most common encoding schemes, there may be proprietary ones you come across. In the event you do come across such a card, mslib will still give a valid bit sequence. However, it won’t be able to decode the bits into characters. At present (v1.3-01), MagRead is unable to show you this bit sequence.
There is a checkbox on the screen marked “Show Partial Data”. This relaxes the checksum verification to allow for partial reads to be displayed. Any swipe with a known start sentinel will have its data shown, with characters that pass checksum displayed, regardless of whether the LRC is valid. You do not need to stop recording to enable this.
You do not need to return to the home screen in order to read another card. The application will continue listening for card swipes until you press the “Stop” button.

I’ll add more tips to this page as questions are asked of me, or as they come to mind. There are features presently in the git repository that are not in Maemo’s extras-testing package (v1.3-01). When I push out the updated version in the near future, I’ll be updating this page to reflect the new features (most of which are custom settings configuration).

Feel free to ask me questions via the comments section, email, IRC or other means.

mslib – A library for decoding magnetic stripes

After a number of posts describing how to decode audio streams of magnetic stripes to bits and then characters, I now present a C library that does all that.

The creatively named “mslib” can take a signed 16bit little endian PCM stream of a magnetic stripe and decode it.

My own usage tests have been with 48khz, but it should allow higher frequencies and slightly lower ones. When changing the frequency from 48khz, it may be required to alter the “peakOffset” and “peakThreshold” values using the functions defined and documented in mslib.h.

The code is written in C and available on github here. It’s presently released under the GPLv3 and has a single library dependency of glib-2.0. [Edit: No longer has a glib dependency — should be entirely c99 compatible]

The library is tersely documented like most open source code, but comes with an example/test utility. Decoding a stream consists of the follow steps:

Create the mslib object and load the PCM stream as a signed array of short integers using ms_create().
Create a list of peaks, used for decoding the stream to bits, using ms_peaks_find().
Filter the list of peaks for duplicates using ms_peaks_filter().
Decode the peaks to bits using ms_decode_peaks(). The bits as written on the card may now be fetched as a NULL terminated char array with ms_get_bitStream().
Decode the bits to characters using ms_decode_bits(). The characters as written on the card may now be fetched as a NULL terminated char array with ms_get_charStream().

The library can successfully decode any magnetic stripe to bits (subject to a clean audio signal). It will decode the bits to characters for streams encoded using the ABA and IATA encoding schemes as found on almost all magnetic stripes.

Development is still continuing, with a top TODO item of removing the glib dependency [Edit: glib has been removed in current builds]. Bug reports, comments and feature requests are welcome and can be left on github’s site.

Some example PCM streams that can be decoded with the utility:

capitalone.pcm – A Visa credit card

wamu.pcm – A Visa debit card

wamu-bad.pcm – An example of a bad swipe of the Visa debit card.

(All credit card numbers are of course no longer valid [not just expired]. All the streams are from track 2 and use ABA encoding)

MagRead

MagRead is a Qt based GUI application originally written for Maemo 5.

The application is designed to read all types of magnetic stripes that follow the ABA or IATA encoding schemes. In addition, it attempts to format the data from cards it recognises.

Magnetic stripe data is read over the audio input jack on the mobile phone. A hardware adapter is required, which primarily consists of something similar to an old cassette tape head. The start of the application displays a minimalistic welcome screen:

MagRead Start Screen — Start screen for MagRead

At present, credit cards and identification cards are the only cards that will display with their data formatted.

Credit Cards

Visa, MasterCard, American Express and Discover are currently shown:

Visa Card — Capital One Visa Credit Card (inactive number)

If an expired credit card is swiped, the “Expiration Date” field will appear in red:

Washington Mutual Visa Debit Card (inactive number)

AAMVA Driver Licence/Identification Cards

Cards in compliance with AAMVA standards will also be recognised. These are identification cards and driver licence cards issued by any government based in North America (approx. 22 US states/territories, several Canadian provinces and at least one Mexican state).

When swiped with a track 2 reader, the issuing authority (state/province/agency/territory) is displayed, along with the person’s unique ID number, date of expiration, date of birth, and their present age:

AAMVA Compliant CA DL — California Driver Licence (DL# Redacted)

Expired cards will put their expiration date in red, and include the word in caps “EXPIRED” underneath. Additionally, if the person’s age is under 18, it will appear in red. Those over 18 but under 21 will have their age appear in yellow.

Expired California Driver Licence (DL# Redacted)

Miscellaneous Cards

Cards which are not identified simply have the data on them displayed:

Partial Reads

On the home screen, there is a “Show Partial Data” checkbox. When checked, MagRead will display data read off of cards that my contain errors or be incomplete. By default, only swipes that have passed several parity checks are displayed.

Screenshot-20100820-032343 — Ralphs Rewards Card -- Red |s indicate characters with bad parity checks

Screenshot-20100820-032412 — Capital One Visa Card Partial Read

MagRead is still in active development, but will be available for download shortly. It will be released under the GPL.

At present, it is written in C and C++ and designed to run on the Nokia N900 with Maemo 5 only. Work is currently in progress to port the application to standard Qt 4.6, and it may be usable on Symbian devices with Qt support.

Magnetic Stripe IATA Encoding

This post coves the encoding scheme defined by the IATA, originally for use in the airline industry. This is the encoding found on airline boarding passes, in fact.

It is commonly found on track 1 of magnetic stripes, and track 3 on some cards. Unlike the ABA encoding, this is alphanumeric.

Character Encoding Overview

Each character is 7 bits in length. Six bits for the character itself and one odd parity bit.
The character set includes 64 symbols, ranging from 0x20 – 0x5f on the ASCII table.
Encoded characters are offset from ASCII by 32. E.g., the number 0 is stored as 16 decimal. Adding 32 gives us 48, the ASCII character 0.
Values are encoded in Least Significant Bit (LSB) order.

The encoding follows most of the ABA encoding scheme, but with a larger character set and different ASCII offset. Calculating the LRC and parity bits however is identical, and the same code can actually be used to decode both ABA and IATA.

Data Formatting

Always starts with a percent sign, the start sentinel.
Always ends with a question mark, the end sentinel.
The field separator is a caret (^). This typically separates the account number from the name, and the name from the expiration date.
A 7bit Longitudinal Redundancy Check character follows the end sentinel. Like ABA encoding, it uses an even parity bit. The 7th bit is a odd parity bit of the first 6 bits, just like a normal encoded character.
On track 1, can hold up to 79 characters. On track 3, it can hold up to 101 characters.

For information on how to calculate the LRC, follow the instructions in the ABA encoding post.

Common Data Formatting

Start Sentinel
Format Code. A single character to define the layout, usually ‘B’
The Primary Account Number (PAN), up to 19 characters in length.
Field Separator
Name of account holder. Format is “SURNAME/GIVEN NAME M” (middle initial). Supports up to 26 characters. Cards vary on how they truncate, some eliminating the / used to separate surname from given. Not all cards will have a middle initial.
Field Separator
Expiration date in YYMM
Additional data up the remaining space left on the track.
End Sentinel
LRC

Not all cards follow this format, and some even use a different field separator.

Magnetic Stripe ABA Track 2 Encoding

An overview of how data is encoded on track 2 of magnetic stripe cards

This post will cover the encoding scheme defined by the ABA for magnetic stripe cards. This format is the de facto standard for track 2 of magnetic stripe cards.

Character Encoding Overview

Each character is five bits in length. Four bits for the character itself, and an odd parity bit.
The character set includes 16 ASCII symbols, ranging from 0x30 – 0x3f on the ASCII table. These are numerical digits 0-9 and the characters : ; < = > ?
Encoded characters are offset from ASCII by decimal 48 (thus, the numeric digit 0 is stored as 0, and a value of 48 must be added to get the ASCII character 0).
Values are stored in binary in Least Significant Bit (LSB) order. E.g., the number 4 is stored as 001 and not 100.

I’ll go over a few quick examples of this.

The start sentinel is the semi-colon character. This is ASCII value 59, but we subtract 48 from this. We then have decimal 11, and in LSB binary: 1101. We then apply the parity bit to the end, which is 0 here as there is an odd number of 1s. Thus, the encoded character is 11010.

Another example, the number 5. This is ASCII 53, but becomes the number 5 once 48 is subtracted. In LSB binary, it is 1010. There is an even number of 1s, so the parity bit is 1. The encoded character is 10101.

Data Formatting

Always starts with a semi-colon, the “start sentinel”, to indicate beginning of data.
Always ends with a question mark, the “end sentinel”, to indicate end of data.
The equal sign is used as a field separator. Traditionally this is put between an account number an expiration date. Some cards will contain no field separators, others may contain multiple.
Only numeric digits should be used to store data — the remaining characters, : < >, are for hardware control purposes.
A 5 bit Longitudinal Redundancy Check (LRC) character always follows the end sentinel. It uses an even parity bit scheme, as opposed to the odd parity used for characters. Its fifth bit is an odd parity of the other 4 bits, and not an LRC parity bit.
A total of 40 characters can be stored, including start/end sentinels and field separators.

Longitudinal Redundancy Calculation

The LRC is calculated by looking at all the other encoded characters on the card and calculating an even parity bit.

I’ll go over an example here. Let’s assume we have the string ;12=34? as data to encode:

Just do addition down each column. If the total is odd, then the LRC parity bit for that position is 1. If the total is even, it’s a 0. The fifth bit of the LRC is an odd parity of the first four bits. In this case, the LRC is 1011, which is an odd number of bits, making the encoded LRC 10110. The fully encoded stream is then:

1101010000010001011011001001001111110110

Next I’ll do a quick overview of how data is commonly ordered on a card:

Common Data Ordering

Start Sentinel
Primary Account Number (PAN). Up to 19 digits in length. Overflow can be stored in the custom data section.
Field Separator
Expiration date in the format YYMM
Custom data. Length is up to the remaining space.
End Separator

This format is found on many cards, such as credit cards. Odds are if you have a magnetic stripe card with an identification number and expiration date, it follows this pattern.

Not all cards will follow this, refer to the data formatting section above for some rules that should be consistently followed.

Decoding the Waveform of a Magnetic Stripe

A method to decode magnetic stripes stored as PCM audio data.

This post is going to cover the method I use for decoding magnetic stripe waveforms recorded using an audio interface.

For in-depth information on how magnetic stipes are encoded, see:

http://en.wikipedia.org/wiki/Biphase_mark_code

And many other websites which can be found by Google.

First off, an overview of what we’re working with. I am using raw PCM audio files in the format S16 LE at 48KHz. I’ve worked with 441.KHz streams reliably, and easily scaled up to 192KHz — overall, 48KHz seems optimal. A card swipe at 48KHz generally consist of about 10k samples, and are roughly 20kb. The data on the card is represented using amplitude peaks of alternating polarity.

Here are some screen captures of the waveform:

To get the bits from the stream, we have two overall steps:

Find the location of each peak in the stream.
Decode the frequency of the peaks to output 0s and 1s.

To find the locations of the peaks, I follow a simple model. I take the waveform and an offset of it, and determine where the original waveform and its offset intersect. Here is an example of an offset waveform:

I then mark each time the two waveforms intersect, ignoring any intersections over a certain threshold. The threshold is to eliminate intersects that occur around 0 amplitude. There will typically be more than one intersect found per peak:

Waveform intersects — Intersects of the Two Waveforms Marked by Blue Crosses

In this example, with an amplitude threshold of 750, we have consistently found two intersects per peak. It is also possible to have situations where the little hump before the actual peak results in several intersects — all of these can be filtered out very simply.

The filtering process I apply is to group the intersects detected by amplitude polarity. For instance, in the above picture, the first positive amplitude two blue crosses around sample 1010 would be a group. The second group would be the two negative amplitude crosses around sample 1075, and so forth. Once I have a group, I can pick the highest absolute value of amplitude, and reliably assume this is the true peak.

Here is the signal, now filtered:

I can then move on to the second step, which is to use the frequency of the peaks to decode the stored bits.

A quick overview of how the encoding works:

Beginning and end of stream are padded with 0s.
Stream is self-clocking. The padding 0s tell you the clock speed, and you adjust the clock speed as needed while traversing the stream.
A zero bit occurs when the distance between peaks is a full cycle. A one bit appears when there are two peaks in the full cycle.

Going through the list of peaks and determining the difference between them produces a valid bitstream.

Here’s an illistration of the start sentinel above broken into cycles:

Decoded Start Sentinel — Start Sentinel Decoded to Binary Data

As the stream progresses, the clockspeed will reduce significantly, generally starting around 90 samples and reducing to as few as 10 to 15 samples. Since it happens gradually, you can recalculate your clockspeed after every cycle.

At the end, the card I’m using as an example is decoded to:

00000000000000001101001101000011100100001110011110000 10000100010001010101101011011100100100000010100001000 10011111001011000100100111000001000000010000110000111 11100110000000000000000000000000000000000000000000000 00000000000000000000000000000000000

Ultimately, I’m left with the decoded data:

;6030374425663402297=4912001?

An Albertsons Supermarket card was used as the example data on this post. The PCM file used may be downloaded here.