The Corelatus Blog - Entries from March 2009

GTH audio streaming: why stream over TCP?

Fri, 6 Mar 2009 16:30:38 GMT

GTH lets you stream audio from a TCP socket to a timeslot on an E1/T1 line. Some people are surprised by the choice to use TCP. When I added that support back in 2002, my first thought was to use RTP (RFC 1889). RTP is simple: you just dump the audio in a UDP packet with some timestamping information and shoot it out on ethernet at the right rate.

I'd worked with RTP before and I'd been at a couple of SIP interops where most of the attendees had trouble emitting audio at 'the right rate', i.e. 8000 samples/s. One manufacturer's system would emit 8007 samples/s. Another would play it back at 7999 samples/s. What do you do with the extra 8 samples per second? If you do nothing, you get endlessly growing delays and, eventually, a buffer overflow. If you come up with a strategy for throwing away samples, it's bound to interact badly with something, sooner or later.

The thing is, when you're streaming in pre-recorded audio, you don't need it to be at the right rate. You just need to make sure it doesn't overrun or underrun the GTH's internal buffer. I.e. you need flow control, not rate control. TCP has flow control, and everyone knows how to use TCP sockets. In 2002, doing things that way was right at the limit of what our 50MHz embedded CPU could keep up with. Now it's no problem at all.

Python

I'm playing around with python at the moment. Here's how to put some data on an E1 timeslot, straight from the python shell. First, set up a listening TCP socket:


import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.listen(0)
addr, port = s.getsockname()

Next, open another socket to the GTH command port (2089) and tell it we want to stream in audio on the socket we opened above:


a = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
a.connect(("172.16.2.7", 2089))
my_ip, _port = a.getsockname()
command = "" % (my_ip, port)
header = "Content-type: text/xml\r\nContent-length: %d\r\n\r\n" % len(command)
a.sendall(header + command)

Finally, accept() and send the data:


d, _ = s.accept()
d.sendall("hello world")
d.close()

That looks OK to me, though I imagine the style betrays my Erlang mindset. There's a more complete python example at the bottom of the API page.

The timestamp field in signalling headers

Mon, 23 Mar 2009 20:40:32 +0100

When the Corelatus GTH is used to monitor (sniff) signalling, it sends each sniffed packet to your server over a TCP socket, along with a header. For instance, for SS7 MTP-2 the header looks like this:

octet 0x00:  Length (16 bits)
octet 0x02:  Tag (16 bits)
octet 0x04:  Flags (16 bits)
octet 0x06: Timestamp (48 bits)

Every field is big-endian, i.e. the most significant byte comes first. Here's an actual header from a GTH, octet by octet:

00 1c 00 00 00 00 01 20  34 ee fa 61 99 99 99 99 ...

The timestamp is thus 0x012034eefa61, or decimal 1237838658145. For most applications, you just want to know which packet came first, so the interpretation of that number doesn't matter much, though it's useful to know that it's the number of milliseconds since the unix epoch. (wikipedia has a decent article about unix time)

Sometimes, though, you want to represent that as a human-readable time. Unix (and, most likely, Win32) provides functions to do that in the C library, so, after throwing away the last three digits (the milliseconds), this C program does it:


#include <time.h>
#include <stdio.h>

int main() {
  const time_t time_stamp = 1237838658;
  printf("%d corresponds to %s\n", time_stamp, ctime(&time_stamp));

  return 0;
}

The output agrees with what the clock on my wall says:

1237838658 corresponds to Mon Mar 23 21:04:18 2009

Python

Since I've been messing around with python, the same thing in python:


>>> import time
>>> time.ctime(1237838658)
'Mon Mar 23 21:04:18 2009'

Erlang

Erlang doesn't have an interface to the 'ctime' call, but you can use the gregorian calendar functions:


1> Epoch = calendar:datetime_to_gregorian_seconds({{1970, 1, 1}, {0,0,0}}).
62167219200
2> calendar:gregorian_seconds_to_datetime(1237838658 + Epoch).
{{2009,3,23},{20,4,18}}

Why use milliseconds?

Why is the GTH timestamp in milliseconds instead of either seconds or a 'timeval'-like seconds + microseconds?

We chose millisecond resolution for several reasons. Firstly, the shortest possible useful packet in SS7 takes a bit more than a millisecond to transmit at 64kbit/s. Secondly, the practical limit of NTP time synchronisation over the internet is about one millisecond at a typical site.

Decoding MTP-3 and ISUP

Wed, 25 Mar 2009 13:55:20 GMT

Sometimes, you want to look at the signalling on an E1 and use it to figure out when telephone calls start and stop. In SS7 networks, call setup and tear-down is done by the ISUP layer, which fits in to the SS7 stack like this:

Layer 4: ISUP
Layer 3: MTP-3
Layer 2: MTP-2
Layer 1: MTP-1 (typically an 2Mbit/s E1 or a 1.5Mbit/s/T1)

If you have a GTH connected to the E1 you're interested in, either via a DXC or a monitor point, the GTH takes care of layers one and two. That leaves MTP-3 and ISUP to you.

The easiest way to decode MTP-3 and ISUP is to let wireshark do it for you. There's a note about how to do that on Corelatus' official site. But this blog entry is about how to decode MTP-3 and ISUP yourself.

A signal unit (packet)

In SS7, packets are usually called "signal units". Here's what an SS7 signal unit looks like 'on the wire', octet by octet, with MTP-2 and MTP-1 already decoded:

  8d c8 1f 85 02 40 00 00  35 00 01 00 21 00 0a 02
  02 08 06 01 10 12 52 55  21 0a 06 07 01 11 13 53
  55 00 6e 00

MTP-3

The start of the packet is the MTP-2 (ITU-T Q.703) and MTP-3 (ITU-T Q.704) headers. These headers are easy to decode because they are always fixed-length:

Octet(s)	Value	Purpose
00--01	8d c8	MTP-2 sequence numbers, safe to ignore
02	1f	MTP-2 length indicator. Anything less than 3 is reserved for MTP-2 itself and should be discarded.
03	85	MTP-2 SIO. The SIO tells us which 'service' the signal unit is intended for. Q.704 sections14.2.1 and 14.2.2 tell us that anything ending in hex 5 is for ISUP.
04--07	02 40 00 00	MTP-3 Routing label. The routing label is just a "from" and "to" address in the SS7 network. For most applications we can ignore it. Q.704 figure 3 shows what's in the routing label.

Upshot: to see calls start and stop, all we have to do for MTP-3 is:

Look at the length indicator (offset 2) and discard any signal unit where it's less than 3.
Look at the SIO (offset 3). Discard if (SIO & 0x0f != 5)

ISUP

The rest of the signal unit is ISUP. Annex C in ITU-T Q.767 tells us how to decode ISUP. ISUP is fiddly because there are several types of ISUP packets, because several of those types have optional fields and because some of those fields are variable length. Here are the octets we have left after removing MTP-2 and MTP-3:

  35 00 01 00 21 00 0a 02  
  02 08 06 01 10 12 52 55 
  21 0a 06 07 01 11 13 53 
  55 00 6e 00

The first two octets are the CIC. The third octet is the Message type.

The CIC (Q.767 C.1.2) tells us which circuit this call uses. All the signalling for one call has the same CIC. In ITU networks, it's a 12-bit value packed into the field in little-endian byte order. In this case CIC=0x0035. We're sniffing an E1 line, so C.1.2.a tells us that the lower five bits correspond to the timeslot (timeslot 5) and the rest identifies the E1 itself.

The Message Type (Q.767 Table C-3) field tells us what sort of ISUP message this signal unit is. 0x01 is an IAM. 0x10 is RLC. For a minimal "show me what calls are going through the system" hack, we only need to look at the IAM (comes at the start of the call, contains the A and B numbers) and the RLC (sent when the call is finished) messages.

Now we know that the CIC=0x35, that the message is an IAM and we still have about a dozen octets to decode. Q.767 table C-16 tells us how to decode an IAM. There are some uninteresting fixed-length fields followed by the B number and then the A number. Look at the code (or Q.767, section C.3.7) if you're interested in the details. All we really care about is that these octets

  06 01 10 12 52 55 21

represent the B number: 21255512. You can see the number in the raw data if you skip the first three octets and swap every second digit.

Turning those ISUP steps into an algorithm to decode one signal unit:

Save the CIC
Is the message an IAM? Decode it as an IAM, which is fiddly.
Is the message an RLC? Just print the CIC.

That's all you need to do to make a simple system which prints the start and end of each call. To do something useful, you need to maintain a table of in-progress calls and match up the IAM and RLC messages with the same CIC. You also need to handle things like systems restarting.

Erlang code

Everything discussed above is implemented in the ss7_sniffer.erl example on corelatus.com. It makes good use of Erlang's binary syntax, e.g. here's the MTP-3 decoder:


    mtp3(<<_Sub:4, Service_indicator:4>>, <<DPC:14, OPC:14, SLS:4,
						    Rest/binary>>) -> 
	case Service_indicator of 
	0 -> % Management 
        ignore; 
	1 -> % Test/maintenance 
        ignore; 
	3 -> % SCCP 
        ignore; 
	5 -> 
        isup(DPC, OPC, SLS, Rest); 
	9 -> % B-ISUP; similar to ISUP, but not compatible. 
        ignore; 
	X -> 
        io:fwrite("ignoring SU with unexpected service indicator=~p\n", [X]) 
	end.

It looks a lot like one of the examples in the original paper about the binary syntax.

Python Code

The same thing done in Python is fairly straightforward once you discover the Python 'struct' library, which is basically the same thing as PERL's pack/unpack. The code is in sniff_isup.py, inside the GTH python examples zip.

It feels like I haven't discovered whatever it is python people use to unpack bitfields, e.g. something neater than:


    is_even = ((ord(num[1]) &amp; 0x80) == 0)

or, better still, a clean way to decode the MTP-3 routing label.

Comments

Bartosz, 28. January 2010

Hey. Just stumbled on this page while I was googling after Mtp3 procedures. I'm wondering if you have any materials that you could share regarding link setup - I'm working on open source stacks for ss7. Im able to setup mtp2,3 and layer 4(signle link), now I'm working on getting it work with more than one and frankly Qs for mtp3 are not very helpful.

(website)

Matt, 28. January 2010

@Bartosz: I only really work with MTP2. My customers use MTP3, so I know quite a bit about it, but you probably know more. For MTP2, the standard is difficult to read, but comprehensive. MTP3 seems harder. I have a copy of "Signalling in Telecommunications Networks" by van Bosse et al, it's useful for getting started, but I always end up trudging through the standards.

Interesting to see someone working on an open SS7 stack. Up until now, I've only been aware of openss7.org (written in C, some parts seem complete, the project seems to be a one-man effort).

Or: I don't think I can help much, but what you're doing looks interesting.

Matt

Emza, September 15, 2009

I found it very important but still not enough for me. I was looking for ISUP MTP-3 and ISUP decoding in C or in C#(not only start and end singnal unit but general which includes all message types). I would like to thank you but can help in my case please.

Thanks again
Embza

Matt, September 15, 2009

@Emza, decoding the rest of the message types is 'just' a matter of working your way through ITU-T Q.767 (ITU standards are now available for free at http://www.itu.int) and writing code to handle each and every section in the standard. Same basic idea as the messages I did. Doing it all is a few days of drudge work, which is why I haven't done it here.

About doing it in C or C#. I know nothing about C#. I work with C most days, though. Doing this sort of protocol decoding in C is fairly straightforward, but it's inevitably going to be more tedious than doing it in something like python, perl or erlang. In some applications the performance gain might be worth the extra effort.

Meskerem David, September 15, 2009

It is a great job. Keep working...

Can u please me one brief algorithm of decoding ISUP. Especially I am not clear how to decode the variable length parameters and the optional ones.

Thanks
Meskerem

Matt, September 15, 2009

@Meskerem, In an IAM, the A-number and B-number parameters are variable length. The algorithm for finding the start is trivial, it's just a pointer offset. Here's what it looks like in the python example code:

    

      def isup_iam(_, CIC, sif):
      # First 5 octets can be ignored
      bnum_pointer = ord(sif[5])
      anum_pointer = ord(sif[6])
      bnum = sif[5 + bnum_pointer:]
      anum = sif[7 + anum_pointer:]
      print "IAM called party: %s calling party: %s CIC=%d" \
      % (isup_number(bnum), isup_number(anum), CIC)

      # And here's how the example code figures out the length:

      # Decode an ISUP number, as per C 3.7
      def isup_number(num):
      length = ord(num[0]) - 2

the complete code is at http://www.corelatus.com/gth/api/gth_python_examples.zip

(the site also has exactly the same decoding routines in Perl and Erlang)

The Corelatus Blog - Entries from March 2009

GTH audio streaming: why stream over TCP?

Python

The timestamp field in signalling headers

Python

Erlang

Why use milliseconds?

Decoding MTP-3 and ISUP

A signal unit (packet)

MTP-3

ISUP

Further reading

Erlang code

Python Code

Comments

Bartosz, 28. January 2010

Matt, 28. January 2010

Emza, September 15, 2009

Matt, September 15, 2009

Meskerem David, September 15, 2009

Matt, September 15, 2009