Thursday, April 12, 2012

The STM32W-RFCKIT as a 802.15.4 network analyzer on Linux

Summary: The STM32W-RFCKIT together with ST Microelectronics application note AN3406 is a low cost 802.15.4 (ZigBee, MiWi etc) network analyzer. Unfortunately the solution is Windows only and closed source. In this article I have reverse engineered the firmware protocol used in this app note so that a similar solution can be implemented on Linux, Mac OSX and other platforms. I have also written a first draft of this software for Linux.
USB dongle
The STM32W-RFCKIT is a low cost evaluation kit for the STM32W108 802.15.4 SoC from ST Microelectronics. The kit retails for about €33 ($40?) and comprises a USB dongle and a battery powered remote control device with buttons.

I was drawn to this kit because I had recently purchased a STM32F4Discovery board and I was impressed at how easy it was get a GCC tool chain up an running on Linux. I'm currently on the look out for a reliable, low cost 802.15.4 network analyzer solution and given its low cost,  the hope of Linux friendliness and the 802.15.4 transceiver I figured I had a good shot with this hardware.

I was pleasantly surprised to find the kit already comes with a free 802.15.4 network analyzer.  One one of the associated application notes, AN3406, is a 802.15.4 packet sniffer for the dongle. This works by loading special firmware onto the dongle. This then communicates with a Windows server program which translates the dongle's captured 802.15.4 packets (encoded in their own proprietary frame format) into PCAP. The Windows server program also invokes Wireshark and feeds the packet data to it.

Block diagram of the ST Microelectronics AN3406 packet sniffer solution (taken from application note AN3406).
Unfortunately this is a Windows only solution. As a Linux user it's always a pain to have to boot a Windows virtual machine (consuming a huge chunk of my available RAM) just to run one tool. So I set about writing a tool to facilitate the same functionality in Linux.

Although the AN3406 software is free, it is not open source and there is no documentation on the dongle to Wireshark server protocol. So I had to reverse engineer the dongle's protocol to get the same functionality with Linux.

I used the free (and excellent!) serial port monitor tool at to capture the packets to/from the device while opening the Wireshark server and starting a packet capture. Fortunately everything looked very straightforward and it wasn't difficult to write a Linux based server.

The packet structure is

All frames are prefixed with two bytes 0x15, 0xFF. This is followed by a one byte frame length field which is the number of bytes in the frame (excluding the 0x15, 0xFF prefix, the checksum and 0x0C terminator). All frames have a packet type or command code, therefore the minimum valid value for the frame length is 2.

This is followed by a one byte packet type or command code. I notice that frames sent from host to dongle have the command code most significant bit (MSB) cleared while those from the dongle to the host have the command MSB set. For example command 0x10 (set channel) is responded by command 0x90.

Depending on the command there may be zero, one or more data bytes.

Next a frame check sum which is calculated by summing of all bytes from the length field to the last data byte in a 8 bit register and then doing a bitwise NOT.  Finally all frames are terminated with byte 0x0C.

The known command codes are:

Command CodeDirectionDescriptionParameters / Data
0x01 Host to dongle No idea. However it's required for packet capture to work None
0x81Dongle to hostResponse to 0x01 command.One byte: 0x00.
0x10Host to dongleSet 802.15.4 channelFollowed by one byte of data which will contain the channel number (11 to 26).
0x90Dongle to host Set channel responseSame as 0x10
0x11Host to dongleStart relaying packets.No parameters
0x91Dongle to hostStart relay response.No parameters
0x12Host to dongleStop relaying packets.No parameters
0x92Dongle to hostStop relay response.No parameters
0xF0Dongle to host802.15.4 packet data802.15.4 packet metadata (timestamp, RSSI) followed by actual packet data. See section 'Packet capture format' for details.

Packet capture format:

802.15.4 packets are returned along with some packet metadata in the data portion of frames with command code 0xF0. After observing a few frames it quickly became apparent that the actual 802.15.4 packet data started about 8 bytes in. Thus the first 7 bytes is some sort of metadata.

I'll refer to these bytes by their index values starting at 0. Byte 5 remained constant throughout the entire capture and happened to be the same value as the 802.15.4 channel number. So I can safely say that's the channel number. Bytes 1 to 4 also grew in value consistent with a time stamp. As for byte 0, I wasn't sure about that. It seemed random consistent with the LSB of a time stamp, but I thought 5 bytes for a time stamp was odd. Also I figured that there must be a link quality indicator (RSSI) in there somewhere also. Byte 6 looked like a good candidate for the RSSI. So how to know? Plotting these values as a function of time can often reveal their meaning:

Byte 0 (blue) seems to be completely random. So my guess is that's the the least significant byte of a 5 byte time stamp clock. Byte 6 (red) on the other hand must be the RSSI. There are several ZigBee devices on the network. Each device will have an RSSI value based on it's distance/location/antenna that won't vary quickly with time. So each of those red traces corresponds to one ZigBee devices.

So it seems that byte 0, byte 1 and the lower nybble of byte 2 form a 20 bit fraction of a second (ie if treated as an integer each unit is 1/(2^20) seconds.  The upper nybble of byte 2, byte 3 and byte 4 form a 20 bit seconds field.

Putting it all together:

To try this, you must first load the sniffer firmware to the dongle. Right now the only way I know to achieve this is using the AN3406 Windows software. I've documented the procedure in this blog post. And it's also covered in AN3406. However I have no doubt that this can also be accomplished with some of the Linux tools used to program the STM32-F4Discovery board. If you know how, please let me know.

I wrote a translator from this firmware protocol to a PCAP format which can be piped into Wireshark. It's released under the BSD licence. Here is my first draft (version 0.1).

To run:

wireshark -k -i <( ./stm32w-wireshark /dev/ttyACM0 12 )

The command line tool takes two arguments: the device (usually /dev/ttyACM0) and the 802.15.4 channel number (11 to 26). Use the -h flag to get information on more options.

Note: there is a problem with launching Wireshark this way. If you close the Wireshark window, the stm32w-wireshark utility does not die automatically and remains running in the background.  Starting it again will mean that two stm32w-wireshark processes will be grabbing data from the dongle causing massive packet corruption. If you see packet corruption, check for running processes (ps -auxw | grep stm32w-wireshark) and kill any lingering processes. If you have suggestions for a better way of handling this please let me know.

What next?

There are a few lose ends I would like to tidy up (and I would appreciate any suggestions):
  • What's the best way of launching this tool together with Wireshark? Closing the Wireshark window should cause the tool to die. Should I be using named pipes?
  • It's a shame that the RSSI value cannot be recorded in the PCAP file (there simply is no field for arbitrary metadata). So if one wanted to use RSSI information together with Wireshark how can this be achieved? 
  • I would like to be able to upload this firmware without having to boot windows. It's the only Windows dependency left and it would nice to remove it. I suspect one of the standard tools used with the STM32 kits will do the job, but I don't have time to figure that out right now. 


Graham said...

That looks interesting. I'm a Wireshark comitter, although this isn't an area I've worked on I'd like to help you out.

To include the RSSI data (and the channel and clock info if you wish), yo need a new encapsulation type (a DLT) and then just pass the extra data prepended onto the 802154 packet. When Wireshark reads that type it can call the appropriate dissector that strips the preamble, decodes it and adds it to the display and then passes the remaining 802154 packet over to the regular dissector.

Email if you want some help.

jdesbonnet said...

@Graham: Oh, that's interesting. I wasn't aware of DLT. I must implement that. Let me google it first and I'll get back to you if I need help.

Drozdov said...

hi, please write to me to , i have the question about com-sniffering, thx

legodude said...

Got your software working and am receiving data from my Itron electic meter, pretty neat. Thanks! I'm going to play around with it some more and let you know if I come up with any improvements

legodude said...

Hi Joe,
Poking around more, I realised that packets I am interested in are encrypted. I actually found your message about Zigbee NWK decryption key, but I don't know if I have it working correctly.

You wrote:
Security Level: AES-128 Encryption, 32-bit Integrity Protection
Network Key: 39:30:65:63:6E:61:69:6C:6C:41:65:65:42:67:69:5A
(that's the ASCII values of ZigBeeAlliance09 *in reverse*)

I put this into the ZigBee NWK setting, as a preconfigured key with 'normal' byte order. Does keyname matter?

I can't seem to get it to decrypt any data payloads, any hints?


jdesbonnet said...

@legodude: No, the name you give the key doesn't matter. With the latest version of Wireshark you can enter multiple keys and it will try them all until one works. So if in doubt plug in a few variations. The "ZigBeeAlliance09" key is default for Home Automation, but I'm not sure about Smart Energy.

Is this a meter installed by a utility company? If so, it's possible the key is not public (after all, do you want anyone near your home knowing how much electricity you're consuming, when you are at home etc).

student said...

I got to see zigbee frames using windows. And I try to get it working on Linux but wireshark just says "Waiting for capture input data..." even though i have a device transmuting zigbee back and forth.
I am thinking there is an error because when i enter the command to run wireshark like so:

wireshark -k -i <( ./stm32w-wiresharkv2 /dev/ttyACM0 15 -d 9 )
DEBUG: debug level 9
Writing command 0x10 (set channel),

I think the last line is an error but I don't know lots of c.

student said...
This comment has been removed by the author.
jdesbonnet said...

Try leaving out Wireshark for the moment. Just spew hex to the screen:
./stm32w-wireshark -f hex /dev/ttyACM0 15

One side effect of invoking Wireshark the way I mentioned is that when you close it the stm32w-wireshark process does not automatically go away. You may need to check if existing processes are already running:
ps -auxw | grep stm32w-wireshark
and kill them:
killall stm32w-wireshark

I must come up with a better way of invoking this tool + Wireshark together.

student said...

I got it working this time.
I think I was using the dongle with out it being flashed.

Manuel Pietschmann said...

Thank you very much for this tool,
a 802.15.4 / ZigBee sniffer in Linux is very cool :-)

I'm using it with a fifo/ named pipe as follows, works really well here.

1) mkfifo snifferout
2) ./stm32w-wireshark /dev/ttyACM2 15 > sniffout
3) sudo wireshark -k -i sniffout

Brian said...

Has anyone figured out how to flash these devices under Linux?

Brian said...

Has anyone figured out how to flash these devices under Linux?

jdesbonnet said...

I believe the Contiki project is close to implementing 6LoWPAN on that hardware. I don't have the details right now, but I'm sure there is some tool to flash the devices there.