The STM32W-RFCKIT as a 802.15.4 network analyzer on Linux

Summary: The STM32W-RFCKIT together with ST Microelectronics application note AN3406 is a low cost 802.15.4 (ZigBee, MiWi etc) network analyzer. Unfortunately the solution is Windows only and closed source. In this article I have reverse engineered the firmware protocol used in this app note so that a similar solution can be implemented on Linux, Mac OSX and other platforms. I have also written a first draft of this software for Linux.

STM32W-RFCKIT
USB dongle

The STM32W-RFCKIT is a low cost evaluation kit for the STM32W108 802.15.4 SoC from ST Microelectronics. The kit retails for about €33 ($40?) and comprises a USB dongle and a battery powered remote control device with buttons.

I was drawn to this kit because I had recently purchased a STM32F4Discovery board and I was impressed at how easy it was get a GCC tool chain up an running on Linux. I'm currently on the look out for a reliable, low cost 802.15.4 network analyzer solution and given its low cost,  the hope of Linux friendliness and the 802.15.4 transceiver I figured I had a good shot with this hardware.

I was pleasantly surprised to find the kit already comes with a free 802.15.4 network analyzer.  One one of the associated application notes, AN3406, is a 802.15.4 packet sniffer for the dongle. This works by loading special firmware onto the dongle. This then communicates with a Windows server program which translates the dongle's captured 802.15.4 packets (encoded in their own proprietary frame format) into PCAP. The Windows server program also invokes Wireshark and feeds the packet data to it.

Block diagram of the ST Microelectronics AN3406 packet sniffer solution (taken from application note AN3406).

Unfortunately this is a Windows only solution. As a Linux user it's always a pain to have to boot a Windows virtual machine (consuming a huge chunk of my available RAM) just to run one tool. So I set about writing a tool to facilitate the same functionality in Linux.

Although the AN3406 software is free, it is not open source and there is no documentation on the dongle to Wireshark server protocol. So I had to reverse engineer the dongle's protocol to get the same functionality with Linux.

I used the free (and excellent!) serial port monitor tool at www.serial-port-monitor.com to capture the packets to/from the device while opening the Wireshark server and starting a packet capture. Fortunately everything looked very straightforward and it wasn't difficult to write a Linux based server.

The packet structure is

All frames are prefixed with two bytes 0x15, 0xFF. This is followed by a one byte frame length field which is the number of bytes in the frame (excluding the 0x15, 0xFF prefix, the checksum and 0x0C terminator). All frames have a packet type or command code, therefore the minimum valid value for the frame length is 2.

This is followed by a one byte packet type or command code. I notice that frames sent from host to dongle have the command code most significant bit (MSB) cleared while those from the dongle to the host have the command MSB set. For example command 0x10 (set channel) is responded by command 0x90.

Depending on the command there may be zero, one or more data bytes.

Next a frame check sum which is calculated by summing of all bytes from the length field to the last data byte in a 8 bit register and then doing a bitwise NOT.  Finally all frames are terminated with byte 0x0C.

The known command codes are:

Command Code	Direction	Description	Parameters / Data
0x01	Host to dongle	No idea. However it's required for packet capture to work	None
0x81	Dongle to host	Response to 0x01 command.	One byte: 0x00.
0x10	Host to dongle	Set 802.15.4 channel	Followed by one byte of data which will contain the channel number (11 to 26).
0x90	Dongle to host	Set channel response	Same as 0x10
0x11	Host to dongle	Start relaying packets.	No parameters
0x91	Dongle to host	Start relay response.	No parameters
0x12	Host to dongle	Stop relaying packets.	No parameters
0x92	Dongle to host	Stop relay response.	No parameters
0xF0	Dongle to host	802.15.4 packet data	802.15.4 packet metadata (timestamp, RSSI) followed by actual packet data. See section 'Packet capture format' for details.

Packet capture format:

802.15.4 packets are returned along with some packet metadata in the data portion of frames with command code 0xF0. After observing a few frames it quickly became apparent that the actual 802.15.4 packet data started about 8 bytes in. Thus the first 7 bytes is some sort of metadata.

I'll refer to these bytes by their index values starting at 0. Byte 5 remained constant throughout the entire capture and happened to be the same value as the 802.15.4 channel number. So I can safely say that's the channel number. Bytes 1 to 4 also grew in value consistent with a time stamp. As for byte 0, I wasn't sure about that. It seemed random consistent with the LSB of a time stamp, but I thought 5 bytes for a time stamp was odd. Also I figured that there must be a link quality indicator (RSSI) in there somewhere also. Byte 6 looked like a good candidate for the RSSI. So how to know? Plotting these values as a function of time can often reveal their meaning:


Byte 0 (blue) seems to be completely random. So my guess is that's the the least significant byte of a 5 byte time stamp clock. Byte 6 (red) on the other hand must be the RSSI. There are several ZigBee devices on the network. Each device will have an RSSI value based on it's distance/location/antenna that won't vary quickly with time. So each of those red traces corresponds to one ZigBee devices.

So it seems that byte 0, byte 1 and the lower nybble of byte 2 form a 20 bit fraction of a second (ie if treated as an integer each unit is 1/(2^20) seconds.  The upper nybble of byte 2, byte 3 and byte 4 form a 20 bit seconds field.

Putting it all together:

To try this, you must first load the sniffer firmware to the dongle. Right now the only way I know to achieve this is using the AN3406 Windows software. I've documented the procedure in this blog post. And it's also covered in AN3406. However I have no doubt that this can also be accomplished with some of the Linux tools used to program the STM32-F4Discovery board. If you know how, please let me know.

I wrote a translator from this firmware protocol to a PCAP format which can be piped into Wireshark. It's released under the BSD licence. Here is my first draft (version 0.1).

To run:

wireshark -k -i <( ./stm32w-wireshark /dev/ttyACM0 12 )

The command line tool takes two arguments: the device (usually /dev/ttyACM0) and the 802.15.4 channel number (11 to 26). Use the -h flag to get information on more options.

Note: there is a problem with launching Wireshark this way. If you close the Wireshark window, the stm32w-wireshark utility does not die automatically and remains running in the background.  Starting it again will mean that two stm32w-wireshark processes will be grabbing data from the dongle causing massive packet corruption. If you see packet corruption, check for running processes (ps -auxw | grep stm32w-wireshark) and kill any lingering processes. If you have suggestions for a better way of handling this please let me know.

What next?

There are a few lose ends I would like to tidy up (and I would appreciate any suggestions):

• What's the best way of launching this tool together with Wireshark? Closing the Wireshark window should cause the tool to die. Should I be using named pipes?
• It's a shame that the RSSI value cannot be recorded in the PCAP file (there simply is no field for arbitrary metadata). So if one wanted to use RSSI information together with Wireshark how can this be achieved? 
• I would like to be able to upload this firmware without having to boot windows. It's the only Windows dependency left and it would nice to remove it. I suspect one of the standard tools used with the STM32 kits will do the job, but I don't have time to figure that out right now. 

STM32W-RFCKIT as a low cost 802.15.4 / ZigBee network analyzer

Only a few years ago 802.15.4 / ZigBee network analyzers were an expensive affair. Now, many of the low cost evaluation kits from manufacturers such as Microchip, Texas Instruments and ST Microelectronics can be configured as network analyzers.

This morning I took delivery of a ST Microelectronics STM32W-RFCKIT (€33 from Mouser). I was pleasantly surprised that it too has a network analyzer in the form of a Wireshark bridge. These days Wireshark is the only game in town for packet analysis, so this was a smart move.

The kit comes in two pieces: a remote control device with 6 buttons and a dongle for the PC.

Unfortunately all this is Windows software. So I powered up a Windows XP virtual machine, and installed the software from the ZIP file SW application to interface Wireshark packet capture tool (AN3406). I also needed to install Wireshark for Windows. I plugged in the dongle and assigned the USB device to my Windows VM (this is the same as physically plugging in the device if it was a real computer). Then the usual windows new hardware detection and driver installation. (I can never quite understand what is going on with Windows when new hardware is plugged in. I did a reboot also, but that was more out of habit... it might not have been  necessary).

Next, start the STM32W108 Wireshark Server. In the Settings menu set the location of the Wireshark application (usually under Program Files/Wireshark) if this has not been already set.

Right out of the box, the dongle does not have the right firmware installed. So select the Tools menu and select "Flasher". Click the 'Flash' button and all going well, the packet sniffer firmware will be transferred onto the dongle.

Now back to the main window of the Wireshark Server application. Select the right serial port. The Play button should now become active. Hit the play button and all going well Wireshark should start sniffing packets from the dongle.

Linux is my OS of choice, so it would be great to have this working without having to boot a Windows VM. Unfortunately it seems that despite the wise choice of using the Wireshark tool and extolling the virtues of open source code in the documentation, they've neglected to include the source code for the firmware and server software. However it shouldn't be too difficult to reverse engineer. If I get a chance over the coming days I'll give it a stab.

A screen grab of  Wireshark sniffing 802.15.4 / ZigBee packets from the dongle supplied with the STM32W-RFCKIT.

Update (3 April 2012): I just exchanged emails with STM tech support to see if there were willing to share the sniffer firmware source code. Unfortunately they are not. So it's either reverse engineer what's already there or write new firmware.

Update (7 April 2012): I did a quick comparison with the STM32W dongle vs my Microchip ZENA (first version). Both devices are on my desk, about 50cm apart. I ran a Wireshark session on both devices at the same time on the same channel. After a few minutes the STM32W device captured 424 packets vs 290 packets from the ZENA.  It seems the STM32W-RFCKIT makes for a better packet sniffer!

Update (9 April 2012): I've made good progress reverse engineering the firmware protocol. I've now got a Linux command line tool (written in C) that will dump packet hex to terminal and allows the 802.15.4. channel to be changed. I hope to release the first version of the Linux Wireshark server in the next few days.

Update (12 April 2012): The first version of the Linux Wireshark server has been released.

Visualizing diurnal temperature variation

I've been logging temperature from a ZigBee temperature sensor (located in my shed in the back garden) since about November last year. I thought it would be interesting to see the diurnal (daily) temperature variation over time. On the x-axis are days (from about 10 November 2011 to 7 March 2012) and on the y axis is time-of-day, with midnight being at the very top and bottom. Temperatures vary from about -3C (deep blue) to +15C (red). Periods where there is no data are gray. Sorry no legend or axis markings.

The chart was constructed by averaging temperatures into 15 minute bins (with about 5 samples per bin) and plotting each bin as a rectangle in SVG. I then used rsvg to convert to a PNG image.

For comparison here is a chart for the same time period of my home office temperature. The color scale is different, deep blue being 12C and red 22C.

Interfacing Sensirion temperature / humidity sensors to the XBee using 'bit-banging' technique

This article documents an technique to interfacing digital sensors such as the Sensirion SHT21 to an XBee Series 2 radio module running ZigBee end device firmware without using any additional hardware components. The same technique should apply to similar digital sensors.

The XBee Series 2 radio modules from Digi are ideal for developing prototype and low production run wireless sensors. The modules are pre-certified in US, Europe and other regions eliminating the requirement to under go RF certification. In addition to a UART port for communications the XBee module also provides  several IO lines which can be configured in digital input, digital output, ADC and PWM modes.

Digi provide several protocol options including DigiMesh (their own proprietary protocol) and ZigBee. For interoperability with other vendors equipment and security I use ZigBee. Each protocol family has several firmware options which can be loaded on the module: usually Coordinator, Router and End Device.  For battery powered applications the End Device firmware must be used to achieve reasonable battery life. In this mode the module spends most of its time asleep using less than 1µA current. Periodically it will wake (briefly consuming 15 - 40mA), query its parent router to see if there are any waiting packets. If there are none it will go back to sleep. In this mode a set of AAA batteries can last a year or more.

Analog sensors can be coupled to XBee IO lines in ADC mode (10 bits resolution ranging from 0 - 1.2V) and read remotely using Digi's remote sampling API. However analog sensors will probably require signal conditioning circuitry (op amps, filters etc) to make the best use of the ADC voltage range. This conditioning circuitry is likely to require individual calibration to achieve good results.

The headache of analog design can be conveniently side-stepped by using a sensor with a digital IO interface. Sensirion produce a range of temperature / humidity sensors with an impressive resolution (14 bit temperature, 12 bit humidity) and accuracy (less than ±0.3°C for some versions). These sensors talk I2C protocol (or a variant of I2C). But there is a catch: the XBee firmware from Digi does not directly support I2C.

A common solution is to use a low cost MCU as a bridge between the sensor and the XBee's UART. The MCU waits for a command from the network (via the XBee's UART), queries the sensor and relays the result back to the UART for transmission on the ZigBee network. By monitoring the XBee's SLEEP pin the MCU can also spend most of its time in a low power sleep, waking only when the XBee wakes.

There is an alternative... a common solution in a situation where there is no direct hardware support for a serial protocol: "bit banging". Each of the XBee's IO lines can be set high, low or in high impedance (input) state by remote control: everything needed to realize the I2C and SPI protocols. But there is a catch: unlike a MCU bit banging its own IO lines which can happen at clock speeds exceeding 100kHz, remotely bit banging XBee IO lines is a very slow process. Fortunately the Sensirion sensor datasheets do not place any lower bound on the protocol clock speed. So a clocking speed of just 1Hz will work just as well as 100kHz... it's just going to take a while to complete a query.

This is what the test setup looks like. A SHT75 sensor is at the bottom of the photo. A XBee board (the Grove XBee Carrier board from SeeedStudio) with DIO1 and DIO2 connected to the SHT75 clock and data pins respectively. The 3.3V power supply from the board powers the sensor.

Implemenation details:

The details of how to communicate with an XBee module are beyond the scope of this article. A good introduction to the topic is the book Building Wireless Sensor Networks from O’Reilly.

It’s important to note that the I2C protocol requires pull-up resistors on the clock and data lines. Fortunately the XBee has configurable 30k internal pull-up resistors for digital inputs which are configured with the ATPR command. They are enabled by default.

A sensor query begins by first issuing a remote ATIR command to the XBee to initiate frequent IO sampling. I found 100ms sampling period gave good results. ATIR must be followed by ATAC (commit) for the change to take effect. The sampling will keep the (normally sleeping) XBee awake. Also each time a sample packet is transmitted the acknowledgement will let the XBee know if there are incoming packets waiting for it. So it helps keep packet latency relatively low. Samples packets comprise the high/low state of any digital input pins and the ADC value of any pins configured as ADC.

Before proceeding to the next step wait until the first sample arrives. This may take several seconds depending on where the XBee is in its sleep cycle. When it starts to transmit samples you know it is awake and will stay awake.

Now issue ATD commands to set the clock and data lines into the necessary state. For example, ATD13 sets DIO1 into input (or high impedance) state, ATD14 sets DIO1 into a output low (0V) and ATD15 sets DIO1 into output high (3.3V). Any ATD command will need to be followed by an ATAC before the change will take effect.

To read a data line during the read phase of an I2C conversation, wait for a IO sample to arrive after the low-to-high transition of the clock.

Finally when the I2C conversation is complete set ATIR=0 to stop automatic sampling, followed by ATAC. On reception the XBee should go back to sleep.

I found that a short delay (about 100ms) between each AT command was required for acceptable results.

This is an oscilloscope trace of a temperature query of a SHT75 sensor. The SHT7x and SHT1x range use a protocol similar to, but not compatible with I2C. Note the time base is 3.7 seconds per division! The clock is in green, the data in yellow. The blue trace is connected to a third pin which was used for debugging (to help separate out parts of the conversation). Here it is set high while writing out the command (0x03, read temperature) .

The SHT21 and SHT25 are more recent temperature / humidity sensors from Sensirion. These sensors use standard I2C protocol. This trace is a I2C read temperature query to a SHT21 sensor:

Some results:

This is a chart of a few hours of data from a SHT21 and SHT75 connected to an XBee using this technique. For comparison a third wireless sensor, a Digi XS-Z16-CB2R sensor is included. All three sensors were enclosed in an insulated polystyrene box to ensure that all sensors were reading the same temperature and humidity. I've also included the supply voltage (a handy feature of the XBee modules).

One slightly disappointing result is that there are quite a few failed queries (compare the number of  CB2R samples in blue to those in red and green). It seems that with my current implementation of this bit banging technique the success rate is about 66%. I believe this can be significantly improved with some changes to the implementation.

Conclusion:

Bit banging serial IO protocols such as I2C, SPI with the XBee IO lines under remote control is feasible. A battery powered sensor unit can be constructed with nothing more than a XBee, a digital sensor (and some means to physically link the sensor to the XBee), a battery holder and suitable housing.  No other components are required.

However there are some down sides: it takes a long time to make a measurement (10 - 30 seconds). During this period the XBee is awake (consuming 15 - 30mA). This is not a problem if  AA or AAA cells are used and the measurements are infrequent (eg once per hour). As currently implemented, reliability is far from perfect (2 out of 3 queries succeed) however I believe this can be improved with some tweaks to the implementation. Also over 100 ZigBee packets are required to complete one measurement: this could be a problem on a congested network.

Code:

Unfortunately the test setup is too complex to package up a simple self contained ZIP file to implement this technique. However here is the source code of the main Java class file which implements the necessary XBee AT commands.

/**
 * Implement temperature and humidity queries to a SHT71 and SHT75 sensor by bit banging
 * XBee IO lines. 
 * 
 * @author Joe Desbonnet, jdesbonnet@gmail.com
 *
 */
public class SHT7x {

 public static final int CMD_TEMPERATURE_READ = 0x03;
 public static final int CMD_HUMIDITY_READ = 0x05;
 
 private XBeeSeries2 xbee;
 
 private int clockPin;
 private int dataPin;
 
 // delay=100, sampleRate=250 does not work
 // delay=150, sampleRate=250 does not work (reliably)
 // delay=200, sampleRate=250 works
 // delay=180, sampleRate=50 works
 // delay=180, sampleRate=100 works

 // delay between sending each packet to the NIC for transmission
 private int delay = 180;
 
 // ms between each sample
 private int sampleRate = 100;

 /**
  * 
  * @param xbee XBee proxy object
  * @param clockPin XBee pin used to implement SCK (0 = DIO0, 1 = DIO1 etc)
  * @param dataPin XBee pin used to implement SDA (0 = DIO0 .. etc)
  */
 public SHT7x(XBeeSeries2 xbee, int clockPin, int dataPin) {
  this.xbee = xbee;
  this.clockPin = clockPin;
  this.dataPin = dataPin;
 }

 /**
  * Implement short delay. Usually used to space ZigBee packets apart.
  */
 private void delay() {
  try {
   Thread.sleep(delay);
  } catch (InterruptedException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
 }

 private void clockHigh() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.HIGH;
  xbee.atCommand("D" + clockPin, param);
  xbee.atCommand("AC");
  delay();
 }

 private void clockLow() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.LOW;
  xbee.atCommand("D" + clockPin, param);
  xbee.atCommand("AC");
  delay();
 }

 private void dataHigh() throws IOException {
  byte[] param = new byte[1];
  // Data high is achieved by high impedance state (HIGH_Z) ie digital
  // input mode
  param[0] = XBeeSeries2.HIGH_Z;
  xbee.atCommand("D" + dataPin, param);
  xbee.atCommand("AC");
  delay();
 }

 private void dataLow() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.LOW;
  xbee.atCommand("D" + dataPin, param);
  xbee.atCommand("AC");
  delay();
 }

 /**
  * Reset communications with sensor. Required if the previous query did not complete.
  * 
  * @param xbee
  */
 public void resetComms() throws IOException {

  int i;

  dataHigh();

  // Pulse clock 9+ times while data high
  for (i = 0; i < 10; i++) {
   clockHigh();
   clockLow();
  }
 }

 /**
  * Start sequence.
  * 
  * @throws IOException
  * 
  */
 private void startSequence() throws IOException {
  clockHigh();
  dataLow();
  clockLow();
  clockHigh();
  dataHigh();
  clockLow();
 }
 
 /**
  * A command is 8 bits (MSB first) followed reading an ack bit from the device. 
  * In this implementation I ignore the result of the ack bit (but there
  * still must be a 9th clock pulse). Bits are written by setting the
  * data pin to either 0V (logic 0) or high impedance (logic 1). The data is
  * read by the sensor during a low to high transition of the clock signal.
  * 
  * @param command
  * @throws IOException
  */
 private void sendCommand (int command) throws IOException {
  int i;
  boolean lastBit = true;
  boolean currentBit = false;

  // MSB first
  for (i = 0; i < 8; i++) {
   currentBit = ((command & 0x80) != 0);

   // Only change data pin if there is a change. This reduces the number
   // of ZigBee packets transmitted.
   if (currentBit != lastBit) {
    if (currentBit) {
     dataHigh();
    } else {
     dataLow();
    }
    lastBit = currentBit;
   }

   // Pulse clock
   clockHigh();
   clockLow();
   command <<= 1;
  }

  // If data currenly low bring to high_z/input mode
  if (currentBit == false) {
   dataHigh();
  }

  // I don't bother to read the ACK bit, but the clock
  // must still be pulsed for it. 
  clockHigh();
  // don't bother sampling data pin -- will assume all ok
  clockLow();

 }

 /**
  * Reference datasheet §4.3.
  * @return Temperature in °C
  * @throws IOException
  */
 public float readTemperature() throws IOException {
  int v = makeReading(CMD_TEMPERATURE_READ);
  return -39.7f + 0.01f * (float) v;  
 }
 
 /**
  * Reference datasheet §4.1.
  * 
  * @return Humidity as RH%
  * @throws IOException
  */
 public float readHumidity() throws IOException {
  int v = makeReading(CMD_HUMIDITY_READ);
  return (float)(-2.0468 + 0.0367*(double)v - 1.5955e-6*(double)v*(double)v);
 }
 
 /**
  * Reset comms, send start sequence, command and read 16 bits of data.
  * 
  * @param what One of SHT7x.CMD_TEMPERATURE_READ or SHT7x.CMD_HUMIDITY_READ
  * @return
  * @throws IOException
  */
 private int makeReading (int what) throws IOException {

  int i;

  //
  // Configure XBee to send frequent IO samples. This has two important functions.
  // First it keeps the XBee end device awake. Also an end device transmitting a 
  // packet (which must go via its parent) has the side effect of polling the 
  // parent for any incoming packets. So frequent transmission also means low
  // latency in receiving packets.
  
  byte[] params2 = new byte[2];
  params2[0] = (byte) (sampleRate >> 8);
  params2[1] = (byte) (sampleRate & 0xff);
  xbee.atCommand("IR", params2);
  xbee.atCommand("AC");
  
  // Now wait for the first sample to arrive before proceeding. At this point we'll
  // know the end device awake.
  long t0 = System.currentTimeMillis();
  while (xbee.getLastIOSampleTime() < t0) {
   delay();
  }
  
  // XBee End Device should now be awake and responsive

  // The last query may not have completed leaving the communications in an undefined
  // state. Reset communications to a known state.
  resetComms();
  delay();
  delay();

  // Start sequence ref §3.2.
  startSequence();
  delay();
  delay();

  // Send 8 bit command (and read ack bit)
  sendCommand(what);

  // Wait for measurement to complete. We could poll the DATA line. It will be pulled
  // low by the sensor when the reading is complete. However the overhead of doing
  // this makes it not worth the effort.
  try {
   Thread.sleep(200);
  } catch (InterruptedException e1) {
   // ignore
  }

  //
  // Read 16 bits
  //

  int v = 0, sample;
  for (i = 0; i < 16; i++) {

   v <<= 1;

   clockHigh();

   // wait for IO sample
   t0 = System.currentTimeMillis();
   while (xbee.getLastIOSampleTime() < t0) {
    try {
     Thread.sleep(100);
    } catch (InterruptedException e) {
     // ignore
    }
   }
   // sample = xbee.getIOSample();
   sample = xbee.getLastIOSample();
   if ((sample & 0x04) != 0) {
    v |= 1;
   }
   clockLow();

   // write ack bit
   if (i == 7 || i == 15) {
    // write ack bit (0)
    dataLow();
    clockHigh();
    clockLow();
    dataHigh();
   }
  }


  //
  // Return end device to normal sleep pattern by disabling sampling (ie by
  // setting sample period to 0).
  //
  params2 = new byte[2];
  params2[0] = 0;
  params2[1] = 0;
  xbee.atCommand("IR", params2);
  xbee.atCommand("AC");

  return v;

 }

}

Controlling XBee IO lines with ZigBee commands

Summary: this post explains to send remote AT commands to an XBee module using ZigBee commands. The end point, profile ID, cluster ID and command and response format is documented. This is useful if you need to control a XBee's IO lines from a non-XBee device.

The Digi XBee Series 2 module.

The Digi XBee module is a popular RF module featuring a UART and several IO lines that can be configured as input/output or in some cases ADC or PWM. The "Series 2" version of these modules can be flashed with ZigBee compatible firmware. The modules then participate in a ZigBee mesh network.

The modules are configured and controlled by  AT commands (like the modems of old) which are issued through the module's UART by a computer or microcontroller. There are two varieties of the firmware: AT mode and API mode. The latter provides access to more advanced features of the module.

One feature of the API mode is the ability to send AT commands to remote XBee modules. This can be used to control relays, read the state of switches, read temperature etc. This in theory allows many small control applications to be accomplished with nothing more than a XBee module on it's own -- no need for an attached microcontroller.

There is a problem however. All the examples and documentation I've seen to date (I may very well have missed something, but not for the lack of trying) only cover controlling a XBee module through the XBee API. So if your interface to the ZigBee network is not a XBee (or equivalent Digi product) you're out of luck. In my case I have a Texas Instruments CC2530 based USB dongle and I require that software running on a computer control the state of a remote XBee's IO lines.

So, reverse engineering time! I wrote scripts to control the XBee digital IO lines from another XBee, and simultaneously ran a 802.15.4 packet sniffer. This is what I learned:

For remote AT commands the sending XBee issues a ZigBee command to the remote XBee on endpoint 230 (0xE6), with profile 0xC105 (Digi private profile), cluster 0x0021. The command is formatted as follows:

0x00, 0x32, 0x00, frame-id, sender-ieee-addr, 0x00, 0x00, atcmd0, atcmd1, [param]

frame-id: one byte API frame ID used in many XBee API calls
sender-ieee-addr: the 64 bit IEEE address (8 bytes, the most significant byte first)
atcmd0: the ASCII code of first character of the AT command (eg 0x4d or 'M' if command is ATMY)
atcmd1: the ASCII code of the second char of the AT command (eg ...)
params: zero, one or more optional parameter bytes

For example: to send ATMY (get 16 bit network address) the command will be:
0x00, 0x32, 0x00, 0x0f, 0x00, 0x13, 0xa2, 0x00, 0x40, 0x3c, 0x15, 0x5c, 0x00, 0x00, 0x4d, 0x59

The bytes with values 0x00 and 0x32 may have some significance, but I have no idea what it might be. I'm not sure if the sender IEEE address is important. It seems to work the same no mater what address I use.

Responses are sent on cluster 0x00a1. In response to the ATMY command I get:
0x0f, 0x4d, 0x59, 0x00, 0xd1, 0xed

So that seems to be:
frame-id, atcmd0, atcmd1, 0x00, atresponse...

The XBee digital IO lines are configured with the ATDn commands, where 'n' is the number of the IO line. The lines can be configured as input, output low, output high, analog and PWM (not all lines are capable of all the functions). The two functions that are of interest to me are output high (parameter value 5) and output low (parameter value 4).

Note: there is potential for some confusion encoding AT commands. Take for example the command ATD04. What this means is AT command D0 with parameter 4.  The digit 0 in the command part is encoded as the ASCII code ie 0x30, but the digit in the parameter part (4) is the byte value 0x04. So the command and parameters is encoded as 0x44, 0x30, 0x04.

Other clusters in endpoint 230 have other functions, which I'll document in another post.

Update (2 Oct 2011): Small edit. I omitted the frame-id in the format of the AT command response.

A makeshift MRF24J40MB module breadboard socket

I wanted to experiment with a Microchip MRF24J40MB 802.15.4/ZigBee module, but having only one (and at $30 each) I wasn't willing to commit to soldering it into place just yet. So came up with this idea: a makeshift temporary socket comprising pin headers and an elastic band!

Does it work? Almost. I found that a few pins weren't making electrical contact. Not one to give up easily, I stripped some stranded wire and wrapped individual strands around the pins. This extra bulk and roughness around the pins seemed to have done the job.

If you try this, double check all your contacts with a continuity checker. This is obviously not something you want to use in a production setting!

Update (17 Aug 2011): The MRF24J40 based modules from Microchip have come down in price significantly since this I wrote this blog post. In single quantities the MRF24J40MB is now about $15, and the MRF24J40MA about $10.

Using the Microchip ZENA ZigBee/802.15.4 network analyzer with Linux

Summary: The Microchip Technologies Inc ZENA is a 2.4GHz 802.15.4 (ZigBee, MiWi etc) network analyzer. It comes with free Windows-only software. There is no support for Linux and cannot be used with powerful tools like Wireshark. There is very little documentation available for the device hardware. In this post I've documented reverse engineering efforts by myself and others and present a small C utility for pcap packet capture on the Linux platform. 

Update (23 July 2013): Mr-TI has written an updated version of this tool to work with the latest version of the ZENA hardware: https://github.com/Mr-TI/ZenaNG

Update (3 Mar 2012): This article was originally written in Feb 2011. Microchip have just released a new version of the ZENA which comes in a USB 'thumbdrive' casing. This new version is based on their own MRF24J40 transceiver chip. This article relates to the older model.

The ZENA is a 802.15.4 network analyzer from Microchip Technologies Inc. It costs about $130.  It comes with free (closed source) Windows software. If you need encryption support you'll need to separately purchase an enhanced version of the ZENA software for $5.

The Windows ZENA isn't bad. It displays plackets as horizontal slabs and color codes each part of the packet. There is some basic filtering functions. For more comprehensive reviews see [1] and [2]. If you need to use this with Linux or you need a long packet capture you're out of luck. The export from the Windows software is limited to about 21 kB of data at a time.

Documentation on the hardware and USB protocol is scant. Joshua Wright did some reverse engineering work on the ZENA back in 2009 and wrote a Python script that selected a 802.15.4 channel of your choice and dumped raw data from the USB port to the screen. Due to the relatively high cost and limitations (packet injection not possible) of the ZENA his work on this device as shelved in favor of the cheaper and more flexible alternatives (eg Atmel's RZUSBStick)

I've taken Joshua's work and brought it to a point where it can be used to capture long ZigBee packet dumps in pcap format and analyze them with Wireshark.

The ZENA hardware

The ZENA comprises 75mm x 40mm PCB with a PIC 18LF2550 MCU clocked at 16MHz, a MRF24J40   CC2420 2.4GHz 802.15.4 radio transceiver, PCB antenna (with the option of installing a SMA connector for external antenna) and a mini-USB port to communicate with the host computer and provide power for the device.

The PCB seems to have been designed with other applications in mind also, having an unused area for a button cell on the underside and pads for switches on top. For some odd reason the MRF24J20 chip is unmarked.  Update: I've been informed that the ZENA predated Microchip's own MRF24J40 so the Texas Instruments CC2420 chip (originally made by Chipcon before being acquired by TI) was used instead. The markings must have been removed by Microchip for commercial reasons but are still just visible under the right light.

There are 3 green LEDs marked D2, D4 and D3. D2 appears to be a heartbeat which flashes with a one second on / one second off duty cycle while the ZENA is in sniffer mode. The rhythm is disrupted whenever a packet is received. D3 also flashes with network activity but I cannot deduce a pattern. D4 is illuminated only during the powerup test sequence.

ZENA USB Protocol

The ZENA (vendor ID 0x04d8 , product ID 0x000E) presents itself as a Human Interface Device (HID).

"lsusb" is a useful utility to discover metadata about USB devices. The following command will dump information about the ZENA to screen: "lsusb -v -d 04d8:000e". This output looks like this (I've highlighted lines of interest in bold):

Bus 002 Device 003: ID 04d8:000e Microchip Technology, Inc. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         8
  idVendor           0x04d8 Microchip Technology, Inc.
  idProduct          0x000e 
  bcdDevice            0.00
  iManufacturer           1 
  iProduct                2 
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           41
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 No Subclass
      bInterfaceProtocol      0 None
      iInterface              0 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.01
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength      34
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1

USBSnoop and usbmon are another useful tools which can be used to eavesdrop on USB traffic when used with the supplied Windows software.

This is what I know about the ZENA USB interface so far (lots gleaned from Joshua Wright's blog post and the Python script linked above):

• There are two end points: a control channel (0x01) and a packet channel (0x81)
• There is only one known control function: to select a 802.15.4 channel in the range 11 to 26. This is achieved by allocating a 64 byte buffer, zeroing it and setting the channel number (11 to 26) in byte offset 1 (second byte of the buffer). Send this to the control end point 0x01 using  usb_interrupt_write()
• To read 802.15.4 packet data allocate a 64 byte buffer and issue a usb_interrupt_read() to end point 0x81. If there are no packets available, usb_interrupt_read() will block until a packet arrives or the timeout period is reached. A timeout can be detected by checking the return status for -110.
• The first 6 bytes in the buffer are a ZENA header. Byte 0 is always 0x00. 
• ZENA header bytes 1 - 4 is a packet timestamp. Bytes 1 and 2 is the fraction of a second in 2^-16 second units (first byte is the least significant). Bytes 3 and 4 is the seconds part of the timestamp.
• ZENA header byte 5 is the number of bytes of data remaining. The remaining data at this point is the 802.15.4 packet data (excluding FCS) plus two reception quality bytes at the end.
• If the length field is greater than 58 bytes then one or more additional usb_interrupt_read() requests must be issued to retrieve the remaining data. The data will continue at byte offset 1 (for some reason the first byte returned by usb_interrupt_read() to this device is always 0x00)
• The ZENA does not return the 802.15.4 FCS. You will need to recompute it if you need it. It does however return a single FCS OK bit in the last byte of the data.
• Instead of the FCS, the last two bytes is reception quality information: I assume the values are as described in the MRF 24J40 datasheet (see sections 3.7 and 3.6) CC2420 datasheet section 16.4: "The first FCS byte is replaced by the 8-bit RSSI value. This RSSI value is measured over the first 8 symbols following the SFD... The 7 least significant bits in the last FCS byte are replaced by the average correlation value of the 8 first symbols of the received PHY header (length field) and PHY Service Data Unit (PSDU). This correlation value may be used as a basis for calculating the LQI."

The LQI and RSSI values from a sample packet capture look like this:

The ZENA command line utility

I've opted to use plain C instead of Python. I'm more familiar (albeit very rusty) with C and it eliminates dependency on Python and PyUSB. You will still require libusb and associated development files which may not be installed by default. I am currently using libusb version 0.1. I plan to move to the more recent version 1.0 for the next release. The file can be downloaded from the download area at http://code.google.com/p/microchip-zena/

To install on Ubuntu make sure libusb (version 0.1) is installed:

sudo apt-get install libusb libusb-dev

To compile just do:
gcc -o zena zena.c -lusb -lrt

(Update 20 Feb 2011:  version 0.2+ of the zena utility has slightly different dependencies and compile instructions. See comments near the top of the C file on how to compile and run.)

To run:

You'll probably need to be logged in as root to do all of these things. On Ubuntu you can do
sudo bash
to avoid prefixing everything with "sudo".

Prior to running you need to make sure the ZENA device is "unbound" from any kernel drivers. If you get a "ERROR: ZENA device not found or not accessible", that is likely your problem.  I would love to be able to check for this and unbind programatically, but I don't know how to go about it. Suggestions welcome. (Update 20 Feb 2011: It is possible to check if a device is bound to a kernel driver and unbind it in libusb 1.0 using libusb_kernel_driver_active() and libusb_detach_kernel_driver() functions. This will be incorporated in the next release – version 0.3).

This is how to "unbind" it: While ZENA is *not* plugged in, look at files in /sys/bus/usb/drivers/usbhid/
There should be files eg "1-3.1:1.0". Now connect the ZENA. You will find a new file in that directory. Make note of it. For this example assume it’s "1-3.3:1.0". Now do:

echo 1-3.3:1.0 > /sys/bus/usb/drivers/usbhid/unbind

(replace the "1-3.3:1.0" with whatever you find for your system)

Once the device is available you can run the zena utility.

zena -c channel [-f format] [-v] [-h] [-d level] [-h]

-c selects 802.15.4 channel and is mandatory. A number between 11 and 26 is expected.

-f selects packet dump format. Two output formats are supported:

• pcap (default):  packet capture file which can then be imported into Wireshark and other tools
• usbhex: this dumps the raw 64 byte chunks of data obtained from the USB port in hex. One line per 64 byte chunk. Each chunk is prefixed with a timestamp obtained from the host computer

-v will display version information and quit.

-h will display usage information and quit.
-d will display debugging information to stderr. A level of 0 implies no debugging (default), 9 maximum verbosity.

The packet capture output is sent to standard output.

Important note: I have observed kernel crashes running this utility on a fully updated (15 Feb 2011) Ubuntu 10.4 (kernel 2.6.32-28-generic).  A fresh Ubuntu 10.10 (kernel 2.6.35-22-generic) installed from CD image seems not to be affected by this problem.  I can only assume it's a bug in the USB part of the kernel. I suggest bringing the computer to single user mode when you try this the first time (or just be prepared to power cycle if you have to). Alternatively install Linux in a VM, attach the ZENA device to the VM and run from there: the kernel crash is contained within the VM. The Python script which this utility is based on also causes a similar crash -- so it's not specific to my implementation. (Update 22 Feb 2011: this seems not to happen any more since the move to libusb 1.0)

No binary is provided at this time. But it's trivial to build if you have gcc and libusb installed.

Corrections or additional information would be greatly appreciated. I'd be glad to expand this utility to use the ZENA to it's full potential. Let me know what you need.

Project files are hosted at Google:
http://code.google.com/p/microchip-zena/

I can be contacted at jdesbonnet (at) gmail (dot) com

References:
[1]
http://homewireless.org/wp/2010/05/a-tale-of-two-packet-sniffers-microchip-zena-and-avr-rz-usbstick/
[2]
http://www.willhackforsushi.com/?p=198

ZENA is a trademark of Microchip Technologies Inc.