Experiments with the RFM69 433MHz digital transceiver module

Summary: 

This post is an overview of an board I'm developing which encapsulates the RFM69 radio module with a MCU to expose a simple UART API which can be tailored for specific applications. This simplifies embedding a radio into existing applications with very little additional expense in terms of hardware or power budget. The API is implemented by a light weight radio OS (currently just 5KiB) which is open source and the bill-of-materials for the board can be as little as $8 / €7.

Background to project:

WRSC2014 contestant Team FASt from University of Porto.

The World Robotic Sailing Championship 2014 was held in Galway Ireland (my home town) in September this year. It was originally anticipated that each contestant would use a RFM69 radio module to relay real time positioning information back to a central hub for recording the race and scoring. It was also hoped that the race organizers could use the same radio network to broadcast race waypoints, starting times, obstacles etc. However given the short time available it became clear that race contestants would not have sufficient time to incorporate this module into their hardware and write the associated supporting firmware.

Based on the experience of WRSC 2014 I believe the best solution for such competitions is to develop a low cost ready made 'black box' which is given (ready made and programmed) to each contestant. The black box would expose a very simple API to the boat's software which should not take long to integrate. Or if it's a case the contestant simply does not have time to make any changes to their software the radio can be mated with a GPS module and it will send real time position, speed and heading back to the race organizers.

This project is the first step in such a system.  An entire race system will incorporate other elements such as base station software, real time visualization tools, race protocols etc. This blog post only discusses the radio board.

RFM69 family of modules:

The RFM range of modules are made by HopeRF and based on the Semtech radio ICs. The RFM69 comes in several varieties. The one I've been using is the higher power version: the RFM69HW [1] which is based on the Semtech SX1231H [2]  chip.

Feature highlights:

• Digital SPI interface
• Operates on 433MHz unlicensed ISM band
• Continuous or packet mode (66 byte FIFO)
• Built in AES encryption and CRC-16 engine
• Configurable bit rates up to 300kbps
• Configurable power settings
• Line-of-sight ranges in excess of 1km
• Sleep current < 1µA
• Only $4 in small quantities

The radio board:

This is the general architecture of the board: a MCU acts as the go between the application hardware and software and the RFM69 radio module. The MCU takes care of all the complex register configuration of the RFM69 and takes of low level functions (such as responding to pings, remote configuration commands etc) without ever bothering the application. The application talks to the MCU thru a standard UART interface using a very simple protocol.

In addition to the core API there is also the option to compile in application specific functions (for example I've build in support for a rain tip-bucket and a temperature sensor: more on that later).


Prototype mk1:

Unfortunately these modules have a 2mm pad pitch making them difficult to use directly with standard 2.54mm spacing prototype board or breadboard. So for the first prototype I glued the module onto a prototype board and used thin 30AWG wire to link the module pads to pads on the protoboard.

Reasonably need looking from above...

The problem is on the underside...

Even though it's only a hand full of wires, cutting, stripping and soldering these is very time consuming and error prone (well in excess of three hours per board).

For the first prototype I used a LPC810 ARM Cortex-M0+ MCU as the controller.  The ARM Cortex-M range is my MCU of preference these days due to the ease of 32 bit programming, open source and well supported tools and a wide range of vendors (NXP, TI, Freescale etc).

This 8 pin DIP, 4KiB program memory device was adequate for initial tests, but I quickly ran up against the 4KiB program limit. The 8 pin DIP package also had the advantage that I could socket it and try different versions of firmware just by swapping out the device.

With two of these modules I did some range tests. I soldered 17cm wire (quarter wave at 433MHz) to both sides of the board just at the RFM69 antenna output pad. I was able to achieve two way communication of about 1km along the (line-of-sight) length of the Doughiska Road in east Galway city.


Prototype mk2:

For the next prototype I moved to the LPC812: in many ways identical to the LPC810 but with more IO pins and more memory (16KiB program memory, 4KiB SRAM). Rather than use time consuming protoboard I decided it was worth spending some money on a PCB. I designed a two layer board using the free version of CadSoft Eagle and used Eurocircuit's standard FR4 pool to fabricate. This job cost €80, but for that I got 20 boards. The turn around time was nominally 10 working days, but I had them in a week.

The board in the photo actually comprises two boards:  one for the RFM69 discussed here and another for a similar (better?) module the RFM98 which I'll discuss in another blog post. The two halves are separated with a hacksaw.

One very handy feature of the LPC81x range of MCU is the switch matrix which allows many of the MCU functions to be mapped to an arbitrary pin.  This makes PCB layout a lot easier.

After populating with the MCU, module and 3 other components (LED, resistor and decoupling capacitor for MCU) this is what the board looks like. There is the option to use a wire as antenna, or alternatively splash out on a SMA connector (probably the most expensive component on the board!). This photo shows a SMA connector with a 433MHz stub antenna obtained from Farnell [3].

On the left is the UART connector for the radio API, pins for Serial Wire Debug (SWD) for debugging and reprogramming. Left top is the power connector (3.3V nominal, but will work down to 1.8V).

There is also a LED at the very top-left. This is a general purpose indicator that's under control of the MCU. I use this to flash a sequence after boot up tests are complete and flash it each time a packet is sent or received.

In the center there is an option for another connector which will give direct access to the RFM69 SPI port for debugging, or simply using the PCB as a breakout board for the radio module (leaving the MCU pads unpopulated).

The bill of materials is PCB €2;  LPC812M101JD20 €1.50; RFM69HW €3 ; passives, pin headers €0.50; SMA connector (optional) €3.50 ; stub antenna (optional) €5. That's a total of €7 without the SMA connector option. Those prices with the exception of the PCB are for single quantities from Element 14's catalog.  Significant discounts are usually available when ordering 20+. Also components can often be sourced at a much lower cost from Chinese retailers (eg Alibaba and Deal Extreme). The PCB cost is more complex: many fabricators have a fixed job charge (it was about €50 for Eurocircuits) and they then charge per PCB after that. So if you make a lot of them the price per board goes down quickly. If you make just one it will be very expensive.


The radio OS firmware:

My firmware is written in C and currently uses the LPCXpresso IDE to build. This uses the GCC compiler under the hood. The main purpose of the firmware is to expose an simple UART API which the host application uses. The API protocol is simple: a single character command/response code followed by parameters separated by spaces followed by a carriage-return to end the command. All parameters are in hexadecimal. Example:

application to radio:
T 44 5232

'T' means transmit a packet. It takes two parameters: the to-address and the packet payload.

radio to application:
p FF 42 7A14051F003D AD

The 'p' response code is transmitted to the application whenever a packet is received. There are four parameters: the to-address (FF hex being the broadcast address),  from-address, the packet payload and the RSSI (received signal strength indicator). 

Other commands include 'R' and 'W' to read/write RFM69 radio configuration registers. 'M' to switch operating mode from an active awake/listening (20mA current use) to a sleep-polling mode using an average of about 140µA.

Some advanced experimental commands are remote register read/write, allowing a remote radio to be reconfigured. Remote watchdog timer configuration which will reset the radio to default configuration should there be a prolonged loss of communication. Remote command execution (which can be used for remote configuration and to relay packets), remote battery voltage measurement.

Test application:

As mentioned earlier the intended application is for robot sailboat telemetry. But I'm conducting initial tests by using one of these radios with a rain tip-bucket and temperature sensor in my garden. I've got the radio board in an IP66 enclosure with a small PV panel and LiPo battery (and associated charger). 

The tip-bucket causes a magnet to fly past a reed-switch each time the bucket tips. I use this in conjunction with one of the spare LPC812 pins and an interrupt channel to count each time this happens. I've also got a DS18B20 temperature sensor on another spare pin. 

I run this in low power polling mode. Every 30 seconds the MCU comes out of deep sleep mode, makes a temperature measurement and fires up the transmitter and sends a status packet with the tip bucket counter, current temperature, and battery voltage [4]. It listens for about 100ms for a response packet. If none is received it goes back into deep sleep mode. 

If I need to communicate to the out door unit I must wait for a poll packet and then immediately send my command. If multiple commands need to be issued, the first command is to switch the radio into active listen mode, issue all the necessary commands and then put back into low power mode.

So far tests have been successful with the only communications disruptions attributed to either running out of battery charge (it's winter here and the days are short: unless  it's a sunny day there is not enough stored charge to keep it going through the night) or a problem with the USB/Serial cable at the receiving end.

This is 48 hours of some sample data. The disruptions are due to problems with the USB/Serial cable at the receiving end: the radio worked flawlessly. Also note the elevated temperatures are due to the sun falling directly on the IP66 box. 

Current and future work:

I'm currently working an a web browser based application/UI to interact with these radios. The application will allow configuration of the local and remote radios, performing experiments with various radio settings. Monitoring network performance etc.

Also I'm planning to make a third prototype. This will be smaller than the current board, perhaps putting the MCU on the underside of the board. I'm also considering moving to the recently announced LPC82x MCUs [ref] which feature more program flash, more SRAM, a full ADC and other new features.

Features for future firmware include mesh networking, encryption, remote program execution and over-the-air firmware upgrades.


Conclusions:

So far I'm very impressed with these radio modules. At only $4 they're a steal. I've verified two way communication at over 1km distance line-of-sight and I've heard reports of success well in excess of 1.5km line-of-sight. With the addition of an LPC812 MCU with my radio OS and simple UART API, embedding this radio into an application is far simpler than building the radio driver directly into the application software. The entire bill of materials is about $7 with the option of a SMA connector which might bring it to $10. My PCB layout is available under the BSD licence on GitHub [5] and the firmware is also available on GitHub [6].


Footnotes:

[1] RFM69 Datasheet
http://www.hoperf.cn/upload/rf/RFM69-V1.3.pdf

[2] Semtech SX1231H product information and datasheet
http://www.semtech.com/wireless-rf/rf-transceivers/sx1231h/

[3] Farnell/Element 14 SKU 2305893, €4.86 in single quantities.

[4] One feature I miss in the LPC81x range of MCU is a full ADC. There is however a comparitor which can be used with a voltage ladder to make crude voltage measurements. I use this for battery voltage measurement (Vcc thru voltage ladder on one comparitor input, and the built in 0.9V voltage reference on the other). In my tests it can detect low battery voltage condition, but due to the coarse resolution I get a warning only about a hour before the voltage drops below the cutoff threshold.

[5] https://github.com/jdesbonnet/RFMxx_LPC812

[6] https://github.com/jdesbonnet/RFM69_LPC812_firmware

Updates:

11 Feb 2015: Added link to firmware repository on GitHub.

The STM32W-RFCKIT as a 802.15.4 network analyzer on Linux

Summary: The STM32W-RFCKIT together with ST Microelectronics application note AN3406 is a low cost 802.15.4 (ZigBee, MiWi etc) network analyzer. Unfortunately the solution is Windows only and closed source. In this article I have reverse engineered the firmware protocol used in this app note so that a similar solution can be implemented on Linux, Mac OSX and other platforms. I have also written a first draft of this software for Linux.

STM32W-RFCKIT
USB dongle

The STM32W-RFCKIT is a low cost evaluation kit for the STM32W108 802.15.4 SoC from ST Microelectronics. The kit retails for about €33 ($40?) and comprises a USB dongle and a battery powered remote control device with buttons.

I was drawn to this kit because I had recently purchased a STM32F4Discovery board and I was impressed at how easy it was get a GCC tool chain up an running on Linux. I'm currently on the look out for a reliable, low cost 802.15.4 network analyzer solution and given its low cost,  the hope of Linux friendliness and the 802.15.4 transceiver I figured I had a good shot with this hardware.

I was pleasantly surprised to find the kit already comes with a free 802.15.4 network analyzer.  One one of the associated application notes, AN3406, is a 802.15.4 packet sniffer for the dongle. This works by loading special firmware onto the dongle. This then communicates with a Windows server program which translates the dongle's captured 802.15.4 packets (encoded in their own proprietary frame format) into PCAP. The Windows server program also invokes Wireshark and feeds the packet data to it.

Block diagram of the ST Microelectronics AN3406 packet sniffer solution (taken from application note AN3406).

Unfortunately this is a Windows only solution. As a Linux user it's always a pain to have to boot a Windows virtual machine (consuming a huge chunk of my available RAM) just to run one tool. So I set about writing a tool to facilitate the same functionality in Linux.

Although the AN3406 software is free, it is not open source and there is no documentation on the dongle to Wireshark server protocol. So I had to reverse engineer the dongle's protocol to get the same functionality with Linux.

I used the free (and excellent!) serial port monitor tool at www.serial-port-monitor.com to capture the packets to/from the device while opening the Wireshark server and starting a packet capture. Fortunately everything looked very straightforward and it wasn't difficult to write a Linux based server.

The packet structure is

All frames are prefixed with two bytes 0x15, 0xFF. This is followed by a one byte frame length field which is the number of bytes in the frame (excluding the 0x15, 0xFF prefix, the checksum and 0x0C terminator). All frames have a packet type or command code, therefore the minimum valid value for the frame length is 2.

This is followed by a one byte packet type or command code. I notice that frames sent from host to dongle have the command code most significant bit (MSB) cleared while those from the dongle to the host have the command MSB set. For example command 0x10 (set channel) is responded by command 0x90.

Depending on the command there may be zero, one or more data bytes.

Next a frame check sum which is calculated by summing of all bytes from the length field to the last data byte in a 8 bit register and then doing a bitwise NOT.  Finally all frames are terminated with byte 0x0C.

The known command codes are:

Command Code	Direction	Description	Parameters / Data
0x01	Host to dongle	No idea. However it's required for packet capture to work	None
0x81	Dongle to host	Response to 0x01 command.	One byte: 0x00.
0x10	Host to dongle	Set 802.15.4 channel	Followed by one byte of data which will contain the channel number (11 to 26).
0x90	Dongle to host	Set channel response	Same as 0x10
0x11	Host to dongle	Start relaying packets.	No parameters
0x91	Dongle to host	Start relay response.	No parameters
0x12	Host to dongle	Stop relaying packets.	No parameters
0x92	Dongle to host	Stop relay response.	No parameters
0xF0	Dongle to host	802.15.4 packet data	802.15.4 packet metadata (timestamp, RSSI) followed by actual packet data. See section 'Packet capture format' for details.

Packet capture format:

802.15.4 packets are returned along with some packet metadata in the data portion of frames with command code 0xF0. After observing a few frames it quickly became apparent that the actual 802.15.4 packet data started about 8 bytes in. Thus the first 7 bytes is some sort of metadata.

I'll refer to these bytes by their index values starting at 0. Byte 5 remained constant throughout the entire capture and happened to be the same value as the 802.15.4 channel number. So I can safely say that's the channel number. Bytes 1 to 4 also grew in value consistent with a time stamp. As for byte 0, I wasn't sure about that. It seemed random consistent with the LSB of a time stamp, but I thought 5 bytes for a time stamp was odd. Also I figured that there must be a link quality indicator (RSSI) in there somewhere also. Byte 6 looked like a good candidate for the RSSI. So how to know? Plotting these values as a function of time can often reveal their meaning:


Byte 0 (blue) seems to be completely random. So my guess is that's the the least significant byte of a 5 byte time stamp clock. Byte 6 (red) on the other hand must be the RSSI. There are several ZigBee devices on the network. Each device will have an RSSI value based on it's distance/location/antenna that won't vary quickly with time. So each of those red traces corresponds to one ZigBee devices.

So it seems that byte 0, byte 1 and the lower nybble of byte 2 form a 20 bit fraction of a second (ie if treated as an integer each unit is 1/(2^20) seconds.  The upper nybble of byte 2, byte 3 and byte 4 form a 20 bit seconds field.

Putting it all together:

To try this, you must first load the sniffer firmware to the dongle. Right now the only way I know to achieve this is using the AN3406 Windows software. I've documented the procedure in this blog post. And it's also covered in AN3406. However I have no doubt that this can also be accomplished with some of the Linux tools used to program the STM32-F4Discovery board. If you know how, please let me know.

I wrote a translator from this firmware protocol to a PCAP format which can be piped into Wireshark. It's released under the BSD licence. Here is my first draft (version 0.1).

To run:

wireshark -k -i <( ./stm32w-wireshark /dev/ttyACM0 12 )

The command line tool takes two arguments: the device (usually /dev/ttyACM0) and the 802.15.4 channel number (11 to 26). Use the -h flag to get information on more options.

Note: there is a problem with launching Wireshark this way. If you close the Wireshark window, the stm32w-wireshark utility does not die automatically and remains running in the background.  Starting it again will mean that two stm32w-wireshark processes will be grabbing data from the dongle causing massive packet corruption. If you see packet corruption, check for running processes (ps -auxw | grep stm32w-wireshark) and kill any lingering processes. If you have suggestions for a better way of handling this please let me know.

What next?

There are a few lose ends I would like to tidy up (and I would appreciate any suggestions):

• What's the best way of launching this tool together with Wireshark? Closing the Wireshark window should cause the tool to die. Should I be using named pipes?
• It's a shame that the RSSI value cannot be recorded in the PCAP file (there simply is no field for arbitrary metadata). So if one wanted to use RSSI information together with Wireshark how can this be achieved? 
• I would like to be able to upload this firmware without having to boot windows. It's the only Windows dependency left and it would nice to remove it. I suspect one of the standard tools used with the STM32 kits will do the job, but I don't have time to figure that out right now. 

STM32W-RFCKIT as a low cost 802.15.4 / ZigBee network analyzer

Only a few years ago 802.15.4 / ZigBee network analyzers were an expensive affair. Now, many of the low cost evaluation kits from manufacturers such as Microchip, Texas Instruments and ST Microelectronics can be configured as network analyzers.

This morning I took delivery of a ST Microelectronics STM32W-RFCKIT (€33 from Mouser). I was pleasantly surprised that it too has a network analyzer in the form of a Wireshark bridge. These days Wireshark is the only game in town for packet analysis, so this was a smart move.

The kit comes in two pieces: a remote control device with 6 buttons and a dongle for the PC.

Unfortunately all this is Windows software. So I powered up a Windows XP virtual machine, and installed the software from the ZIP file SW application to interface Wireshark packet capture tool (AN3406). I also needed to install Wireshark for Windows. I plugged in the dongle and assigned the USB device to my Windows VM (this is the same as physically plugging in the device if it was a real computer). Then the usual windows new hardware detection and driver installation. (I can never quite understand what is going on with Windows when new hardware is plugged in. I did a reboot also, but that was more out of habit... it might not have been  necessary).

Next, start the STM32W108 Wireshark Server. In the Settings menu set the location of the Wireshark application (usually under Program Files/Wireshark) if this has not been already set.

Right out of the box, the dongle does not have the right firmware installed. So select the Tools menu and select "Flasher". Click the 'Flash' button and all going well, the packet sniffer firmware will be transferred onto the dongle.

Now back to the main window of the Wireshark Server application. Select the right serial port. The Play button should now become active. Hit the play button and all going well Wireshark should start sniffing packets from the dongle.

Linux is my OS of choice, so it would be great to have this working without having to boot a Windows VM. Unfortunately it seems that despite the wise choice of using the Wireshark tool and extolling the virtues of open source code in the documentation, they've neglected to include the source code for the firmware and server software. However it shouldn't be too difficult to reverse engineer. If I get a chance over the coming days I'll give it a stab.

A screen grab of  Wireshark sniffing 802.15.4 / ZigBee packets from the dongle supplied with the STM32W-RFCKIT.

Update (3 April 2012): I just exchanged emails with STM tech support to see if there were willing to share the sniffer firmware source code. Unfortunately they are not. So it's either reverse engineer what's already there or write new firmware.

Update (7 April 2012): I did a quick comparison with the STM32W dongle vs my Microchip ZENA (first version). Both devices are on my desk, about 50cm apart. I ran a Wireshark session on both devices at the same time on the same channel. After a few minutes the STM32W device captured 424 packets vs 290 packets from the ZENA.  It seems the STM32W-RFCKIT makes for a better packet sniffer!

Update (9 April 2012): I've made good progress reverse engineering the firmware protocol. I've now got a Linux command line tool (written in C) that will dump packet hex to terminal and allows the 802.15.4. channel to be changed. I hope to release the first version of the Linux Wireshark server in the next few days.

Update (12 April 2012): The first version of the Linux Wireshark server has been released.