Tuesday, February 18, 2014

Capturing a thermal IR video with ffmpeg on Linux

When tethered to a computer with USB the Flir E4 [1] thermal camera can act as webcam. The frame rate is limited to 9fps due to ITAR restrictions [2], but this is sufficient for most applications. On Linux FFMPEG [3] can be used to capture a video with this command line:

ffmpeg -f v4l2 -r 9 -s 640x480 -i /dev/video0 thermal.avi

You can preview while shooting with the camera's LCD. But sometimes you want to see a preview on screen. That turns out to be tricky.

I found a useful solution on this thread [4]:

1. Create a fifo file:
mkfifo /tmp/livevideo.fifo

2. Run mplayer on the fifo:
mplayer  -cache 512 -really-quiet /tmp/livevideo.fifo

3. Run ffmpeg to capture to file and also relay to the fifo:
ffmpeg -f v4l2 -r 9 -s 640x480 -i /dev/video0 thermal.avi -map 0 -f avi -vcodec rawvideo -y /tmp/livevideo.fifo

Here is a sample video captured with ffmpeg:


[1] also the Flir E6, E8, the Exx range and most (all?) recent models as far as I know

[2] http://en.wikipedia.org/wiki/International_Traffic_in_Arms_Regulations

[3] http://www.ffmpeg.org/

[4] http://nerdfever.com/?p=2661

Monday, February 17, 2014

Sox spectrogram log frequency axis and upper/lower frequency limits

Linear spectrogram
As a result of some of the work I did last week on rendering MP3 files to scrolling spectrum waterfall plots, I noticed that most of the interesting detail was in frequencies under 1kHz.  One obvious way to see the detail at lower frequencies, yet still keep the higher frequencies is to plot on a log frequency scale.

Unfortunately this is currently not supported in SoX, so I modified the spectrogram module to add this feature. This I noticed that plotting charts from 1Hz to the nyquist frequency meant that a lot of screen space was wasted in the lower 1 - 50Hz range which is not the most interesting for most audio files. So I also added a switch to set the lower and upper frequencies of the chart (this actually took considerably more work than the plotting to the log scale!).

Log axis spectrogram. Lower frequency details are far more visible.
The modified spectrogram.c file is available here [1]. Visit the SoX site [2], download the source code, replace src/spectrogram.c with this file and compile. This has been tested with the latest code as of 14 Feb 2014.

Summary of changes:
  • Implement -L which plots the spectrogram on a log10 frequency axis. By default from 1Hz to nyquist frequency.
  • Implement -R <low_freq>:<high_freq> : restrict spectrogram frequencies. I also updated the linear scale code to honor this switch.


sox mymusic.mp3 -n spectrogram -L -R 50:8k

Known problems and observations:
  • At lower frequencies (1 - 100Hz) each spectrum bin in q->dBfs[] array occupies several pixel rows: so it looks blocky. A lesser problem: at the top some frequency bins will be ignored because there will be more than one bin for each pixel row. I could fix the blockyness by getting a more detailed spectrum.
  • If lower and upper frequencies are in the same decade then there are no y axis tick labels. Hopefully will have a fix for that soon.
  • When compiling I'm getting this warning when using log10f() and powf() : "warning: passing argument 1 of 'log10f' as 'float' rather than 'double' due to prototype". According to the man pages for those function they should accept float args! I see some discussion on this relating to compiler switches: but I don't want to go changing anything there.
  • I created two new functions to parse the -R frequency range switch to cut down on code duplication.
    parse_range (const char *s, int *a, int *b)
    parse_num_with_suffix (const char *s, int *a)
    I'm not familiar with the sox code base, so I'm not sure if similar functions already exist (I couldn't find any). I'm also concerned I'm making a temporary change to a string in parse_range when the type is const char *.  I need to brush up on my C :)


[1] https://github.com/jdesbonnet/joe-desbonnet-blog/tree/master/projects/sox-log-spectrogram

[2] http://sox.sourceforge.net/

Saturday, February 15, 2014

Boosting audio volume in a video file

For some reason I'm finding that some video files played directly on my new Samsung TV (by mounting a USB drive) have a low volume and boosting the TV's volume beyond 40 (of presumably 100) yields no additional gain.

This ffmpeg command can be used to boost the volume in the file:

ffmpeg -i myvideo.mkv -vcodec copy -af "volume=12dB" myvideo-boosted.mkv

You can vary the boost by chaning the volume gain parameter. I found 12dB was about right. Negative values are allowed also.

Wednesday, February 12, 2014

Convert MP3 to a scrolling spectrum waterfall plot video

There are many utilities that display a scrolling spectrum waterfall plot [1] from a signal, but I was unable to find any open source utility that converted an audio file into a video file with a scrolling waterfall plot + the audio.

The SoX [2] sound utility can generate a static spectrum waterfall plot image from an audio file (or part of it), but it can't make a video. So I wrote a script to do this.

It's a very brute force approach. It requires lots of CPU time and lots of temporary disk space. The script depends on SoX, GNU Parallel, mencoder (or ffmpeg). GNU Parallel is optional, but will result in significant speed up on a multi-core system.

To use, create an empty directory on a volume with plenty of disk space and run with the audio file as a parameter.  Eg:

./make-spectrogram-video.sh -t "My Music File" mymusic.mp3

The output will be written to output.avi and output.mp4. Other options include setting frame rate, the speed of the scrolling, audio credit text etc. To get full help do this:

./make-spectrogram-video.sh -h

The script is available on GitHub here [2]. Here is a sample output video of Bach's Toccata and Fugue in D Minor [3] :


17 Feb 2014: I noticed that all the interesting details in music is squashed down at the very bottom of the spectrogram. So I updated the spectrogram module in SoX to have the option of plotting on a log axis. This isn't in the offical sox distribution. See this blog post for more details [4].


[1] http://en.wikipedia.org/wiki/Waterfall_plot

[2] https://github.com/jdesbonnet/audio-to-waterfall-plot-video

[3] Music MP3 file from https://archive.org/details/ToccataAndFugueInDMinor. YouTube video at http://www.youtube.com/watch?v=utp95bprqeg

[4] http://jdesbonnet.blogspot.ie/2014/02/sox-spectrogram-log-frequency-axis-and.html

Tuesday, February 11, 2014

Simple is better when it comes to microwave oven UI

About a year ago the magnetron (the magic yolk that generates microwave radiation) on my old Sharp microwave oven failed. This is not economically repairable failure mode, so I went shopping for a new oven. I resisted the temptation to get one with lots of lights, buttons and bluetooth. Instead I opted for a €49 oven with a simple mechanical user interface.

After a year of use I have to say it's the best oven user interface I've ever used. It's faster to set than the keypad of the old oven. The only feature I miss from the old oven is a kitchen timer. And it's a missed opportunity here: all they needed to do was add a zero power option on the power setting knob!

Monday, February 3, 2014

A fine stretch in the evenings

This is small web app (click to try) I wrote, inspired by that phrase often said at this time of the year at this latitude: "a fine stretch in the evenings". It is also inspired by a low-latitude friend who thought our long summer days a bit freaky :-)  [1]

It's a well known phenomena that day/night lengths vary throughout the year with latitude. From no variation at the equator to the extreme 24 hour days and nights near the poles. This app helps you visualize this variation. Enter your latitude/longitude or choose your location on a map. You'll get a visualization of the length of days/nights throughout the year at that location. The system will estimate the appropriate timezone for each location and adjust for 'wall clock' time accordingly.

On the chart black is night (obviously), yellow is day and the blue band is twilight. If there is daylight saving in the timezone then you will see a discontinuity in the sun rise/set curve during the transitions.

You can have one or two locations (although a HD monitor will be required to fit both charts side by side).

Some interesting well-known observations:

1. Here in Ireland the days get depressingly short during the winter and we all get a bit cranky, sick, moldy and drink too much. [2]

2. At the equator, day/night is pretty much same all year around. They all have a happy disposition and they don't drink at all! [3]

3. Up near the arctic circle there comes a point in the year where the sun doesn't rise at all. Oddly up there there they don't seem to get nearly as cranky, sick or moldy as we do in Ireland, but they still drink too much.

Some lesser known observations:

1. The length of twilight varies with latitude. In fact you can estimate latitude using just that [4] 

2. The shortest day of the year, the day of latest sun rise and the day of earliest sun set are not the same! (ie the sun rise and set curves are not exact mirrors of each other)

This app uses HTML5 has been tested with the latest Chrome and Firefox browsers. It will probably will cause Internet Explorer to explode.
[3] There are exceptions!

[4]  http://www.birdtracker.co.uk/ 

Friday, January 31, 2014

Generating a spectrogram of The Shepard Tone

Earlier today I saw a tweet from Tim O'Reilly pointing to an interesting article in The Atlantic on audio illusions.  The Shepard Tone illusion sounds like a continuously increasing or decreasing tone which goes on indefinitely... which is impossible because you'd eventually end up at at inaudibly low or high pitch.

Curious as to what was going on I downloaded the YouTube video, extracted the audio and ran off a spectrogram with the open source sox audio tool. I had done this before last year, so I was familiar with the process.

1. Download the YouTube video:
youtube-dl -o shepard_tone.mp4 http://www.youtube.com/watch?v=DfJa3IC1txI

2. Extract the audio from the video
ffmpeg -i shepard_tone.mp4 -f mp3 shepard_tone.mp3

3. Convert to mono (the YouTube video has stereo audio)
sox shepard_tone.mp3 shepard_tone_mono.mp3 remix 1,2

4. Create the spectrogram
sox shepard_tone_mono.mp3 -n spectrogram -o spectrum.png

One small problem: most of the action is down at the lower frequencies. So I resampled the audio and regenerated the spectrogram so that the interesting bits were more visible.

5. sox shepard_tone_mono.mp3 -r 4k -o shepard_tone_4k.wav

6. sox shepard_tone_4k.wav -n spectrogram -o spectrum.png

Tools used :




Thursday, December 12, 2013

Heating a room for 8p a day with tea lights YouTube video analysis

This video was recently brought to my attention: 

My first thoughts were:

  • Oh god, another free energy / perpetual motion machine idea
  • Hmm: this guy seems to understand basic energy (given his observation that the computer contributed to the room heat budget)
  • He works for a boating magazine (those guys tend to know their basic science/math).
So looked I at the figures.

It seems he's taking about these tea lights: http://www.ikea.com/ie/en/catalog/products/50097995/

They're actually €2 here in Ireland (about 1.70 GBP).

Measuring some tea lights I have here (they can't be too different to those IKEA candles), the diameter is 39mm x 11mm high. That's a total volume of 1.314 litres of wax for all 100 tea lights. It's most likely parafin wax in those candles. The density of parafin wax is 900kg/m3 so that's 1.183kg of parafin wax.

Looking at the heat of combustion per unit mass  parafin wax and kerosene / heating oil are pretty much identical energy wise at 46MJ/kg [1]. So the tea lights represent 54MJ of energy [2].

Looking at the latest prices for heating oil (kerosene) in Ireland (December 2013) is €0.83/litre. The density of kerosene is slightly less than parafin wax at about 800kg/m3. So to get 54MJ of energy I'll need 1.479 litres of kerosene. That's €1.22 at current prices.

Which is almost identical to 1 GBP at the time of writing. So fuel wise, this is on par with home heating oil (at the time of writing).

So what's with the flower pots? From a basic physics point of view there is no energy advantage here that I can see. But what it does do is contain the flame and make the apparatus safer than naked flame.


Tea light candles are handy, store well and don't spill. They are also (probably) safer than liquid kerosene. At 1 GBP it's on parity with home heating oil for energy/cost. At the €2 price it's not quite as efficient as heating oil, but still in the same ball park. There might be a CO risk for small poorly ventilated spaces... but I guess no different to that of candles. Ok.. there might be something to this after all.. but it's no energy miracle.

[1] http://en.wikipedia.org/wiki/Heat_of_combustion

[2] Burning stuff is a rather efficient way of releasing energy. You'll get pretty much every scrap of that in heat.

Saturday, August 31, 2013

Using Sox spectrogram tool to analyze audio noise

While out on a walk near Lagos (Algarve, south Portugal) I noticed a loud buzzing/crackling sound. It sounded very similar to the kind of buzz/crackle you'd hear near high voltage transmission lines, but there was no power lines in sight. So I though I'd capture a few seconds of audio with a voice recorder app on my phone and look at the spectrum later to rule in/out an electrical origin: any thing electrical would have peaks at exactly 50Hz and harmonics of 50Hz.
It turns out that sox has a neat tool to create a spectrogram from any audio file:

sox recording.wav -n spectrogram -o spectrum.png

However the frequencies I was interested in were down well under 1kHz, so I first resampled at a rate twice the highest frequency of interest:

sox recording.wav -r 2k -o t.wav
sox t.wav -n spectrogram -o spectrum.png

So this is the result:

The buzz sound can be seen in the spectrograph as horizontal streaks at about 60Hz and 120Hz. Not being exactly 50Hz rules out an electrical origin. The frequency can also be seen to vary a little with time.

Here is a spectrogram of another recording I took on the way down to the beach later. There was multiple sources in this recording (audio file here) but fainter/further away.

and other (audio file here):

You can see multiple horizontal streaks  of varying frequencies.


It's clear now the noise is from an insect. What insect, I have no idea. (Update: I think it might be Cicada)

If puzzled about the origin of a strange sound, record it and create a spectrogram... it might yield clues. Anything relating to utility power will be at the AC frequency (50Hz most of the world, 60Hz US).

Wednesday, August 28, 2013

LPC800 Mini Kit sous vide cooking controller


This project uses a LPC800 Mini Kit evaluation board as a sous vide cooking controller together with a DS18B20 temperature sensor and a modified timer unit and a rice cooker. A simple user interface (using just one microswitch and one LED) allows the user to select a set temperature. PID control and other techniques are used to heat and maintain water at the set temperature. An optional computer attached via a serial port captures debugging and temperature logging data. The firmeware source is open source and is available on GitHib. A video overview of the project is here.


NXP sent a free LPC800 Mini Kit board a few weeks ago.  As part of the promotion they are running a "LPC800 simplicity challenge"... for weird and wacky apps using the LPC810 (open until end August 2013). This is my entry for this competition.

The LPC810 comes in a 8 pin DIP package of which two pins are reserved for power and ground. It costs about $1 in single quantities and that price drops to less than half in large quantities. The LPC800 Mini Kit board features this LPC810 but also a side connector for a serial port, two micro switches and a blue LED as well as some prototyping space.

I've been experimenting with sous vide cooking recently. This is cooking technique were meats and vegetables are sealed in a vacuum packed bag and slow-cooked in a bath of hot water at a very carefully controlled temperature. In this application temperature accuracy and control is paramount. For example: when cooking beef steak, the difference between a rare and a well done steak is just 5°C (10°F) (from 55°C to 60°C). Moreover if the water temperature is under 50°C (120°F) bacteria can multiply raising food safety risks.

Any appliance that heats water electrically can (in theory) be converted into a sous vide cooker. My choice was a 400W rice cooker which was on special at Lidl a few weeks ago (for €20?). The sous vide controller measures the temperature of the water and toggles the state of the heating element to precisely (and accurately) control the water temperature.

I used a hacked mains timer unit for the element control. I removed the user interface components of the device and broke out the low voltage relay control line. I've written about this in a previous blog post. For electrical safety I electrically isolated this with a Lite-On LTV-817S opto-isolator.

My first attempt at water proofing a DS18B20
temperature sensor.
For temperature sensing I used a DS18B20 digital temperature sensor. This is a three lead device (TO-92 package) which requires just one digital IO line to operate (the other to leads are for power and ground). The choice of a digital sensor avoids the need for analog to digital conversion (the LPC810 has no regular ADC capability). Water proofed probes using this sensor exist and I have one on order, but I'm not expecting it to arrive for a few weeks. So I improvised my own water proofed probe.

My first attempt involved a small section of PVC tubing. I injected silicon bathroom sealant into about 5cm (2 inch) of tube and then shoved in the sensor soldered to a 1 meter length of cabling. It's important that all the exposed metal is enclosed in the sealant. Failure to do so will result in the rapid corrosion of the sensors leads due to electro-chemical effects and a failure of the sensor in a matter of hours.

The water proofing worked, but the performance was terrible. Its performance was measured by allowing the sensor to cool to room temperature and then dunking it in hot water. The resulting exponential curve as a result of the step function of being dropped into water reveals the sensors 'time constant'. Using curve fitting this was found to be 67 seconds.

My second attempt at making a water proof DS18B20 involved soldering it to 3 wires of ribbon cable, then dunking that into a tube of silicone sealant so that the sensor and the exposed metal was well covered. I cut of the sealant at the very top of the sensor so that the top of the DS18B20 package was exposed directly to water. This worked much better with step time constant of just 13 seconds.

User interface

I was very limited in what I can do for the user interface (UI)  because the LPC800 Mini Kit has just one user LED (blue) and two microswitch. One switch I need for reset,  so that leaves just one switch and LED available. I could of course add extra hardware communicating SPI or I2C, but for this challenge I wanted to restrict myself to only the hardware available on the LPC800 Mini Kit. Also with just 4kBytes of program memory, there is little space to implement a complicated UI.

My temperature range of interest is from 55°C to 60°C for cooking beef steak. So I took advantage of that narrow range in the UI design. The operation is actually quite simple: After reset the user presses the switch marked "ISP" once for each degree over 54°C. So, for example, for a set temperature of 57°C three button presses are required. After a brief pause the LED will blink back the selection.

At any time during normal operation the user can query the temperature of the water by pressing the ISP button. The temperature will be read back by blinking the LED. There will be two sets of blinks with a pause between them. The first set reads out the temperature and the second set the units. So 54°C will be 5 blinks, pause, then 4 blinks. A zero digit is represented by one very short blink.

Temperature Logging

Debugging information and temperature logging data is sent using the LPC810's built in UART (serial port). An optional connected computer can then be used to chart the data in real time. The serial port is also used to re-flash the firmware. Temperature can be charted using gnuplot using a simple script like this: [insert]

Temperature Control Algorithm

This is still a work in progress. PID (proportional, integral, differential) control is usually used in these applications. However I found that it was difficult to avoid an initial overshoot. So I started to experiment with a two-phase approach. The first phase is the warm up: a test heating with the element on for 90 seconds and measuring the temperature change as result of this. The remaining time can then be calculated to bring the water right up to temperature. The second phase hands over control to a PID algorithm.

An interesting limitation of the Cortex M0 / M0+ MCUs is the lack of hardware floating point support. With just 4kBytes of program memory linking in float point library is not an option.  So I've used fixed point to implement the control algorithm.

The firmware

I'm using LPCXpresso, although the code can also be compiled with the GCC ARM compiler for bare metal systems. The firmware is open source and is available here: https://github.com/jdesbonnet/LPC810_SousVide. It compiles to about 4kBytes, unfortunately leaving very little room for enhancement.

Power requirements

So far I've been using the serial cable linking it to the laptop to power the controller. However this could easily work off a battery.  The ARM Cortext M0+ is designed for super low power applications and in this application the unit spends much of it's time idle. The UI requires practically no power at all (the odd LED blink).  In the program space available I've attempted to apply simple measures to reduce power consumption. For example all timing, delays etc is accomplished with the SysTick timer and anytime the program needs to wait for an event I invoke the WFI (wait for interrupt) instruction. I am not using deep sleep / power down modes at this time.


The LPC800 Mini Kit makes a successful sous vide cooker controller. The user interface is basic but is simple to use and understand. There is just enough program space and plenty of SRAM to implement the UI, control loop and serial port logging.

Project risk assessment and disclaimer

There are two risks to be aware of if attempting a similar project: first there is a potential shock hazard in designing a switching unit to control the heating element. The second is a food safety: cooking meats at too low temperature (<50°C) will result in bacteria growth and can cause food poisoning. There are many of sources of information on sous vide cooking on the internet so do your research! I would recommend validating the set temperature with a second thermometer until the firmware has proved reliable. If you undertake this project, I take no responsibility for your actions.