Offline YouTube videos on your Android mobile device

So you going somewhere with time to kill, but with slow internet, or no internet. There are many YouTube videos you'd like to watch.... but can't.

It's a common problem while travelling or on vacation. There are many solutions. Here is my solution. This is for Linux people with an Android mobile device who are running a Linux computer somewhere (home, office, in the cloud, or a Raspberry Pi).  I'll refer to this computer as the 'server'.

So the idea is I use Bit Torrent Sync to communicate between the mobile device and the server. BTSync is handy because it's peer-to-peer and doesn't require any firewall configuration.

1. Install Bit Torrent Sync on the server and run it (you'll probably want to configure the server to start this automatically on boot).
2. Create a directory and generate a shared key for the directory. 
3. Install Bit Torrent Sync for Android on the mobile device
4. Create a directory and assign the key created in step 2. You can do this using QR codes.
5. Install a simple text editor app on the Android device if you don't already have one. I use Jota.
6. Download and install youtube-dl tool on the server if you don't already have it.
7. Download the youtube_fetch_daemon.sh script. Review it and satisfy yourself it does what you expect it to (it's less than a screen of code). Then run it and let it continue to run. Or set it up to start on boot.

So let's say you're in an airport waiting to board. You have access to moderate WiFi, but would like to download a few videos to watch on the flight. This is how to do it:

• Navigate to the video in the YouTube app.
• From the menu chose 'Copy URL'.
• In the Android text editor paste the URL and save the file as a .txt file in the directory created in step 4. This file will now transfer to your server and will cause that YouTube video to download.
• After allowing a suitable amount of time (depending on the quality of the connection on the server and the size of the video), use the BTSync app to view remote files. If you see the video there, touch that video and it will start to download to your device.
• Use your video player to play off local storage.

Edits:

28 July 2013: Link to youtube_fetch_daemon.sh script fixed.

The effect of temperature on the efficiency of photovoltaic panels

A few years ago while arguing about the economic viability of photovoltaic (PV) electricity in Ireland I learned that the ambient temperature can have a significant impact on the performance of the PV panels and it is an important variable in return-on-investment calculations. The lower the temperature the more efficient the panels. [1]

Today we have a rare heat wave in Ireland, so I thought it was a good time to see this effect for myself. I recorded the experiment on video. What I've got here are 2 x 15 watt peak (rated) panels charging a 40Ah sealed lead acid (SLA) battery and a Raspberry Pi [2]

Conclusion

At the start the panels were reading 55°C (read from a non-contact thermometer) and were delivering 710mA of current to the battery/Pi system. After pouring water to cool the panels the temperature dropped to 42°C and the current rose to 820mA. I'm going to use current as a measure of efficiency here [3]. So that's about 14% efficiency increase for a 13°C drop in temperature.

Footnotes

[1] http://pveducation.org/pvcdrom/solar-cell-operation/effect-of-temperature

[2] It's a Raspberry Pi Model A (the one without ethernet or inbuilt USB hub). I've got an external unpowered hub connected to the model A's sole USB socket and to that a WiPi WiFi dongle and a Arduino Leonardo.  A 12/5V DC/DC car converter down converts the 12V to the 5V required by the Pi. The current draw from the Pi at the 12V end of the converter is 210mA.

[3] Measuring and extracting power from photovoltaics is not a trivial topic.

Solar Pi

2 x amorphous silicon 15Wp panels
(Make/brand is IPC Global Battery Saver Pro)

About 10 years ago when Maplin opened a local store, I picked up some solar PV panels, a 40Ah sealed lead-acid (SLA) battery and a charge regulator on special. For the past 10 years these panels have been sitting in a window in my guest room doing nothing but keeping my battery nicely charged.

Finally, I've put this setup to some better use. I'm experimenting with the idea of keeping a Raspberry Pi powered 24/7 using nothing but PV panels and a battery. I'm based in Galway, Ireland (53N, 9W) and here we don't get too much direct sun (clouds keep getting in the way :)... so this goal isn't completely trivial.

My setup so far: 2 x 15W-peak amorphous Silicon panels (300mm x 1000mm) usually laid flat on the ground. A 12V lead acid battery charge regulator, a 40Ah SLA battery. The charge regulator's function is to stop charging the battery when the battery reaches 14.2V.

So I have a nice 12V DC power supply from the battery. But the Raspberry Pi requires the standard USB 5V. A linear regulator could do the job but would be at best only 42% efficient (5V/12V). Instead I'm using a low cost USB car charger (listed as a recommended Pi accessory at Farnell / Element 14, SKU 2113630, cost about €5). This is a DC/DC switch mode converter.  I don't have efficiency figures for this, but I suspect it's somewhere in the 80% efficiency region.  Update: 2 July 2013: I've measured this to be 73% [1]

Low cost 12V to 5V (USB) DC/DC
converter available from Farnell
(SKU 2113630)

I chose a Model A Pi because it uses significantly less current than the Model B (due to the missing USB hub / ethernet interface chip). The down side is I need a USB hub if I want to connect more than one device (which I do!). I'm using a low cost "Technica" brand USB (unpowered) hub from Tesco. For networking I'm using the WiPi WiFi USB dongle (Farnell SKU 2133900).

I want the Pi to be able periodically log the battery voltage. Unfortunately the Pi doesn't have an ADC, so it's the Arduino (Leonardo) to the rescue, connected to the Pi via the USB hub.  But... the Arduino ADC only handles 0 to 5V, so a voltage divider is used to bring the maximum possible battery voltage (about 14V for 12V lead acid battery) into the 0 to 5V ADC range. I'm using a 39k/12k divider bringing the 0 to14V range down to 0 to 3.3V (I had originally planed to use a 3.3V logic board you see!).

My experimental setup. The yellow device (top right) is the battery charge regulator. The PV panels are outside in the garden.

Note there is a trade off in the choice of resistor values: you want them much lower than the impedance of the ADC input, but remember that current continuously flows through this divider, so if you make them two low you'll drain the battery! In my case the current is V/R = 12V/(39k+12k) = 0.24mA: an insignificant line item in the power budget. And those values are low enough for use with the Arduino [2] [3].

My original strategy was to take one ADC value and echoed it to the serial port, pause for a few seconds and loop. But I found the data was very noisy. So instead I wrote this Arduino sketch:

void setup() {
  Serial.begin(9600);
}

void loop() {
  uint64_t a=0; // Need 64 bits due to size of sum
  int i, n = 1<<14;

  for (i = 0; i < n; i++) {
    a+=analogRead(A0);
  }

  Serial.print((int)(a>>10));
  Serial.print ("\n");
}


So I sum 16384 (2^14) ADC samples, but divide by 1024 (2^10). This is ADC oversampling and not only smooths out the noise in the signal, but gives me extra bits of ADC resolution! [4]

So my battery voltage under load (Vbat) vs time chart looks like this:

The goal here is to always keep the battery voltage under load over 12.1V (which I'm estimating to be about the 50% discharge point). [5]

For the first two days I had the panels indoors against the window. I was losing about 0.1V at the end of each day: clearly not sustainable. So I moved the panels outdoors, and this looks a little better. On days 19 June to 23 June (cloudy days with a little hazy sun) I'm losing 0.05V/day. But 24 June and 26 June were sunny and resulted in a net gain in battery voltage.

So the conclusion so far: if we had sunnier weather in Galway this setup would probably provide sustainable 24/7 power to the Raspberry Pi, at least during the summer months when the days are long. But realistically that isn't going to happen. Also in a few months the days will get shorter and the sun lower on the horizon so the available power budget will be greatly diminished. I'm not going to invest in more PV panels, so therefore I need to find ways to shed some load. Hopefully I'll follow this up with a Part 2 in a few weeks.

Footnotes:

[1] I did a quick measurement of the efficiency of the car USB charger adapter using a 10Ω resistor as a dummy load. The input (at 12V) was drawing 244mA and the output USB 5V was drawing 426mA. So the efficiency is (5 * 0.426) / (12 * 0.244) = 73%. When connected the the Raspberry Pi, hub, Arduino and WiPi I measure a mean current of 212mA  entering the 12V side of the DC/DC converter. The (approximate) efficiency of a linear regulator is Vout/Vin.

[2] The Arduino (Uno, Leanardo etc) ADC impedance is often quoted as 10k. But that's a worst case scenario where you may be using several different ADC inputs with substantially different voltages. If you're continuously sampling something close to a DC voltage (as is the case with a battery) the impedance is more in the megaohm region.
http://forum.arduino.cc/index.php?&topic=116491.msg876799#msg876799

[3] For ultra low power applications a FET transistor could be used in series with the voltage divider to switch it off when not needed.
http://www.microbuilder.eu/Tutorials/Fundamentals/MeasuringBatteryVoltage.aspx

[4] See this link for an app note on ADC oversampling:
http://www.atmel.com/Images/doc8003.pdf

[5] Information about a 12V lead acid battery:
http://en.wikipedia.org/wiki/Automotive_battery

[6] Note on 'noise' in ADC signal. Actually this is mostly due to power fluctuations in the Pi (and peripherals). I notice a small spike each time I copy the the ADC log file over the network. Or if I run any CPU intensive task. Interestingly, Vbat appears to increase when drawing more current. This is because the additional current causes a small sag in the 5V supplied to the Arduino. The Arduino is using the 5V supply as the reference for the ADC. A small sag in the reference voltage means a higher ADC reading given the same battery voltage.

Other notes:

Similar projects:

• http://pi.qcontinuum.com/project.html

Using a powered USB hub to both expand the USB bus and power the Pi

While the Raspberry Pi is an inexpensive little computer, all the little accessories can add up. For many applications you'll need a powered USB hub to expand the number of devices that can be connected to the Pi. But you may also be able to power the Pi from the same hub, saving you from purchasing a USB charger. It tried this with the 7 port hub available from Farnell (SKU 2115058, €10) and it worked just fine with a Model A Pi. This compares favorably with a dedicated USB charger which typically costs €6 - €10.

Update (14 June 2013): It seems it's not even necessary to use the micro USB cable (at least in the case of this hub).

Update (19 June 2013): I just noticed that the possibility of powering a Pi from a USB hub that back feeds power was mentioned in the release notes for the Revision 2 of the Rapberry Pi hardware. This is made possible by the removal of the USB port poly fuses that were present in Revision 1 hardware. However it does note that the hub must not supply more than 2.5A under fault conditions (presumably because there is no over current protection supplying power this way).

Measuring the alcohol content of homebrew wine

A little over a week ago I started a batch of 3 week wine (my first in about 15 years).  According to the instructions (Cabernet Merlot "California Connoisseur" brand)  most of the fermentation takes place in the first week. But I was surprised that all airlock activity ceased after day 6. I really thought the airlock bubbling would decay exponentially over the 21 days or so.

Checking a sample with a hydrometer the specific gravity was 0.992 which is well in the range of a fully fermented wine. It also smelled and tasted like wine, and after drinking the 100ml sample... felt like fully fermented wine also!

However to gauge alcohol content I needed the pre-fermentation specific gravity reading, which I never bothered to take.

Instead I came across an alternative method:

1. Check that the specific gravity of your tap water is exactly 1.000. If not you will need distilled or rain water.
2. Take a small sample of wine (I used a 100ml graduated cylinder) and measure the specific gravity, SG1. This was 0.992 ± 0.001 in my case.
3. Then transfer to a pot and boil off about half of the liquid. This ensures that all the alcohol is boiled off. I noticed a distinct change in odor after a few minutes of boiling... from a strong alcohol smell to a sweet odor.
4. Allow to cool (hydrometers are normally calibrated for room temperature)
5. Pour back all the liquid back into the graduated cylinder. In my case this amounted to 45ml.
6. Top up with water back to the original 100ml mark.
7. Measure the new specific gravity, SG2. In my case it was 1.008 ± 0.001.

To calculate alcohol content, use the following formula [1] :

SI = (SG2 – SG1)*1000

%ABV = (0.008032927443 * SI^2) + (0.6398537044 * SI) – 0.001184667159

So in my case that's SI = 16 ± 2, %ABV = 12.3% ± 1.8% which is a reassuring result, except the error bar is very large. Must see if I can get myself a narrow range hydrometer and repeat with more accurate specific gravity measurements.

Now, here is the interesting bit for me: where does this formula come from? The source at [1] is vague on that. When I try to derive a formula from first principles, I get %ABV = 100*(SG2-SG1)/0.211 which is also mentioned in [1] and said to be wrong. I'm also suspicious of the large number of significant digits of the coefficients. I'm wondering is this just a curve fit of the table in [2]?

If anyone has any insights, please post a comment below.

Update: after some more googling, I see that my attempt at deriving a formula from first principles was assuming that say 80ml water + 20ml ethanol would result in 100ml of liquid. This is not the case with water + ethanol. This may explain the quadratic term.

[1] http://homebrewsake.com/home/2011/01/02/measuring-your-sake-%E2%80%93-part-two-percent-alcohol-by-volume/

[2] http://valleyvintner.com/Tips&Links/MeasuringAlcohol.pdf

Dangerous Prototypes QFP breakout board tips

Today was my second attempt at using the Dangerous Prototypes QFP 0.5mm pitch breakout board to solder  LQFP-64 IC.  My first attempt was messy due to a slight misalignment of the pins. Today's attempt was more successful.

Almost everyone agrees the general workflow is to tack down two opposite corners with a soldering iron and when alignment is verified use your favorite technique to solder the remaining pins in place. But getting the IC perfectly aligned (and keeping it there while tacking the corners) is tricky.

Two things I figured today:

1. The solder mask can act as a physical alignment guide. Put the IC approximately in place and then firmly push down on the IC and attempt to slide it up and left (assuming the normal orientation). You'll find that when the IC's pads reach the solder mask there is resistance to this sliding motion. At this point the IC should be perfectly or nearly perfectly aligned. [Update 4 Mar 2013: this worked just fine with an Analog Devices ADAS1000 but I just tried this with a smaller 32 pin package and noticed the IC's pads were shorter so this tip didn't work: so this is very much package dependent.]

2. Next while keeping the IC pinched firmly in place I used a washing line peg to clap it in this position. I found the peg applied enough force to keep it reliably positioned, yet it was still possible to nudge it slightly should the alignment need adjustment.

I then tacked two corners, removed the peg and soldered the rest of the pins.

DeLonghi Cafe Treviso (BAR14F-E) Repair

(Note: I wrote this blog post back in February 2013, but for some reason never I published it. Fixing that now.)

In the last week my low cost coffee machine (DeLonghi Cafe Treviso BAR14F-E) clogged up. Turns out (as I suspected) it was a build up of limescale.  I’m writing this short blog post to aid anyone else who might have the same problem. I suspect many appliances like this are dumped and replaced, when a 1-2 hour repair is all that’s needed to prevent more e-junk entering the world’s landfills.

Usual disclaimers apply: this worked for me, if you try it yourself you do so at your own risk. If your device is still in warrantee  it most likely voids it. There are exposed live mains contacts all over the insides of this machine: disconnect before opening and use common sense.

I got some helpful tips from this write up. Not the same model, but there were several useful tips (eg using a regular flathead screwdriver to remove the security screws).

I suggest you photograph everything before disassembling. In that way if you can’t remember what goes where, you can refer to your photos.

Warning: this is a mains appliance. Inside there are many live contacts. Disconnect before opening (I suggest physically unplugging and keep the plug in view so that there is no doubt that it’s unplugged).

I originally hoped I could perform the entire repair by accessing from the top without removing the mechanism, but found during re-assembly I needed to remove the entire mechanism. So best do this from the start.

Remove the water reservoir.

The top and bottom covers use security screws Torx screws (T20H), however it seems a regular 2.5mm flat head will get sufficient purchase to loosen without causing any damage to the screws.

Remove the bottom cover. Remove the mains lead retaining clamp from the external case by sliding it down. This will facilitate separation of the mechanism from the case.

Remove the top cover. You will need to remove the screw covers (use a knife to pry open). You will also need to remove the steam valve by pulling upward (moderate force is required).

Remove the steam hose from the pressure vessel by clenching the retaining clip with a pliers.

Before this next step carefully note the positions of the electrical contacts to the switches (eg take a photo). Remove the contacts from the switches (some of the spade connectors have a tiny clip which you need to depress to remove). Remove the switches and the element indicator neon (that took a little bit of fiddling).

Remove 3 x cross head screws which secure the pressure metal chassis to the plastic outer case. At this point it should be possible to separate the mechanism from the case.

Remove the 4 x hex bolts which fasten both halves of the pressure vessel and secures it to the chassis. You should now be able to separate the top and bottom half of the pressure vessel. Clean out. Also clean the flow valve as described in Step 19 in this write up.

Reassembly is the reverse of what was just described. Take care not to damage the rubber seal between the two halves of the pressure vessel. If it gets dirty clean it before reassembling. Make sure the seal is good and tight.

Misc notes: There was a manky fiberglass wool like thermal insulator keeping the incoming mains wires separated from the walls of the pressure vessel (which I’m sure gets quite hot). Probably because of it’s age it was falling apart and quite dirty. I thought about using some cardboard instead, but because of the temperature and live voltage operating within mm of this I thought it was too much of a fire hazard. So I opted to leave out that insulator when I reassembled. I’m not sure this is a good idea, I guess I’ll find out soon.

Saving streams from the TG4 player

I notice that howto guides on doing this seems to change frequently. This worked for me in early January 2013 and will probably be out of date soon. I use Linux... but I'm sure these instructions can be adapted for other OSes.

You will need rtmpdump.

Open a decent modern browser (eg Chrome or Firefox) and open the developer tools. Monitor HTTP traffic with the tools while you open the program you wish to record. When the program starts you will see several GET and POST requests in the background. Look for a request which features a fragment that looks something like this:


mp4:videos/1290862567001/1290862567001_2032327037001_WCL016084-14-4.mp4

This will form the -y parameter of the rtmpdump command.

Also look for traffic going to a host that looks like cp156323.edgefcs.net. The host name part of that address may vary. Match the host name in the -r parameter to this host name.

 Now try this (making the adjustments mentioned above):


rtmpdump -r "rtmpe://cp156323.edgefcs.net/ondemand"  \
-y "mp4:videos/1290862567001/1290862567001_2032327037001_WCL016084-14-4.mp4" \
-o myvideo.flv

These instructions may also apply to the RTE player, but I haven't tried it.

Plotting a weight loss curve from CanOfSleep's WWDiary app

I've been using the Android app WWDiary published by CanOfSleep.com as a food / weight diary. And it works! ... for me anyhow. Another few kg to go and I should be at my goal early in 2013. It's always encouraging to see your long term progress plotted visually. Unfortunately this app has no plotting capability so I wrote a short GnuPlot script to do this.

I manually transcribed the values from the weight diary into file 'weight.dat' with a format that looks like this:


20121226 74.0
20121227 74.6
20121228 73.4
20121229 73.1
20121230 74.1
20121231 74.9

And the following GnuPlot script generates the above chart:

set xdata time

set timefmt "%Y%m%d"

set format x "%d/%m"

set title "Weight loss curve 2012"

set xlabel "Time"

set ylabel "Mass (kg)"

set term png size 800,400

set output "weight.png"

plot 'weight.dat' using 1:2 with linespoints

Updates:

10 June 2014: I've got an updated howto on doing this which includes extracting the data automatically from the wwdiary database file. See http://jdesbonnet.blogspot.ie/2014/06/weight-charts-from-wwdiary-part-1.html

Change of blogger template makes a big difference to traffic

Update (27 June 2013): No I was wrong: that increase in traffic is some weird artifact of that template (a template which I quickly grew to hate and reverted back to something simple a few days ago).

I was experimenting with blog templates last week. I changed my template to one of the new Blogger dynamic templates.  It looked better than the last one, but I didn't think it would make any practical difference.

I was surprised to see this week that my traffic numbers have soared over 300% on last week. Initially I thought I got some coverage at a popular blog or website, but there was no specific traffic source or destination page. I also noticed this surge traffic started on the same day I changed the template.

So there you go... one click and an instant 300% traffic increase!

Unfortunately I didn't have any screen shots of the blog as it was before, and I was unable to find the exact same template again, but this approximately what it looked like before (left) and after (right).