TL;DR: Pre-configured git repo. Clone it and run ndk-build.

In an earlier post I described that power consumption is our main issue with Gonzo. We need the camera to run for a month or so on a single charge, and everything you do with the camera eats power away. A major cost in terms of battery life is uploading the photos that the camera takes. The photos that we take are 640x480, encoded as JPEG at 70% quality. That gives an average file size of a bit over 50 KB, and that takes 13 seconds to upload over a 2G connection. If we can get that down, that means less transmission time, and thus less power consumption.

There has been quite some fuss over new image formats that should compress better, of which BPG looked very promising. Unfortunately it’s built on top of x265, and licensing issues are withholding me from using it in any production system. Next thing: WebP, Googles image format based on VP8.

Mozilla thought about better compression too, and created a separate project named MozJPEG that gets better results at encoding images than Firefox’s default library libjpeg-turbo. Note that this won’t replace the default JPEG library in Firefox, but is something especially for encoding images. However, when I was working on the issue MozJPEG 3 was not out yet, and I didn’t look at how much improvement could be made over libjpeg-turbo.

As Gonzo is binary compatible with Android, and libwebp ships with an Android.mk file, it was quite easy to compile the project through the Android NDK.

$ ln -s $PWD $PWD/jni
$ ndk-build

As you can see I have no clue how to compile on Android without a /jni folder, so I just symlink to the current dir and all seems to be fine. Don’t try this at home.

This generates two ARM compatible binaries of cwebp and dwebp (encoder and decoder). Now when pushing the cwebp binary to Gonzo (through normal adb push) I could not encode any images:

PNG support not compiled. Please install the libpng development package before building.

I asked a question about this on the webp mailing list but no ready answer was present. First thing I was wondering about was why the compiler did not complain about the lack of libpng and libjpeg, until I realized that in Android.mk the flags -DWEBP_HAVE_PNG and -DWEBP_HAVE_JPEG were missing. Adding the following line helps (delete obj/ directory first):

WEBP_CFLAGS += -DWEBP_HAVE_PNG -DWEBP_HAVE_JPEG

And gives me a useful error message:

jni/examples/jpegdec.c:21:21: fatal error: jpeglib.h: No such file or directory
#include <jpeglib.h>

While I was doing this I didn’t realize I could change Android.mk to compile libpng & libjpeg as static libraries and then use them later on from one file, as is done for the example_util library in libwebp/examples/Android.mk, so instead I:

  • Added libjpeg, libpng and zlib to my source directory
  • Added their .c files to the LOCAL_FILES in Android.mk (don’t add everything, quite some files don’t compile on Android)
  • Made relative links to the header files instead of libraries (f.e. jpegdec.c)

Now running ndk-build generates binaries that can also encode PNG and JPEG on Android. Victory!

It’s also a victory for Gonzo. We now encode images in WebP @ 60% and have acceptable image quality at 20-30 KB per photo. That shaves off half our transmission time (and adds 1 second for encoding) and saves us around 1,000 mAh in battery capacity!

As described in an earlier blog post we’re using Firefox OS as the OS for Gonzo, our wireless camera. The big advantage is the great set of built-in APIs that we can use to leverage functionality around dialing/SMS/camera/power management. However there are a couple of things we missed from the platform.

Because the way Firefox OS is architectured all user code, like the actual code that powers Gonzo, runs in a web context. That means that the code is isolated in a sandbox and we cannot execute code outside of the sandbox or outside our privilege level. For IoT or embedded purposes that is actually quite a shame, as all the functionality that you want to use needs to be built into Gecko (the JS engine that powers Firefox & Firefox OS). For normal developers this is actually great. If I want to monitor the value of the proximity sensor it is a lot easier to write:

window.addEventListener('deviceproximity', function(e) {
  if (e.near) {
    // Do something
  }
});

Rather than reading the file descriptor (or whatever, it’s device specific) that the driver for the proximity sensor opened for you. For instance, the raw reading of the proximity sensor on the Geeksphone Keon is some 200 lines of C++.

The reason why we don’t allow access to this is two fold: first of all it’s complicated, and device specific. No one should bothered with that. The second thing is part of the sandboxing, it should not be allowed for a web process to play around with the underlying file system. Even though this sounds great for anything web related it hinders our usage of Firefox OS as an embedded platform. Things we would like to do from our JavaScript code are for example:

  1. Read and write to the GPIO pins
  2. Execute non-JavaScript code like image encoder

While we could fix the first issue by creating a GPIO JavaScript API and the second one by cross-compiling the encoder through Emscripten, those are not favourable options on a device that has limited power and battery. We need low level access! Because we still want to write the largest part of our code in normal JavaScript we need to write modules that allow us to do File IO and to spawn child processes straight from normal web content. It’s time to add these modules to Gecko for every web application to use!

Reading my bash_history file

Disclaimer: there is a very good reason that websites cannot read arbitrary files or execute processes on your computer. But hey, it’s an embedded device that will only run your own code, and we’re all hackers here, so that’s OK.

Continue reading »

When developing Gonzo, our wireless camera, one of the big questions that pops up is: ‘Why does it run Firefox OS?’. At first it indeed seems like a weird fit, a camera that runs an operating system for mobile phones. An operating system that is even mostly targeted at the developing world. But Firefox OS is a better fit for embedded devices than you might think at first glance.

If we’re talking about the Internet of Things from a prototype / hobby perspective there are two major players that create development kits. Arduino and Raspberry Pi. Both are electronics platforms that allow you to connect sensors, and use the sensor data as input for your applications. While that seems as a great fit for embedded IoT devices, both solutions came out extremely expensive for a camera project like Gonzo. The least we need are a dev kit, a camera, a GSM shield and an accelerometer (leaving the battery out), which translates into the following bill of material.

Module Price
Arduino Uno €20.00
Camera module €25.73 ($31.95)
GSM Shield €69.00
Accelerometer €12.02 ($14.95)
Total €126.75

This shows in other components as well. Adding a decent touch screen to the material list will set us back another $144.95. When you think of this it’s actually completely crazy. Why do I need to shell out ~126 euros on Arduino modules while I can get a full smartphone with processor, memory, touchscreen, GSM, bluetooth, WiFi and accelerometer for… TWENTY-FOUR DOLLARS. That’s right. $24 retail price for a full smartphone off contract. And even better, it runs Firefox OS. If you look at the architecture of a phone like this (actually any Firefox OS phone), it looks like this:

Continue reading »

Yes, a live feed from the Amsterdam canals through Gonzo, for everyone to admire. Actually the moment we speak the sun is setting, and Gonzo is taking a photo every 30 seconds to capture that magic moment. To see the live feed from Gonzo in your browser: click here.

Photo taken at 16:26 (GMT+1)

Continue reading »

For the Gonzo exploratory engineering project I’ve been working on the backend receiving data as well as the web interface where said data can be viewed. Compared with the other parts of this project which were replacing the system app on Firefox OS with a camera and 3D printing enclosures for a new kind of device I definitely got the most straightforward one. However, since we didn’t have enough unknowns on this project I’ve jumped on every emerging web fad and gone down every rabbit hole I’ve encountered on my way. It’s been great fun and it’s made me think a lot about where Web UI is heading.

Continue reading »