Carburettor synchronization

I'm a happy owner of Yamaha XJR1300 which I still consider to be the most beautiful bike. Yeah sure it is not a beast one would take to challenge R1 or CBR1000 on the track days but it certainly is a machine that will take you hundreds of miles in the seat letting you to enjoy the ride without any back or wrist pain and still agile enough to put your knee down if you like.

Some time ago I found subtle vibrations noticeable at around 4000rpm clearly a sign of dis-balanced power delivered by the four cylinder engine. Well, it was a time to synchronize the carburettors. A few year back I went to a certified Yamaha dealer in London to do the synchronization but they said it was not necessary but after a few rounds of convincing talks they did it with a good result. The only thing was I had to wait a week or so for the appointment and then a day for the job to be done. Finally they charged me about £80 which was a bit expensive adventure for what it was. This time I decided to do it myself since I already had quite a bit of experience with engines and carburettors tuning. The only thing I didn't have was the carburettor synchronizer so I decided to make one. Here is the story.

Synchronize carburettors? Why?

For multi cylinder combustion engine, in order to run smoothly, all cylinders need to deliver the same amount of power regardless of the rpms or load. If one produces more or less power than the rest the engine will vibrate and won't run well. This will be noticeable mainly in the idle where the engine will keep slowing down and speeding up, just about staying alive. Some workshops solve this by increasing the idle rpms which is just like giving more painkillers to the sick person instead of treating him. There are many things that influence the power delivered by each cylinder such as good spark plug, ignition timing, valves timing and clearance, correct exhaust back pressure, good piston rings and, of course, the carburettors. If one of these is not the same (good or bad) for each cylinder they won't work together and the engine won't like you and make you suffer by vibrating your body in the seat. For now I'm gonna focus only on the carburettors.

Single vs. multi carburettors engines

There are many two, four and more cylinders engines with only one carburettor or a fuel injection with a single throttle valve. This works well and there is nothing to synchronize (considering we are not going down the route of the fuel injectors synchronization which is a bit different story).

Single throttle body (single point fuel injection)

Four carburettors body

So why to trouble with multiple carbs? Well there are basically two reasons. Usually the main one is to provide a larger volume airflow for each cylinder with as laminar airflow as possible to improve the engine performance and efficiency. If the intake manifold has bends the mixture will be hitting the walls, creating turbulences and the fuel gets washed down as it travels to the cylinder. Also the cylinders closer to the central carburettor get richer mixture since the path through the intake manifold is shorter and without any bends. As you can see in the image above some manufactures make all intake pipes the same length which in result creates more complex path for the mixture thus increases the turbulences which is not good. If we use one carburettor or throttle body for each cylinder we make sure the fuel gets diffused evenly without splashing on the walls reaching the cylinder along the shortest possible path.  We can also easily increase diameter of the intake pipe thus improving the engine performance.

The second reason is to compact the intake system into as small area as possible while preserving a reasonable performance which is usually the case of motorcycle engines. There is not enough space on the motorbike to run an intake manifold that would with reasonable efficiency distribute the fuel mixture to all cylinders. Maybe two but almost never for more.

How to measure the sync.

To synchronize the carburettors or multiple throttle valves in the case of the fuel injection system we need to measure power delivered by each cylinder independently and then have a way of adjusting this power per cylinder.

We can find the power of each cylinder by measuring the vacuum in its intake manifold. On the first sight this may sound a bit puzzling but if you think about it you realize it's actually a common sense. To be precise we won't find the power per say in for example kW or Hp but rather a pressure in kPa (or psi or mHg) that relates to the power produced by the cylinder. This pressures is lower then the atmospheric one and is created by the piston sucking air into the cylinder during the induction cycle. On the other hand the throttle valve is blocking the intake manifold thus making it harder for the piston to suck more air which creates vacuum in the manifold. Other contributing factors are:

• The exhaust back pressure.
The residual pressure of the exhaust gases that stay in the combustion chamber after the exhaust cycle decreases the vacuum the piston can produce during the induction cycle, consequently it also decreases the vacuum (or increases pressure) in the intake manifold.
• Energy produced by the mixture burning.
If the mixture is burning well and quickly the residual pressure of the exhaust gasses will be lower than if it's burning slowly which can curry on even during the exhaust cycle. Again different burning precess will be manifested by a difference in the intake vacuum.
• Ignition and valve timing, quality of the piston rings, how well the valves seals etc. also influences the vacuum in the intake.

Basically everything that has some impact on the residual pressure of the exhaust in the combustion chamber will manifest itself as a change of the vacuum in the intake manifold.

In our case, to sync. the carbs, we don't need to really know even the exact pressure in the intakes, what we need is just to compare one cylinder with another and make sure there is the same pressure in all intakes.

The tools

There is lot to chose from. Analog vacuum gauges, mercury manometers called carb sticks or digital manometers. To save time and money I bought a cheap set of four analog gauges on the eBay for about £13. They proved to be more expensive than I hoped in the time I spent trying to set them up and use them. If you don't want to end up with extra grey hair go for other tool. The problem is that these gauges don't have any integration chamber and transfer the pressure changes straight to the needle movement. As the engine is running the needs is oscillating all over the dial making almost impossible to get any reasonable reading. There is a small dumping screw on each pipe for each gauge that reduces the diameter of the pipe virtually making the manometer itself an integrator. The problem is that it doesn't work well and the needle is either absolutely still or dreadfully lagging or shaking all over the dial. Of course it depends on the type of the gauges so I'm convinced if you get more expensive ones you will be ok. In any case don't buy the ones in the picture below.

These analog vacuum gauges are very hard to use

If you decide to buy your own manometer I would recommend the digital Tecmate Carbmate for $125.99 This electronic manometer has only two inputs which is absolutely fine and can be used to sync even, three, four or more cylinders engines as I will describe in a minute. Other good choice is the Morgan Carbtune for £57. This is quite popular and reasonably cheap and accurate tool. It works straight for four cylinders and yes you can use it for two cylinder engine as well, just connect two inputs only. I would probably avoid the nowadays obsolete mercury manometers or other with liquid inside that can get sucked into your engine if you are not carefully enough. That happens quite easily when you quickly shut the throttle which creates high vacuum in the intakes and if they are a bit out off sync one of them will drink up all mercury from you manometer. This also happens if you forget to connect one manometer input. In this case the mercury, or other liquid, disappears as soon as you touch the starter button. Luckily it never happened to me 🙂 Since I like to make things I decided to build my own, simple, differential carburettor synchronizer. If you have a bit of spare time and some basic workshop I would suggest to go this way, you will end up with as easy to use and accurate tool as the commercial ones. The differential manometer works by comparing pressure above the surface of some liquid in two tanks connected at the bottom. This is not by any means a new idea and you can find many commercial or home made devices of this type all over the internet. In my case I used four old 60ml syringes where two pairs are connected together by the opposite ends as you can see in the image below. Testing the manometer by artificially reducing pressure in the left test tube These two syringe-tubes are then joined together at the bottom and mounted on a solid piece of wood for support and filled with a liquid. It's important to use volume not larger then is about 3/4 of one of the tube. The reason behind that is that if during the tuning process one of the cylinders moves all liquid to one of the tubes it won't reach the input pipe at the top and get into the engine. For higher contrast I used a blue non toxic liquid from an old cooler box. Apart from the high contrast colour this liquid also has a high viscosity which makes it nicely stable while running the engine. There are more pictures from the building process of this tool at the end of this article. Lets do it! The synchronization process is very easy. Basically the steps are: 1. Warm up your bike and prepare your tools. Going for a 10 min ride should be enough. Apart from the manometer you will most likely need some auxiliary petrol tank. I'm using a modified plastic bottle with a silicone house. Auxiliary fuel tank Another think that comes in handy is a fan to keep your engine cool while the bike is not moving. Other tools that you will need depends on the type of your bike. For the XJR1300 you just need about 30cm long cross slot screw driver and that's it. 2. Connect the manometer to the intake manifolds. Again this varies bike from bike. On the Yamaha XJR1300 there are already small brass tubes mounted in the inlet rubber boots ready for this purpose. 3. Find the throttle flap linkage. This is also different on each bike but it's quite easy to find. On two cylinder engines like the Kawasaki 250R the throttle valves are connected by single screw that changes relative position of the valves. This is the only screw you will use to do the synchronization. Kawasaki 250R throttle body For more cylinder engines the carburettors (or throttle valves of fuel injection systems) are usually synchronized in pairs first and then the two pairs of cylinders together. [the above picture is a courtesy of Ben check out his blog] Pick any pair you like and while running the engine in the idle make sure the vacuum in the intakes is the same. With the differential manometer set the ratio of the flaps as such the liquid is not moving form one tank to another. It doesn't matter what level is in each tank they just must stay the same. Rev the engine to 2-3k rpm a few times and make sure the carbs are still in balance. If you have a two cylinders machine you finished. If you have four cylinders synchronize the second pair and then the two pairs together exactly the same way as you did before. If your manometer has only two inputs you will need to stop the engine and reconnect the manometer to the second pair. To sync the cylinder pair connect one input of the manometer to one cylinder in the first pair and another one in the second pair. It doesn't matter which cylinders in the pairs you choose. And that's it, job done. As you are balancing the carburettors you will instantly notice the engine is running more easily and has more harmonic sound. It will also start much better and will keep just humming on rock stable rpms in the idle. You should also notice no vibrations while riding at any revs. And now it's time to go for a test flight 🙂 Happy and safe riding.... Appendix - making the differential manometer Here are some pictures from building the device. They are self explanatory so no need to comment to much. The syringes are glued together by an ordinary gasket seal and fastened by brackets made of copper-clad laminate (PCB) Of course any reasonably strong material is ok for the purpose. An now just pour some nice colour liquid in and the manometer is alive 🙂 Posted in Motorbikes | Leave a comment Real-time visualization of electric power consumption Yesterday I have showed how to read power consumption with a cheap power monitor device. Today I'm gonna put together very basic server and client web applications to store the power usage information in a DB and visualize them in a webpage. Overall design Since a picture is worth a thousand words here is the design. It cannot really get any simpler 🙂 First of all we need a web server with installed Apache, PHP and MySQL. The second step is to create a database with a single table. We will need only these fields: • id - To keep order in our data • ts - A timestamp where the data was stored. • power - Instantaneous power usage in Watts. The table structure in SQL is this: -- -- Table structure for table elusage -- CREATE TABLE IF NOT EXISTS elusage ( id int(11) NOT NULL AUTO_INCREMENT, ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, power int(11) NOT NULL, PRIMARY KEY (id) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1145941; Pushing data on the server Here we need to implement two things, a publisher that will run on your computer at home and the consumer running on the web server. We will push the data on the server by http GET parameter. To make sure no one else will mess up with our data we add a secret token as another GET parameter, something like this: http://www.server.com/el/power.php?token=a5K4Dli8L0&add=123 The number 123 is the power in Watts that our script needs to add to the request. To do the http GET request from Python is very easy, especially if we use a module like the requests. This is how we need to modify our Python application. import requests . . def run(uart_port, webapp="", period_sec=4): # Open the serial line device = serial_for_url(uart_port, timeout=1) # Initialize the device data = exec_cmd(device, START_CMD) print "Startup response: ", print data # Keep reading the power consumption infinitely while True: power = read_power(device) print str(power) + " Watts ", # Sometimes the power meter returns 0 Watts. This can happen if # the power meter itself fails to read data from the main unit over the RF link. if power == 0: print " Ignoring *********************************" continue # Upload on the server if possible if webapp != "": try: result = requests.get(webapp+str(power)) print "Update request: "+str(result.status_code) except Exception as e: print "Connection error: ", e except: print "Unknown exception: ", sys.exc_info()[0] # Pause for 'period' seconds time.sleep(period_sec) device.close() if __name__=="__main__": uart_port = "/dev/ttyUSB1" webapp = 'http://www.server.com/el/power.php?token=a5K4Dli8L0&add=' run(uart_port, webapp)  The full code is available from http://code.google.com/p/electroncastle/source/browse/el_pwr_monitor/meter/elmeter.py Server update script The PHP application is even easier, it receives the http request, verifies the token and adds the data to the DB. This is part of the power.php that is responsible just for that. <?php function connect_db() { // Connects to the database mysql_connect("localhost", "elmonitor_app", "db_password") or die(mysql_error()); mysql_select_db("elmonitor_app") or die(mysql_error()); } function updatePower($val)
{
connect_db();
$result = "INSERT INTO elusage (power) VALUES (".$val.")";
mysql_query($result); } if ($_GET["token"] == "a5K4Dli8L0"){
$val =$_GET["add"];
if ($val != "") updatePower($val);
}

?>


The webpage code

To do a real time visualization on a webpage we need to implement an AJAX script that will periodically query the web server for latest data and the PHP code that does this job on the server. To graph the data I'm using freely available library Highcharts This takes care about most of the work, we only need to pre-fetch enough data to show a graph for required time interval and then periodically pull for the latest power usage data. This is written in Javascript. The complete code including the embedded Highcharts you can view here:

On the server site, in the power.php, is the bellow attached code. The important functions are getLast($minutes) that returns all samples for last$minutes and getNow() which returns only the last sample. The data is encoded in JSON where each sample is a two items array.

[timestamp, power]

The timestamp is a unix timestamp (The unix timestamp is the number of seconds since January 1 1970 00:00:00 UTC) in milliseconds and the power is the usage in Watts.


<?php

function connect_db()
{
// Connects to the database
mysql_select_db("elmonitor_app") or die(mysql_error());
}

function showUsageTable()
{
echo "<a href=index.php>Realtime graph</a><br>";
echo "<center>Electricity consumption</center><br>";
connect_db();

result = mysql_query("SELECT * FROM elusage WHERE ts >= CURRENT_TIMESTAMP - INTERVAL 1 HOUR ORDER BY ts DESC") or die (mysql_error()); ?> <table border="1" align=center> <tr><td width=10>Sample</td><td width=200>Date and time</td><td width=100>Power (W)</td></tr> <?php while (row = mysql_fetch_assoc($result)) { echo "<tr>"; echo "<td>".$row['id']."</td>";
echo "<td>".$row['ts']."</td>"; echo "<td>".$row['power']."</td>";
echo "</tr>";
}
?>
</table>
<?php
}

function getRange($from,$to)
{
connect_db();
$sql="SELECT * FROM elusage WHERE ts BETWEEN '${from}' AND '${to}' ORDER BY ts ASC";$result = mysql_query($sql) or die (mysql_error());$ar=array();
while ($row = mysql_fetch_assoc($result)) {
$pwr=intval($row['power']);
if ($pwr != 0){ array_push($ar, array(strtotime($row['ts'])*1000,$pwr));
}
}
return $ar; } function getRangeJsn($from, $to) { header("Content-type: text/json");$ar = getRange($from,$to);
echo json_encode($ar); } function getLast($minutes)
{
$tm = time();$from = date('Y-m-d H:i:s', $tm -$minutes*60);
$to = date('Y-m-d H:i:s',$tm);
return getRangeJsn($from,$to);
}

function getNow()
{
connect_db();

$result = mysql_query("SELECT * FROM elusage ORDER BY ts DESC LIMIT 1") or die (mysql_error());$row = mysql_fetch_assoc($result); if ($row){
// Convert time to unix timestamp in milliseconds
// The unix timestamp is the number of seconds since January 1 1970 00:00:00 UTC
$ret = array(strtotime($row['ts'])*1000, intval($row['power'])); echo json_encode($ret);
}
}

function updatePower($val) { connect_db();$result = "INSERT INTO elusage (power) VALUES (".$val.")";$add_member = mysql_query($result); } // --------------- ENTRY POINT -------------------- foreach($_GET as $key =>$val){
switch ($key){ case "last": getLast($val);
return;
case "now":
getNow();
return;
if ($_GET["token"] == "a5K4Dli8L0") updatePower($val);
return;
break;
}
}

// Without parameters show simple html table with the power consumption history
showUsageTable();
?>


If the power.php is called without any parameters, function showUsageTable() gets executed. This function generates simple html table with the timestamps and the power consumption data.

The complete code for the publisher,  webserver and the client can be downloaded from the electroncastle repository on the googlecode

If you put everything together you should be rewarded with a nice real time graph looking like this.

If you want to just see it in action check out my house power consumption on address http://fajtl.net/el/ Note this is just a sandbox for experimentation so the link may not be online permanently.

Conclusion

So that's it for today. This application is so simple so it can be put together on the Sunday afternoon just for fun. Besides from the fun It's very interesting to see a power consumption of your house from any place. There is, of course, a huge space for improvements and tons of features in the term of more data analysis, presentation of historical data, improved security etc. The publisher code can be written in any language and run on any computer with USB host (or possibly even RS232) and internet connectivity. Raspberry PI is one of many very cheap computers that can be used for this kind application. Since it is based around CPU for mobile devices it has very low power consumption so it can run 24/7 without any significant running cost.

Many years ago I got a broken Sony VAIO laptop from my friend. It has broken IDE port for HDD but apart from that it's OK. It boots a Slax linux from a CD with no problem. It also has a few USB ports and very very low power consumption without the HDD and the screen off so I decided to use it for this project. This is it in the action 🙂

Electricity power monitor hacking

A while ago, while browsing through a local Maplin store, I found a cheap wireless electricity power meter that is possible to connect to a PC via USB cable. Since I was planning to experiment with the electricity power monitoring for one of my commercial projects anyway, I decide to buy it and find out how to establish a communication with it from a custom piece of software and possibly even present the data on a website with a real time update. The Wireless electricity monitor was, and I believe still is, available for £14.99 from Maplin under code N94KQ This is what you get in the box:

The installation was very simple. The main measurement unit with the transmitter gets clamped on the live phase wire coming to your house. If you have three phase distribution you can buy extra two clamps and connect them to the transmitter. The meter can handle them 🙂 This is how I placed the transmitter next to my main power meter.

Communication with the electricity monitor

After installing the meter at home I realized the provided software for getting data from the unit was only for Widows and pretty rough to say the least. I didn't have any intention to use it anyway but I was quite surprised the manufacture didn't put more effort in it.

To find out how the host computer talks to the meter was quite simple. Plugging the USB cable to my linux machine and running dmesg command quickly revealed what I was dealing with.

jiri@mach:/$dmesg [337148.094624] usb 6-1: new full-speed USB device number 4 using uhci_hcd [337148.257064] pl2303 6-1:1.0: pl2303 converter detected [337148.269252] usb 6-1: pl2303 converter now attached to ttyUSB1 As one would expect the USB cable is a simple RS232 to USB bridge with the very common PL2303 chip. The USB cable is attached to the unit with RJ45 connector with quite large USB connector on the other side. I believe the RS232->USB converter is built in the USB connector itself so the data on the RJ45 should be the raw RS232 but I haven't investigated that. Second step was to find out the communication protocol. I didn't want to do any reverse engineering of the protocol so it tried to find what controller was used in the meter and obtain the protocol from the manufacturer. The product is re-branded for Maplin and to find the manufacturer was not possible. Inside the unit is R5F2L38ACDFP microcontroller but that also didn't help. The fastest approach appeared to be to analyse the data between the unit and the software that came with it. After and hour with a serial port sniffing tool I managed to extract a startup sequence, and two other sequences that need to be sent to the unit to make it to return data with the current power consumption. This is how they look in the python. START_CMD=[0xAA, 0x00, 0x00, 0xAD] READ1_CMD=[0xAA, 0x01, 0x00, 0xAD] READ2_CMD=[0xAA, 0x02, 0x00, 0xAD] So far so good, I can initialize the unit and make it to return a long byte sequence with the power usage information like this. 32000032533033ffff000032533034ffff000032533035ffff000032533036ffff000032533037ffff000032533038ffff000032533039de330d0131533130ffff000032533131ffff000032533132ffff000032533133ffff000032533134ffff000032533135ffff000032533136ffff000032ff How do I know? Well, I recorded several output sequences, each for different power consumption which I controlled by turning on/off my el. kettle 🙂 For each case I wrote down the power usage displayed on the LCD screen of the unit. Then it was just a matter of identifying what bytes changed and determining how they are formatted. The power in Watts is stored in the bytes 69 and 70 in the sequence. This is how to read it in python.  power = data[69] + data[70] * 256 Python application To read the power is very simple. 1. Open the serial port. 2. Write the initialization sequence to the port. 3. Write the READ1_CMD and READ2_CMD sequences. 4. Read the data from the serial line and decode the power. The last two steps can be repeated infinitely Here is a python code to does exactly that. The only dependency is on the pySerial module import sys import time from serial import * START_CMD=[0xAA, 0x00, 0x00, 0xAD] READ1_CMD=[0xAA, 0x01, 0x00, 0xAD] READ2_CMD=[0xAA, 0x02, 0x00, 0xAD] def write_cmd(device, cmd): device.write(to_bytes(cmd)) def read_data(device): data=[] while True: b = device.read() # Read all available bytes if len(b) == 0: break data.append(b) return data def exec_cmd(device, cmd): write_cmd(device, cmd) # The power meeter needs some time to deal with the command time.sleep(0.2) return read_data(device) # Extract the current power consumption in Watts from the data # returned by the power meter def decode_power(data): power = 0 data_len = len(data) if (data_len > 70): power = ord(data[70]) * 256 + ord(data[69]) return power # Returns current power in watts def read_power(device): data = exec_cmd(device, READ1_CMD) data = exec_cmd(device, READ2_CMD) return decode_power(data) def run(uart_port, period_sec=4): # Open the serial line device = serial_for_url(uart_port, timeout=1) # Initialize the device data = exec_cmd(device, START_CMD) print "Startup response: ", print data # Keep reading the power consumption infinitely while True: power = read_power(device) #print str(power) + " Watts ", # Sometimes the power meter returns 0 Watts. This can happen if # the power meter itself fails to read data from the main unit over the RF link. if power == 0: print " Ignoring *********************************" continue # Pause for 'period' seconds time.sleep(period_sec) device.close() if __name__=="__main__": uart_port = "/dev/ttyUSB0" run(uart_port)  The complete code including the dependencies can be downloaded from http://code.google.com/p/electroncastle/source/checkout Given the simplicity one can use any off the shelf hardware with the USB host port and some basic computation capabilities for example the recently very famous Raspberry PI. If the data on the RJ45 is really RS232 then one could use even more basic micro controllers to read the data. Next time I will show how to setup very simple real time visualization of the power consumption in a web page. Posted in Home automation | Leave a comment Building Linux for Pandaboard ES Experimentation with the Skype runtime on the igepv2 board resulted in many days of frustration and endless performance diagnostic leading to a decision to focus on slightly more powerful platform. The Skype audio/video communication works OK on the igepv2 board (OMAP3530) however the CPU is loaded close to 100% which makes the performance monitor in the Skype runtime to keep reducing video framerate and target video bitrate which produces quite unpleasant behaviour. I had a Pandaboard ES lying around and decided to use it for the next experimentation. The Pandaboard ES has OMAP4460 which is dual core running on 1.2GHz so I hope to get some performance boost compare to the igepv2 which is OMAP3530 on 720MHz. I'm using Openembedded (OE) for building custom linux distributions for my embedded projects and the following description will cover just that environment. Configuring openembedded A few years ago I have setup the Angstrom distribution which I'm still using however with significant modifications. The OE setup for the Pandaboard consisted from these steps Environment setup Create a new file with environment variables to configure the OE system. Before building any package with the OE we inject these variables to the system on the command line . I named the file environment-2010-panda export MACHINE=omap4430-panda export DISTRO="angstrom-2010.x" export DISTRO_DIRNAME="angstrom_2010_x" export OE_BUILD_DIR="/attic/angstrom2/build_panda" export OE_BUILD_TMPDIR="\${OE_BUILD_DIR}/tmp-angstrom_2010_x"
export OE_SOURCE_DIR="/attic/angstrom2/sources"
export OE_BASE="/attic/angstrom2/"
export PATH="/attic/angstrom2/sources/bitbake/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin"

export BB_ENV_EXTRAWHITE="MACHINE DISTRO GIT_PROXY_COMMAND ANGSTROMLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS"

export BBPATH="${OE_BUILD_DIR}:/attic/angstrom2/sources/openembedded" Second step was to create the build directory build_panda/conf and file with local configuration build_panda/conf/local.conf with this content # Where to store sources DL_DIR = "/attic/angstrom2/sources/downloads" #INHERIT += "rm_work" # Which files do we want to parse: BBFILES ?= "/attic/angstrom2/sources/openembedded/recipes/*/*.bb" BBMASK = "" # Qemu 0.12.x is giving too much problems recently (2010.05), so disable it for users ENABLE_BINARY_LOCALE_GENERATION = "0" OE_ALLOW_INSECURE_DOWNLOADS="1" # What kind of images do we want? IMAGE_FSTYPES += "tar.bz2" # Make use of SMP: # PARALLEL_MAKE specifies how many concurrent compiler threads are spawned per bitbake process # BB_NUMBER_THREADS specifies how many concurrent bitbake tasks will be run PARALLEL_MAKE = "-j6" BB_NUMBER_THREADS = "4" # Set TMPDIR instead of defaulting it to /tmp TMPDIR = "/attic/angstrom2/build_panda/tmp-angstrom_2010_x" # Don't generate the mirror tarball for SCM repos, the snapshot is enough BB_GENERATE_MIRROR_TARBALLS = "0" CE_VERSION = "latestlatest" TERMCMD =${XTERM_TERMCMD}
TERMCMDRUN = ${XTERM_TERMCMDRUN} And that's basically it. Now we are gonna configure and build the kernel. Adding & building Kernel 3.5.4 in the OE Since the current version of the OE doesn't include last kernel 3.5.4 (as of writing this) I had to add this version to the OE. The only problem was that the module-init-tools-cross_3.12.bb recipe didn't include reference on the depmod-3.5 package. I fixed it by appending virtual/${TARGET_PREFIX}depmod-3.5 to the PROVIDES variable.

PROVIDES += "virtual/${TARGET_PREFIX}depmod virtual/${TARGET_PREFIX}depmod-2.6 virtual/${TARGET_PREFIX}depmod-3.0 virtual/${TARGET_PREFIX}depmod-3.5"

Of course I had to create a new recipe for this version of kernel. There was no surprise here. The recipe looks like this

/recipes/linux/linux-omap4_3.5.4.bb

CORTEXA8FIXUP = "no"
COMPATIBLE_MACHINE = "omap4430-panda"

require linux.inc

DESCRIPTION = "Linux kernel for OMAP processors"
KERNEL_IMAGETYPE = "uImage"

SRC_URI = "http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.5.4.tar.bz2 \
file://defconfig"

PV = "3.5.4"
S = "{WORKDIR}/linux-3.5.4" After creating this recipe we need to configure the kernel. bitbake -c menuconfig virtual/kernel The Pandaboard has two video output (actually three, the third being a composite video but this one is not exposed on the board) HDMI and DVI. The DVI is generated by TFP410 DVI transmitter which is connected on the main display bus that is shared with the LCD display (if connected) The TFP410 is being turned on/off via I2C line. Long story short we need to select the FTP410 panel in the kernel config in order to use the DVI output. CONFIG_PANEL_TFP410=y To configure the audio we need to do more configuration. Audio IO is supported by TWL6040 chip so we need to select the following. CONFIG_TWL6040_CORE=y CONFIG_SND_OMAP_SOC=y CONFIG_SND_OMAP_SOC_DMIC=m CONFIG_SND_OMAP_SOC_MCPDM=m CONFIG_SND_OMAP_SOC_HDMI=m CONFIG_SND_OMAP_SOC_OMAP_ABE_TWL6040=m CONFIG_SND_OMAP_SOC_OMAP_HDMI=m CONFIG_SND_SOC_I2C_AND_SPI=y CONFIG_SND_SOC_DMIC=m CONFIG_SND_SOC_OMAP_HDMI_CODEC=m CONFIG_SND_SOC_TWL6040=m With the kernle 3.5.4 (3.4.2, 3.6-rc2 and most likely other) The following option must be disables, otherwise the audio playback is completely distorted !! # OMAP Feature Selections # # CONFIG_OMAP_SMARTREFLEX is not set # CONFIG_OMAP_RESET_CLOCKS=y Now for the Wifi support select these options. CONFIG_WL12XX=m CONFIG_WLCORE=m CONFIG_WLCORE_SDIO=m The last step is to compile the kernel bitbake -f -c compile virtual/kernel And then force the build, this will update the sysroot and create the kernel package with all modules. bitbake -f -c build virtual/kernel Booting from network with TFTP, PXE & NFS While developing for the embedded devices I like to boot it over network from my dekstop. It's quite nice to have the root filesystem and the kernel on my computer so I can quickly update it and test changes. First thing to do is to setup a NFS and tftp servers and populating the rootfs and boot directories. This is somewhat trivial and there is not much to be said about it. The second think is to create the PXE file with the boot parameters. The PXE configuration file is loaded from the tftp server to the memory on the device. This is basically a configuration controlling from where to get the kernel and what kernel parameters to use for the booting process. Optionally one can also specify to load the init ramdisk. The PXE file needs to be stored in a file int the pxelinux.cfg directory. This of course depends on the version of the u-boot loader you have. I'm using linaro u-boot loader U-Boot SPL 2012.07 The image with the PXE configuration can have many names. The u-boot loader tries several names, the first being the mac address of the network interface. In my case it is: 01-2e-60-09-a7-46-01 And this is the content of my file PXE file stored in /tftp/panda_es/pxelinux.cfg/01-2e-60-09-a7-46-01 The directory /tftp/panda_es/ is the root of the tftp server. default panda-natty prompt 0 timeout 3 label panda-natty kernel uImage append console=ttyO2,115200n8 mem=1G@0x80000000 noinitrd rw init=/init root=/dev/nfs ip=192.168.0.104:192.168.0.107:192.168.0.1:255.255.255.0::eth0:on nfsroot=192.168.0.107:/srv/nfs/porky/seemee/panda_es,wsize=1024,rsize=1024,nolock,vers=3 rootwait earlyprintk smsc95xx.macaddr=01:2e:60:09:a7:46:01 vram=48M omapfb.vram=0:8M,1:8M,2:8M omapfb.mode=dvi:1360x768-16@60 omapdss.def_disp=dvi #initrd panda_es/uInitrd When you start the device you need to tell the u-boot loader to actually get boot from the network. These are the commands you need to run on the Pandaboard. Of course it requires you are connected to the Pandaboard over the RS232 line. In the minicom (or any serial port terminal) type the following commands. setenv bootfile porky/seemee/panda_es/uImage; setenv serverip 192.168.0.107; setenv autoload no; usb start; bootp; pxe get; pxe boot Instead of writing these commands every time you are booting you can create boot.scr file and store it on the SD card in the boot partition (this is the VFAT one:) This is how it can be done: 1. Create panda-pxe-boot.txt file with this contents: setenv bootfile panda_es/uImage; setenv serverip 192.168.0.107; setenv autoload no; usb start; bootp; pxe get; pxe boot 2. sudo mkimage -A arm -T script -O linux -C none -a 0 -e 0 -n "boot.scr" -d panda-pxe-boot.txt panda-pxe-boot.scr 3. copy boot.scr along with MLO, u-boot and uImage in MMC/SD boot partition. If you want to build you u-boot loader you can get one from the linaro git repository and compile it. The latest u-boot that supports the Pandaboard ES is here: git clone git://git.linaro.org/boot/u-boot-linaro-stable.git Now configure the u-boot for the pandaboard ES. make CROSS_COMPILE=/attic/angstrom2/build_panda/tmp-angstrom_2010_x/sysroots/i686-linux/usr/armv7a/bin/arm-angstrom-linux-gnueabi- omap4_panda_config and compile make CROSS_COMPILE=/attic/angstrom2/build_panda/tmp-angstrom_2010_x/sysroots/i686-linux/usr/armv7a/bin/arm-angstrom-linux-gnueabi- When you finish copy the u-boot.img and the MLO to the root of the boot partition on your SD card. Than add the boot.scr that we created in the previous step and reboot your device. Wifi driver When booting the new kernel I noticed the wifi driver was complaining about not having a firmware to load on the wifi WL12xx chip. To sort this out I downloaded a snapshot of the ti-connectivity and copied the content to the /lib/firmware/ti-connectivity on the board. http://git.kernel.org/?p=linux/kernel/git/firmware/linux-firmware.git;a=tree;f=ti-connectivity;h=5c71f576b35d1e15825173d650dc84476d1115b7;hb=HEAD Next time... I'm sorry for this rather rough description of the Pandabaord setup, I did it quite in hurry because I still haven't sorted it out completely which is my focus now. For example I still haven't tested the audio input, OpenGL rendering nor the accelerated video codec. I'm gonna write up more detail and accurate description or if you will a quite how to build a fully working system for the Pandaboard ES with all the extra features of the Omap4460 There are some interesting links that helped me a bit while building the system for the Pandaboard. Here are some interesting links I used. OMAP4 resources http://focus.ti.com/pdfs/wtbu/omap_4_pb_swpt034.pdf Posted in ARM, Embedded, Linux | Leave a comment Matrix exponential to the Rodrigues' rotation formula Last time we have seen where the formula came from and today I will try to describe how to do the matrix exponential by means of sine and cosine of the rotation angle and some multiplication and addition. As we will see at the end the result will be the Rodrigues' rotation formula which should not be a surprise 🙂 Ok so how can we exponentiate a matrix. With a real number we use the Taylor series. Remember that is a skew symmetric matrix of the vector Where the direction of this vector is the axis of the rotation and the magnitude is the angle of rotation in counter clockwise direction. We can split the vector to the axis of rotation represented by a unit vector where and the angle of rotation Upon converting the vector to the skew symmetric matrix so we can write Thus The Taylor series for Ok this is nice but so far nothing really new or too interesting. To move on we need to keep staring at this formula for a bit. We know that sine and cosine functions have very similar power series. The sine and cosine can be added to form with the help of complex numbers, strictly speaking the imaginary number Lets look how it works Now we exponentiate the as such and We get We can group the real and imaginary numbers to get. The real component is the same as the cosine and the imaginary one as the sine function which give us the famous Euler formula. Without diving into more mathematical details we can say the imaginary number in front of the sine function alternates the signs of the elements in the sine power series based on the exponent. This is great but how does it help us? Well if we could find something that alternates the sign based on the exponent we could convert to an easy to calculate sum of the sine and cosine. Luckily this is possible. It turns out that if we exponentiate the skew symmetric matrix we get back the original with just inverted sign. Well unfortunately not for all exponents. We know that the outer product is This is the result of squaring the skew symmetric matrix, now what happens when we do power of 3,4,5..... $[n]_\times^3 = -[n]_\times$ $[n]_\times^4 = -[n]_\times^2$ $[n]_\times^5 = -[n]_\times^3 = [n]_\times$ $[n]_\times^6 = -[n]_\times^4 = -[n]_\times^2$ Lets use this knowledge and reduce the terms in the Taylor expansion We only leave for now. We can see that there are terms with and Lets factor them out. Hmm I'm sure you can already see some similarity with the sine and cosine functions. Well, the sine is quite clear but what about the cosine? We can see that we are missing the 1 at the beginning of the series an also that all signs are inverted. and this is what we have... We can convert our power series to the cosine by subtracting 1 from the cosine power series. Great using this manipulation in our matrix exponential equation we get And here we go, this is the Rodrigues' rotation formula Posted in Computer Vision | Leave a comment Rotation matrix to angle-axis This time I'm gonna go through a process of extracting rotation axis and angle from the rotation matrix . Lets recap some properties of the rotation matrix that we will be using. • Is a square matrix. • Is normalized, column and row vectors have unit length. • Is orthogonal, column vectors are mutually orthogonal to each other. The same holds good for rows. In other words the dot product of any two pairs of row or column vectors is 0 • For orthogonal matrix holds that • or for reflection. To find the axis and angle of rotation we will call eigenvectors and eigenvalues for help. The eigenvalues of an orthogonal rotation matrix must satisfy one of the following: 1. All eigenvalues are 1. 2. One eigenvalue is 1 and the other two are . This corresponds to rotation angle 3. One eigenvalue is 1 and the other two are complex conjugates of the form and 4. The rotation axis is the eigenvector corresponding to the eigenvalue 1 Angle of rotation from eigenvalues As long as the matrix is orthogonal which is true for our rotation matrix the trace (sum of the diagonal elements) is independent of the coordinate system used. In other words the trace is independent of the axis of rotation, it depends only on the rotation angle. We also know the sum of eigenvalues equals to the trace of the matrix. so the rotation angle is Angle of rotation from Rodrigues' formula We can also derive this equation from the Rodrigues' formula. By doing so we will also prove that the eigenvalues of the rotation matrix are . As we said before the trace is independent of the axis of rotation, it depends only on the rotation angle. Lets do the trace of both sides of this equation. The trace of and the trace of any skew symmetric matrix is zero We also substitute The reason for that is that the which helps us during the simplification. expand the and simplify... Earlier we saw that sum of the eigenvalues is From the Rodrigues' formula we proved that And again we get the same equation for the rotation angle From the above we also proved that the sum of eigenvalues equals to There is one issue here. If you invert the rotation axis and negate the rotation angle you get the same rotation. This means you can get two possible angle/axis pairs corresponding to the same rotation. Rotation axis There are several ways we can find the rotation axis. Perhaps the easiest to understand is the following. When we multiply the rotation matrix by a vector which is aligned with the rotation axis of the matrix the product will be the same vector. This vector will not move or get scaled. Let label this vector By moving things around we get Let's define new matrix We can see that when the matrix B gets multiplied by the vector the product is 0. We say the vector belongs to the nullspace of the matrix B or it is a kernel of the matrix B. If B is invertible the only solution would be (trivial solution) which is not what we want. The matrix B is a result of where R is orthogonal matrix with determinant=1 which makes the B matrix singular -> determinant of B is zero. In this case we can find the solution as such This is called non trivial solution and it is our axis of rotation. If the singular matrix has at least two linearly independent rows, then the vector in the nullspace of B is a cross product of these rows. In other words the cross product of two vectors in always lies in the nullspace of the matrix with the vectors as rows. Why is that? Lets look inside the multiplication of the matrix and the vector If is a th row of B then the dot product of each row with the vector must be zero. This means that the vector must be orthogonal to each row in the matrix . Our Matrix B is singular thus at least one vector is linearly dependent. Indeed each row vector in our matrix is a linear combination of the other two. In fact all row vectors lie in a plane. It should not be a surprise that this holds good for the columns as well since we started with an orthogonal matrix. We said that the vector must be an orthogonal to each row vector of the B to satisfy Now, this is quiet easy to find since all row vectors lie in the plane so we just need to find a perpendicular vector to that plane. To do that we take a cross product of two rows, for example the first two and then normalize it. and normalize it The is our normalized axis of rotation. What we have just done was in essence extraction of eigenvector belonging to the eigenvalue = 1. As we said the axis of rotation has this property On first looks you can see that the vector is an eigenvector with a corresponding eigenvalue . Using Cayley transformation To get the axis of rotation we can use the Cayley formula Where is a skew symmetric matrix representing the axis of rotation with components We can also go back to the rotation matrix The matrix is scaled by which we can use to normalize the axis. Rotation axis from Rodrigues' formula Other option is to use this relation Since is skew symmetric then This rule helps us to simplify the following We also know that the outer product is Using these substitutions we get Since is symmetric then And to get the normalized axis of rotation From this we can see the axis of rotation is actually only with length If we don't know we can use the length to normalize the axis vector. Now we need to convert the skew symmetric matrix back to the vector representation . We know so the is Posted in Computer Vision | Leave a comment Rotation matrix from exponential twist Last time I described how to derive a geometric derivation of the the famous Rodrigues' rotation formula. While being quite easy to understand there is another way how to get the rotation matrix from the angle-axis representation. The approach I'm gonna go through today derives the rotation matrix from a constant angular velocity of a rigid body. The goal is to get rotation matrix from axis of rotation and angle of rotation I hope I won't spoil your excitement from the discovery if I show you how it's done right now. The is 3x3 rotation matrix, is a skew symmetric matrix of the unit vector representing rotation axis and the is the rotation angle in radians. By the right hand rule the rotation is counter clockwise. The inverse rotation is described by flipping the rotation axis. Here is how this equation can be derived. Lets define some basic variables first. Angular velocity in . It is a pseudovector whose direction represents the axis of rotation and the magnitude specifies the angular speed. The angular speed is a signed quantity which is reflected by the direction of this vector. Vector from the axis of rotation to the point at the time Position of the vector at the time Rotation matrix that moves the vector to the Instantaneous velocity of the the point at the time Lets start with the idea of rotating the vector around the axis from it's starting position to new position at the time . We know that we can rotate the by pre-multiplying by an orthogonal rotation matrix . This matrix moves the vector to the new position for time so we express it as (1) This means rotates the vector to the new position . In the figure above I labelled the angular displacement as angle . This means the rotation matrix corresponds to the rotation by angle around axis . We can imagine that our asix is aligned with the axis and the point is rotating in the plane. Now we need to calculate instantaneous velocity of the vector . Why we are doing this will become apparent soon. Instantaneous velocity of the point is a perpendicular vector to the vector which we get by differentiating the above equation with respect to the time. Note is a constant. This is all nice and well but still quite far from our goal which is to calculate the rotation matrix from the axis and angle of rotation. Ok lets carry on and see whether we can do something about it. We know the instantaneous velocity of the vector can be also calculated as a cross product of the angular velocity and the rotating point. And now we have something interesting, two equations that calculate the instantaneous velocity of the same vector. Lets substitute to the From the equation (1) we can substitute the with To simplify the calculation we convert the cross product to multiplication with skew symmetric matrix Where the is the skew-symmetric matrix of the Now we have to solve this differential equation. Let's rearange terms into the standard form and see how we can solve it. We can see it is a first order, linear, time invariant equation which has a known solution I'm gonna be more explicit here and show how to solve this equation by the homogeneous and particular solution. This technique is based on finding two solutions, the particular and homogeneous. The result is then the sum of those two. The particular solution is found by guessing a constant variable that would satisfy the equation. Since derivation of constant is zero the differential term drops out. In our case the particular solution is zero. Homogeneous solutions is found by replacing the right side of the differential equation with zero (removing the driving force) and guessing what solutions would satisfy this equation. In our case the equation already equals zero. As the solution we guess The reason for this guess is that the derivative of the which nicely removes the derivation without any mathematical complications. Thus Since the particular solution is zero the complete solution is only the homogeneous one. What we need to do now is to get rid of the time parameter somehow and relate the angular velocity to the rotation angle While this may look like a complicated task in reality it is quite simple. Since we can choose any time interval we want we chose Now the angular speed (the magnitude of the angular velocity ) is so in one second the angle displacement will be just . And there you go we can calculate the rotation matrix from a given rotation axis and the angle of rotation. The direction of the vector is the axis of rotation and the magnitude is the angle of rotation. We can split it into the angle and axis components. For unit vector representing the axis of rotation we get Thus We could stop here and be happy with this formula. In the Octave we can get the rotation matrix from angle axis straight this way. p = pi/3; % Rotation angle in anti-clockwise direction n = [1; 2; 3]; % Axis of rotation % Normalization of the rotation axis vector n_norm = n/norm(n); % Create the skew-symmetric matrix n_skew = [ 0 -n_norm(3) n_norm(2); n_norm(3) 0 -n_norm(1); -n_norm(2) n_norm(1) 0]; % Calculate the rotation matrix R = expm(n_skew*p)  Or in the Python with SciPy library from numpy import * from scipy.linalg import * # Rotation axis n = mat('[1. 2. 3.]'); # Normalize the rotation axis n = n/numpy.linalg.norm(n) # Build skew-symmetric matrix n_skew = mat([[0, -n[:,2], n[:,1]], [n[:,2], 0, -n[:,0]], [-n[:,1], n[:,0], 0]]) #Angle of rotation alpha = math.pi / 3. R = expm3(n_skew*alpha)  The Octave and SciPy in Python nicely hide the calculation of the matrix exponential which is great but in many practical applications we need to do it on our own, many times even with very limited resources. Next time I will dive into how to calculate the matrix exponential and what everything in under the bonnet. Posted in Computer Vision | Leave a comment Rodrigues' rotation formula In many areas of robotics, computer vision and graphic is a need to rotate points or objects by certain angle around a given axis. One way to do it is to use Rodrigue's rotation formula. $R=I+[n]_\times \sin\alpha+[n]_\times^2(1-\cos\alpha)$ Where the is a skew symmetric of normalized vector of the rotation axis and is the rotation angle in radians. The result is a 3x3 rotation matrix It's simple to use and implement in every programming language but what's under the bonnet? How does it work? Ok lets work it out. There are actually at least two ways how to derive this formula, one purely geometric and the other more mathematical based on the matrix exponential. I will describe the geometric approach first. So let's start from the beginning. The Rodrigues formula is used to perform angle-axis rotation, or simply said it rotates a point in the space around a given axis by given angle. It also outputs standard rotation matrix that I'm sure most people reading this article are familiar with. The most basic way of representing a rotation are Euler Angles. Euler Angles Any rotation in the space is represented by three 2D rotations in planes xy, yz, and xz. The matrix form of these rotation is The rotation matrix is a product of the three discrete 2D rotations. $R=ABC$ It's clear the result depends on the order in which we perform the rotations. Rotating around X->Y->Z axes results in different final position than X->Z->Y. This is also given by that fact matrix multiplication is not commutative. The Euler angles rotation is simple to understand and visualize however is not really practical in many applications. Among problems associated with this technique is that the rotation is not smooth (a small change in position of the object can result in dramatic change of one or more Euler angles) and there is also risk of gimbal lock. Angle-axis rotation solves these problems and on top of that is also very simple to understand and work with. Angle-axis Angle-axis rotation is more common the one may think. For instance everything on the Earth is rotation around the Earth's axis which is not the same as the solar system or our galaxy axes. Other example would be a car. Wheels are rotating around their axes that are different from the axis of the steering wheel or the engine crankshaft. Let's look at some more mathematical description. Let say we have a point that we want to rotate by angle around axis to the final position To do that we first define a vector from the origin to the point and a plane in the origin perpendicular to the axis of rotation We also normalize the vector representing the axis of rotation Second step is to project the vector on the axis of rotation and also on the plane in the origin. The vector projected on the rotation axis is called We will calculate it as a vector projection which is basically a dot product of the vector with the unit vector in the direction of $v_p=(n\cdot v)n$ To get the vector we simply subtract from $v_r=v-v_p$ The vector is called vector rejection Now we create a vector perpendicular to the plane We will use this vector to rotate the image of the by the angle on the plane in the origin. $w=n\times v$ It turns out that the cross product between unit vector and the vector has the same length as the vector rejection To do that we perform a 2D rotation of the vector in the plane . We will call this new vector $v_{rr}=v_r\cos\alpha+w\sin\alpha$ We are almost done. There's one more step. To get from the plane in the origin to the point we just need to add vectors and $u=v_{rr}+v_p$ Let's recap all equations and pull them together. $v_p=(n\cdot v)n$ $v_r=v-v_p$ $w=n\times v$ $v_{rr}=v_r\cos\alpha+w\sin\alpha$ $u=v_{rr}+v_p$ If we substitute all equations we get $u=(v - (n\cdot v)n)\cos\alpha + (n\times v) \sin\alpha +(n\cdot v)n$ $u=v\cos\alpha-(n\cdot v)n\cos\alpha + (n\times v) \sin\alpha + (n\cdot v)n$ $u=v\cos\alpha+(n\cdot v)n(1-\cos\alpha) + (n\times v) \sin\alpha$ We can simplify this equation by converting the dot and cross products to multiplication. Assuming that all vectors are column vectors the dot product is calculated as: $v_p=(v\cdot n)n = (n^Tv)n = nn^Tv$ To calculate the cross product we use a skew-symmetric matrix trick 🙂 We first convert the vector to the skew-symmetric matrix and then multiply it with the vector The vector in the skew-symmetric matrix form is $[n]_\times=\begin{pmatrix}0 & -n_3 & n_2\\n_3 & 0 & -n_1\\-n_2 & n_1 & 0\end{pmatrix}$ $w=n\times v = [n]_\times v$ Substituting the dot and cross product equations we get the Rodrigues formula. $u=v\cos\alpha+nn^Tv(1-\cos\alpha) + [n]_\times v\sin\alpha$ So now we have a way to rotate point around the rotation axis by angle . To get the rotation matrix we need to split this equation in the and components. $u=Rv$ thus $u=(I\cos\alpha+nn^T(1-\cos\alpha) + [n]_\times \sin\alpha)v$ $R=I\cos\alpha+nn^T(1-\cos\alpha) + [n]_\times \sin\alpha$ We can still simplify it a bit. We use the identity of the outer product $nn^T=[n]_\times^2+I$ Where is the identity matrix. This identity can be proved by following a few steps. First of all lets write down in the term of element-wise operations. ${nn}^T=\begin{bmatrix}n_1 \\ n_2 \\ n_3\end{bmatrix}\begin{bmatrix}n_1 & n_2 & n_3\end{bmatrix}=\begin{bmatrix}{n_1}^2 & n_1n_2 & n_1n_3\\n_1n_2 & {n_2}^2 &n_2n_3\\n_1n_3 & n_2n_3 &{n_3}^2\end{bmatrix}$ With a little bit of insight we may realize this is very close to the skew symmetric matrix of the squared $[n]_\times=\begin{bmatrix}0 & -n_3 & n_2\\n_3 & 0 &-n_1\\-n_2 & n_1 & 0\end{bmatrix}$ If we square it we get $[n]_\times^2=\begin{bmatrix}-{n_3}^2-{n_2}^2 & n_1n_2 & n_1n_3\\n_1n_2 & -{n_3}^2-{n_1}^2&n_2n_3\\n_1n_3 & n_2n_3 &-{n_2}^2-{n_1}^2\end{bmatrix}$ Now we can see that it really looks like apart from the diagonal. The last step is to somehow turn the diagonal of the to the diagonal of This is quite easy. Since the vector has unit length we can just add one to the diagonal to get For example So now we can write $\begin{bmatrix}{n_1}^2 & n_1n_2 & n_1n_3\\n_1n_2 & {n_2}^2 &n_2n_3\\n_1n_3 & n_2n_3 &{n_3}^2\end{bmatrix}=\begin{bmatrix}-{n_3}^2-{n_2}^2&n_1n_2&n_1n_3\\n_1n_2 &-{n_3}^2-{n_1}^2&n_2n_3\\n_1n_3&n_2n_3 &-{n_2}^2-{n_1}^2\end{bmatrix}+I$ which is ${nn}^T =[n]_\times^2+I$ Using this identity in our equation we get $R=I\cos\alpha+([n]_\times^2+I)(1-\cos\alpha) + [n]_\times\sin\alpha$ $R=I\cos\alpha+[n]_\times^2+I-[n]_\times^2\cos\alpha-I\cos\alpha+[n]_\times\sin\alpha$ $R=I+[n]_\times^2-[n]_\times^2\cos\alpha+[n]_\times\sin\alpha$ And finally we get the Rodriques formula $R=I+[n]_\times\sin\alpha+[n]_\times^2(1-\cos\alpha)$ Next time I will show how the same formula is derived from angular velocity of a rigid object. Posted in Computer Vision | 4 Comments Running Skype on an embedded device - video. Part 3 And now something even more interesting..... a Skype video calling from our ARM embedded device. Unlike the SkypeKit desktop version the one for embedded devices supports only H264 over RTP. This is no problem since I would need to use it anyway due to the HW accelerated video coding/decoding that I'm inteding to run. The RTP video host runs as a standalone process as the audio one does. Here is how it interfaces with the SkypeKit runtime. The video host must • Capture video from the camera or screengrab or photo album or from a DVB-T/S receiver 🙂 etc. • Preview the video • Encode/Decode video to the H264. Great I have the HW accelerated codec already ready • Render the video on screen and/or saves it into a file. The Rendering will be done directly via the OMAP framebuffer. Ok lets build something finally. Go to the directory with the reference implementation of the video loopback . sdp-distro-skypekit_3.7.1.331_1823867/reference/videortphost-loopback and call make TARGET_ARCH=armv7le-eabi TOOLCHAIN_PREFIX=arm-angstrom-linux-gnueabi- As usually I have expected some issues but I was surprised. Not problem at all. The build finished with this new binary sdp-distro-skypekit_3.7.1.331_1823867/reference/videortphost-loopback/build/videortphost-loopback To run the Skype client app with the audio and video hosts I ran these processes. 1. videortphost-loopback 2. voicepcmhost-rtaudio 3. linux-armv7-skypekit-voicepcm-videortp Notice this time I'm running the linux-armv7-skypekit-voicepcm-videortp runtime that communicates with the video host. 4. skypekitclient -t av_test_arm_v1.pem When you run the skypekitclient the SkypeKit runtime should output this  ./linux-armv7-skypekit-voicepcm-videortp
SkypeRuntime Copyright (C) 2003-2012 Skype Technologies S.A.
SkypeRuntime Version: 3.7/linux-armv7-skypekit-voicepcm-videortp_3.7.1.341_1823905
AVTransportWrapper (/tmp/vidrtp_to_skypekit_key): connected!
AVTransportWrapper (/tmp/vidrtp_from_skypekit_key): connected!
AVTransportWrapper (/tmp/pcm_to_skypekit_key): connected!
AVTransportWrapper (/tmp/pcm_from_skypekit_key): connected!

You can see it got connected with the audio and video hosts.

Now you can make a video call. From the command line of the skypekitclient
Cx
<you will be asked for skype Id to call to>

Or call from your dektop on the skypekitclient. In that case you will pick up the call by command
Ca

After that you will need to start sending the video by executing command
CS

...and start receiving video by
CR

At this moment you should see a picture from you desktop camera (assuming you are using you Skype on the desktop to call the skypekitclient  on the ARM board) being returned as a video from the skypekitclient. As with the audio loopback the video loopback just takes the video frames and returns them back to the caller.

This was not so bad was it? I cannot wait to actually decode and display the video on a small HDMI monitor I have attached to the ARM device, but that's for next time.

Running Skype on an embedded device - audio. Part 2

Last time In the article Running Skype on an embedded device. Part 1 I have described how to install and build the SkypeKit API wrapper and the Skype client demo application. I admit it was quite boring mainly because the demo even didn't run. This time I will describe how to build the voice loopback host and a voice host that works with several Linux audio drivers such as ALSA and OSS.

There are two possible voice hosts implementations possible, PCM and RTP. The RTP host is obsolete and has been discontinued in the 4.3 and higher versions so I'm not gonna spend too much time with it.

RTP voice host

The RTP host takes the raw RTP samples with the encoded audio packets, decodes them and play them via an audio device. Similarly for the output audio output the RTP host is responsible for capturing the audio, encoding it and pushing the encoded samples to the SkypeKit. Since this API was discontinued I'm not gonna deal with it.

PCM voice host

works on the PCM level with the SkypeKit which means the developer of his host needs to select, configure and open the audio IO and read audio samples from microphone or other input and feed them to the SkypeKit via its API. The same holds good for the audio output. In general this is quite easy since all audio drivers works with the PCM samples. This is how the skype interfaces with the PCM host.

You can see the SkypeKit handles things such as audio codecs, echo cancellation,  AGC and jitter buffer.

There are three PCM host reference implementation in the SkypeKit

• voicepcmhost-file
Uses a files for audio IO. We would run it like

./voicepcmhost-file -i myInput.wav -o myOutput.wav
• voicepcmhost-loopback
As the name says, it take the received audio and forward it back. This is the basic host I'm gonna build first.
• voicepcmhost-rtaudio
This is the more complete voice host that can actually read audio from a microphone and play it out on speakers.

Building and running the voicepcmhost-loopback host

In order to run the examples from the SDK we need to build at least one voice PCM  backend (host) from the reference implementations in the directory

sdp-distro-skypekit_3.7.1.331_1823867/reference/

We are going to build the loopback one so go to corresponding directory

$cd sdp-distro-skypekit_3.7.1.331_1823867/reference/voicepcmhost-loopback and run make with the architecture name and the tool chain prefix. The architecture must be on of the sub-directories in the bin dir. • linux-armv5le-eabi • linux-armv6le-eabi • linux-armv7le-eabi • linux-mips32be • linux-mips32le • mac-x86 • windows-x86 • linux-x86 This toolchain is the string in front of the "g++" in the file name of you ARM cross compiler. My compiler file name is arm-none-linux-gnueabi-g++ make TARGET_ARCH=armv7le-eabi TOOLCHAIN_PREFIX=arm-angstrom-linux-gnueabi- Here I encountered a few problems. Firstly I didn't have the premake4 tool which is used by the build scripts. This is where I got it: http://industriousone.com/premake When the build finishes you should get voicepcmhost-loopback binary in the build directory sdp-distro-skypekit_3.7.1.331_1823867/reference/voicepcmhost-loopback/build/voicepcmhost-loopback Now you just need to copy this file on your ARM device, or a filesystem accessible from the ARM device and you are ready to run the Skype client! To do that we need to run the voice host first, then the Skype runtime and then the skype client application. For this experimentation we want to run a console window for each process so we can nicely see all log messages. First we run the voice PCM backend $ ./voicepcmhost-loopback
AVTransportWrapper (/tmp/pcm_from_skypekit_key): connected!
AVTransportWrapper (/tmp/pcm_to_skypekit_key): connected!
Init
UseDefaultDevice: INPUT_DEVICE
UseDefaultDevice: OUTPUT_DEVICE
GetCurrentDevice: INPUT_DEVICE, guid0
UseDevice: INPUT_DEVICE, guid0
UseDevice: OUTPUT_DEVICE, guid0
GetCurrentDevice: INPUT_DEVICE, guid0
GetDevices: INPUT_DEVICE
GetDevices: OUTPUT_DEVICE
Stop: INPUT_DEVICE
Stop: OUTPUT_DEVICE
UseDevice: INPUT_DEVICE, guid0
UseDevice: OUTPUT_DEVICE, guid0
GetCurrentDevice: INPUT_DEVICE, guid0
UseDevice: INPUT_DEVICE, guid0
UseDevice: OUTPUT_DEVICE, guid0
GetCurrentDevice: INPUT_DEVICE, guid0
GetCurrentDevice: INPUT_DEVICE, guid0
Start: INPUT_DEVICE
Stop: INPUT_DEVICE
GetCurrentDevice: OUTPUT_DEVICE, guid0
Start: OUTPUT_DEVICE
Stop: OUTPUT_DEVICE
Stop: INPUT_DEVICE
Stop: OUTPUT_DEVICE
Stop: NOTIFICATION_DEVICE

Then the SkypeKit runtime

$./linux-armv7-skypekit-voicepcm-novideo SkypeRuntime Copyright (C) 2003-2012 Skype Technologies S.A. SkypeRuntime Version: 3.7/linux-armv7-skypekit-voicepcm-novideo_3.7.1.343_1823908 Proprietary and confidential, do not share this application. AVTransportWrapper (/tmp/pcm_to_skypekit_key): connected! AVTransportWrapper (/tmp/pcm_from_skypekit_key): connected! And finally the Skype client. Remember you need to pass the development certificate to the client app. $ ./skypekitclient -t av_test_arm_v1.pem
::: Connected to skypekit
3.7/linux-armv7-skypekit-voicepcm-novideo_3.7.1.343_1823908

New account oid 37

::: Type ? for list of available commands

If you get a similar response everything is fine. In my case it worked on first try. Now you can login to your Skype account but typing

aL

When you make an audio call on this Skype client you will get an audio echo. The loopback host simply forwards the audio that is received back to the caller.

Btw the call is answered/terminated from the command line by typing

Ca - answer
Ce - hangup

Building and running the voicepcmhost-rtaudio host

voicepcmhost-rtaudio is another reference implementation of the audio backend that comes with the SkypeKit. This host provides an interface for these audio implementations

• Windows (DirectSound and ASIO)
• Macintosh OS X (CoreAudio and JACK)
• Linux (native ALSA, JACK, and OSS)

The voicepcmhost-rtaudio reference implementation that comes with the Skypekit relies on an open source library rtAudio. The SkypeKit sdp-distro-skypekit_3.7.1.331_1823867 needs rtaudio version 4.0.6 which can be downloaded from http://www.music.mcgill.ca/~gary/rtaudio/release/rtaudio-4.0.6.tar.gz It may well run with the latest one which is by the time of writing this article 4.0.11 but I haven't tried it.

sdp-distro-skypekit_3.7.1.331_1823867/reference/voicepcmhost-rtaudio/rtaudio-4.0.6

You don't have to build the rtaudio beforehand. The rtaudio is very simple project consisting of only three source files that the voicepcmhost-rtaudio compiles and links itself. Got here

sdp-distro-skypekit_3.7.1.331_1823867/reference/voicepcmhost-rtaudio/

and run

make PCMHOST=rtaudio TARGET_ARCH=armv7le-eabi TOOLCHAIN_PREFIX=arm-angstrom-linux-gnueabi-

When the build finishes you should get voicepcmhost-rtaudio in the build directory

sdp-distro-skypekit_3.7.1.331_1823867/reference/voicepcmhost-rtaudio/build/voicepcmhost-rtaudio

The voicepcmhost-rtaudio uses the default audio IO device. This is correct for the audio output but not for the audio input on my platform. Currently I'm testing the audio input from a webcam microphone which works as a standalone sound card with just audio capture interface. To make the voicepcmhost-rtaudio to correctly connect to the webcam microphone I had to set the device ID =1 in the StartInputDevice() function in the RtAudioPCMInterface.cpp file

int RtAudioPCMInterface::startInputDevice(RtAudio *device)
{
uint buffer_frames = 512;
uint fs = RATE;
RtAudio::StreamParameters params;
// params.deviceId = 0;//device;
params.deviceId = 1;//device;
params.nChannels = IN_CHANNELS;//channels;
params.firstChannel = 0;

try {
device->openStream(NULL, &params, FORMAT, fs, &buffer_frames, &input_callback, (void *)this);
device->startStream();
}
catch (RtError& e) {
std::cout << '\n' << e.getMessage() << '\n' << std::endl;
return 0;
}
return 1;
}


After a rebuild I copied the voicepcmhost-rtaudio on my platform and again run the whole skype circus by executing the following processes in this order:

1. voicepcmhost-rtaudio
2. linux-armv7-skypekit-voicepcm-novideo
3. skypekitclient -t av_test_arm_v1.pem

And voila, the audio IO works like a dream:-) Next time I will focus on the video side of the thing, and again as with the audio I will build the video loopback host first.

For now all the best....