Sub Ubi

On March 29th, we gave our first Google TechTalk!

You can hit the link to go to the Google Video site. So far, we’ve had quite a few views, and apparently, people enjoy it (according to the ratings, anyway). We had a hard time targeting the talk; we rewrote it several times before giving it, as we couldn’t decide how best to approach the work we’ve been doing for the last four years. Do we dive straight into how we can use a language like occam-pi for abstracting over clusters, or how we can build safer, concurrent software on embedded platforms?

In the end, we gave an introduction to the language, and then focused a bit on how we can convert parallel models of software directly into code, and motivated this using Jon Simpson’s work with the subsumption architecture on small robotics platforms. We received many positive comments on the talk, and one person did say they thought it was light on “hard-core” detail. But what can you do in 45 minutes with an unknown audience?

In the end, we gave a good talk, and were excited to be able to share our work with the Googlers there and who are now catching the video on-line. (And, for that matter, anyone else who wants to see what we had to say.)

Certainly, it was a lot of fun. And yes, the food in the cafeterias at Google really is that good.

Sunday I take off for San Fran, and am pretty psyched. Christian, Jon, and I are heading to the AAAI Spring Symposium to talk robotics and concurrency with a bunch of other robotically-minded people; that fills our time between Sunday and Wednesday. Thursday, we’ll be giving a Google TechTalk, which should be very fun—at least, we’re planning on enjoying the talk immensely. Then, we’re thinking (but aren’t committed) to doing a little bit of light camping, as this time of year can be soooo nice in the Bay area.

We don’t have housing nailed down for Palo Alto Sunday/Monday/Tuesday, but I think we might just grab a hotel near Stanford. It will be a bit pricey, but otherwise, we’re dealing with cars, and parking, and commuting, and… that’s not so much fun. We’ll see.

I’ve touched base with a few friends in the area (dbort and Tzu), but I’m wondering if there’s anyone I missed. Is there anyone who reads this blog who is in the Bay area I should try and say ‘hi’ to? Is there anyone who reads this blog who knows anyone in the Bay area I should say ‘hi’ to? (I think that’s one degree of separation; more degrees of separation than that, and I suspect I only want to know them if they’re going to give me money for my ideas on fixing email, or if they want to give us a place to sleep.)

This is very, very exciting. I mean… very. Exciting. Very.

OK, so I just like talking about and playing with robots. What can I say?

Although it sounds odd, Damian built a DOS version of the Transterpreter last week.

Oh, the humanity!

This may sound wacky, but the reasoning is sound. We have a large suite of tests that we run the Transterpreter through on a regular basis. Actually, we use buildbot to run all of the tests anytime someone checks code into the trunk. However, this only runs the tests on “big” platforms—it runs them on a Linux box or two, and a Solaris/SPARC machine. We cannot, at this time, automatically run all of our tests on an embedded 16-bit target; while we do have a development board or two we could dedicate to this task, I’m not interested (right now) in doing the scripting to upload code over a JTAG port, run the tests, and check (on the device) to see if we are doing the right thing. It’s a lot of scripting.

The H8/300, the processor at the heart of the LEGO RCX

So, when Jon was having a hard time with the LEGO RCX recently, it became clear that we really needed to run our tests on 16-bit platforms. This would help shake out whether or not the compiler, linker, or run-time were having a hard time when there were fewer ones and zeros running around. As a result, Damian setup the Open Watcom 16-bit DOS compiler, built the VM, booted up a FreeDOS image in QEMU, and ran a bunch of the tests through the VM. We passed all the tests we could easily run, and are working to get the rest compiling. It turns out we need to build some libraries that we’ve never needed before…

So, yes, the Transterpreter can run in 16-bit mode in DOS. There’s a number of features that are untested at this time, but hopefully we’ll find a way to automate the testing of the VM under the FreeDOS image.

The Transterpreter project was born three years ago around this time; it started life as a Scheme implementation of a Transputer bytecode interpreter, and has grown quite a bit since then. Given that we’re starting into 2007, it seemed like a good time to give an update about what is going on in the group. In no particular order:

• Jon Simpson (third-year undergraduate at Kent) has been working on revitalizing the LEGO Mindstorms RCX port of the Transterpreter. The old version was built on top of BrickOS; we were, effectively, running two operating systems on the LEGO, one on top of the other! Jon is at the point where he has the Transterpreter running natively on the LEGO Mindstorms RCX, can handle button input (including a nifty, non-blocking debouncer), and is continuing to slowly build up to more useful end-user interface code. By March he expects to have a complete and usable (native) port of the Transterpreter on the RCX. (/branches/new-lego)

  Looking forward, Jon (and the rest of us) are excited about getting the NXT port of the Transterpreter rolling. We have already had some interest expressed from others about taking part in this effort; we’ll be generating some mail on that shortly.

• Damian ended the year cleaning up his Cell port of the Transterpreter (/branches/cell), with particular attention being paid to developing a native, big-endian port of the Transterpreter (/branches/big-endian). Currently, we pay a penalty on all big-endian hardware, and Damian (and other’s) efforts in this regard will give us full, native performance on big-endian platforms; this effects the PowerPC (Mac, Cell) and Mindstorms RCX, among others. Looking forward, he is coming back to 42 (our experimental compiler), and wants to work out the critical points in generating little/big endian code as well as “fat binaries”.

• Adam Sampson picked up a few Arduino boards (http://www.arduino.cc/), which are driven by an Atmel ATmega8. This little chip has roughly 7KB of free flash and 1KB of RAM; it represents the smallest device the Transterpreter has been targeted at yet (/branches/arduino). While Adam’s intent was to develop an Arduino port of the Transterpreter, he says that “in the process I’ve done a load of cleanups which are probably more useful than the work I was intending to do.”

  The “cleanups” that Adam has made in the Arduino branch are (in his words):

  1. wrapper (currently untested) for the ATmega8

  2. instruction dispatching via switch, so all the instructions end up inlined, which cuts the code size considerably (and should make it a bit faster)

  3. some fixes to comments

  4. consistent ANSI C prototypes throughout

  5. add multiple-inclusion guards to all the headers

  6. make the word size code use inttypes.h and be configurable at compile
     time

  These are all nice cleanups that will ultimately be merged back into the trunk, and therefore benefit all of the Transterpreter ports.

  Adam’s explorations also involved cleaning up the “memory array” code (/branches/mem-array). When originally developing the TVM, we had a “virtual memory”, which made detecting a variety of faults easier; this was the original memory interface for the Transterpreter, but was dropped shortly after porting to C. Adam has cleaned this up as well, and (in his words, again):

  1. make the array memory backend actually work (mostly with the aim of being able to use a non-native word size, although I haven’t yet tested that)

  2. do bounds checking in mem_array so that the TVM stops (rather than
     segfaults) in the event of a bad memory access

  3. fix a number of bugs related to string handling and memory allocation which were revealed by Valgrind; we need to do a Valgrind run with some proper code (using dynamic memory) at some point to try and shake out more

  4. completely rework the code in stiw that builds the memory map, since it had a number of bugs and didn’t work with mem_array before

  5. fix a few bugs in interpreter revealed by the work above

  These are all excellent, and again will find their way back into the trunk. Indeed, as part of his explorations, Adam generated a number of tickets that I added to Trac (http://trac.transterpreter.org/), and we can begin addressing in the interpreter, which is good.

• Matt Jadud (that’s me) has been focusing on the MSP430 port, living in the Tmote Sky branch (/trunk/wrappers/tmotesky). As a side effect of this work (along with Jon Simpson), we developed ‘tinyswig’ (/trunk/scripts/tinyswig.scm), a mini, SWIG-like script that lets us quickly and easily write C extensions to the VM, callable from occam-pi. After this, we worked out the details of interacting with hardware directly from occam-pi, which (in some cases) completely eliminates the need to call out to C from occam. For example, in ‘/trunk/wrappers/tmotesky/Native‘, you can find our initial explorations in this area, where we are bringing up the virtual machine with no connections to the hardware, and then configuring the MSP430 directly from occam-pi. We expect this to make implementing functionality for new embedded platforms much easier for the developer in some cases.

  Although it is unexciting infrastructure, I also began work on migration from Autotools to Scons (/branches/scons). Currently, it is possible to build the Transterpreter for Linux/Intel, MacOSX/Intel, and the MSP430 in this branch using Scons. The build scripts are, I believe, clearer, easier to extend, and will (I believe) give us a better cross-platform build experience (eg. Windows). Currently, we have to maintain a completely separate build system for Windows, which is untenable in the long run; with Scons, we have a chance of bringing things together nicely across all major development platforms. So, while unexciting, I consider this a rather important revision of tools and infrastructure. (Of course, I’m working on it, so you might expect I’d say that.)

  Along with the under-the-radar updates to the build system, I’ve nearly finished refactoring the linker; what had grown to 5000+ lines of Scheme has been reduced to roughly 1500. While the new slinker is not done yet, it is much simpler and much more modular. The new slinker was informed greatly by our explorations with 42—the underlying data structures are far more intelligent than they used to be, and allow us to do complex things very simply. It is, in a nutshell, a joy to work with (as source code), which I can no longer say about the original slinker.

  Looking forward, I’ll be working more with two new hardware development boards for the MSP430, exploring radio communications, analog-to-digital conversion, storage of data to external devices (SD, etc.), and most importantly, the integration of interrupts with the Transterpreter scheduler. The development boards are important, as they allow me to do in-system debugging via JTAG, which will be essential for the implementation of interrupt handling. This will, like many other things we do, impact all Transterpreter ports.

• Christian Jacobsen has recently been sighted trying to set up a “build bot” for the Transterpreter project; this will automatically check out our code, build it, and run the test suite on a regular basis. However, he should not be doing this, but should be submitting the final version of his thesis. We expect that to go out the door ANY DAY NOW. I read it on my flight over to the USA, and believe-you-me, it was very, very exciting. Very.

If this sounds like we’re busy, it’s because we are. We’ve got a few other things up in the air related to our ongoing efforts, and will drop word of those as they see the light of day. If you have any questions about the project, or want to get involved, please feel free to drop me a note (matt at this domain).

The porting of the Transterpreter is straight-foward. You check the sources out, configure your cross-compiler, and go. Some twiddling with ‘make’ is usually the biggest problem we run into.

Porting to the NXT will be slightly less straight-forward.

The core Transterpreter will cross-compile to the ARM7 without any problems. However, there’s a little problem of bringing up the hardware on the NXT that must be addressed. From my experience working on the Tmote Sky, the OS developer needs to check and set a significant number of pins to bring up each piece of hardware. These bit-twiddling operations are tricky; for this reason, it is likely that we’ll want to get a JTAG reader for the Mac and be able to inspect the hardware while developing.

The Atmel AVR is likely to be the first device we bring up; it gives us access to the sensors, motors, and buttons on the device. Once we have rudimentary access to the NXT’s sensors and motors, the simplest way to get feedback from our programs will be to implement sound. In increasing order of difficulty, I suspect the display, the USB port, and Bluetooth radio will follow. Once we get the low-level bindings in place, however, we should be able to write the majority of the NXT port in occam-pi or 42, which will make development a good deal more pleasant.

The LEGO Mindstorms NXT developer kits (software, bytecode/VM reference, hardware, and Bluetooth) have been released.

This means the clock is officially ticking. Porting can begin.