Sub Ubi

Jon and Carl have really pulled out all the stops on this one. It could make you weep it’s so beautiful. What you see here is a screenshot of the typical jEdit interface. We use jEdit because it is open source, lightweight, and runs on Windows, Mac, and Linux without too much fuss. The code in the window is actually a near-copy of the original Surveyor firmware, translated into occam-pi. (Click on the images to see big versions…)

The first thing we need to do is fire up the occPlug. This is our plugin for compiling and running occam programs on a number of different platforms. Currently, the Windows release supports the desktop, the Mindstorms RCX, and the Surveyor SRV-1.

Once the occPlug is started, we can hit the compile button, and as you can see, our code is compiled for the Surveyor SRV-1 target. This isn’t the real magic, though…

When I run this code after compiling it, it is uploaded to the Surveyor and kicked off. Note that this is how all user programs are treated; we’ve turned what used to be the only firmware on the Surveyor into just another program. This is important, because it means it is easy for users to extend the default firmware (so to speak) to work with existing tools on the market, as well as incorporate their own algorithms for processing images, or navigation, and so on.

When I hit run, I get connected up to the Surveyor automatically. You see that “Command” area down below? I can type messages in and send them to my SRV-1 with no fuss whatsoever. Type, hit return (or the “Send” button), and my characters are whisked away into the aether through the magic of 802.11g. One of the commands I can send is the I command (meaning, I typed the letter I and hit return). Why “I”? I don’t know; we’d have to ask Howard. The important thing is that the I command sends me an image. And Jon and Carl have done a wonderful job here; when the terminal catches the header for the JPEG-compressed image coming back, it opens up an image pane, and displays what the Surveyor sees. We don’t have to do anything!

Here, I’ve sent the a command followed by I, and the image pane automatically resizes. Because the a command is interpreted to mean to set the image size to 160 x 128, I get a smaller image back. I then sent an A followed by an I…

And you can’t tell exactly, but that’s a really big picture of my kitchen floor in the background. It took a little longer to send than the tiny image, but we love those big pictures for their general awesomeness. The last thing I did was dial the size of the image back, and fired up my laser cannons. Those things are awesome.

How long does it take to compile that firmware? Let me see… I’m running Windows as a VMWare virtual machine on my MacBook Amateur (2×2GHz, 2GB RAM), and it takes roughly “one Mississippi.” How long does it take to upload the program? About as long. That alone has made developing on the Surveyor under occam-pi an absolute joy: no JTAG, no long compilation, just edit, compile, run, explore, tweak, try again… nice and easy.

Now that things are falling into place, we can start doing some very cool things, I believe. Earlier today, I was exploring the YUV colorspace, and wondering how hard it was to see an orange. Well, my first attempt was pretty lame:

As you can see, I’m highlighting pixels that I believe should be orange-colored. Oops. Well, given that I was squinting at a two-dimensional YUV colorspace plot, it wasn’t a bad guestimate. Note that I apparently have a better orange detector than a baseball detector:

Don’t read too much into this, though. I mean, I was mostly figuring out what Carl and Jon had done, and taking the occasional screenshot.

So, what’s the punchline? Jon and Carl have assembled an excellent IDE for exploring occam-pi and robotics on the Surveyor SRV-1. We’re going to be releasing this to students this weekend, and it goes live for laboratories in the Robotics class at Olin on Monday. We’re using the Surveyor to explore challenges of vision-based navigation in conjunction with simple state machines and behavior engines. What’s more, they’ll be learning how to architect these solutions in parallel-safe ways from day one. I think this laboratory (this is only one 2-week lab of four) is part of what makes Olin’s robotics class completely unique in the world today.

And yeah, I’m pretty jazzed that the software that we’ve all worked on for so long is being put to use for the purposes it was intended. That, really, is the awesome part.

Update: Does it work over the wireless? Yes. This is part of what we like about the Blackfin Surveyor—no wires. You write your code, hit compile, hit run, and the code goes through the air to your robot. You can then talk back and forth completely over the WiFi. And to answer the second qusetion: does it work on the Mac?

You tell me? Because everything in the IDE is WiFi and Java, there’s no real platform dependency. We’ve done a Windows build that will be distributed to Olin students momentarily, and want to discover any obvious problems. Once we’ve done that, we’ll do Mac and Linux builds. Doing releases is not a process that we’ve sufficiently streamlined—something that perhaps needs to be improved.

Regardless, yes, the Mac is a full-peer in this enterprise. Given that the Transterpreter was written entirely on Macs, we’re happy to support it.

(That’s a shot from Jon’s Surveyor, which lives in England and gets nice views out over Canterbury.)

I’ve discovered that programming while helping students and working with colleagues (both in the room and across the ocean) is actually distracting enough to prevent you from writing good code.

By coming in around 6AM this morning, I got some good quiet time that I was able to use productively, and in a short amount of time wrapped my head around YUV 4:22 (UYVY), and got some drawing routines ported into our Surveyor package. I have some questions for Carl and Jon about this, but for the moment, I’m glad my explorations are (mostly) over.

Last night, I got as far as wondering why I was able to make my images green:

This morning, I quickly isolated the luma component (Y) of the image:

Then, it was on to drawing. I think he sub-sampling for JPEG conversion is confusing me, but we can reliably draw shapes over the image:

Now, when students start doing image processing, they can circle or box in regions they are interested in, and see if their algorithms are returning values that make sense. (They can also print debugging information back to the PC, which is useful as well.) Between these two modes of interaction, I’d say we have a reasonably rich platform/API developing upon which students can develop parallel-safe robotics applications on the SRV-1 using occam-pi.

NOTE: We will be unleashing the things you read about here on students at Olin College very soon. We’ll bundle up a release for general consumption (as well as the documentation that we alpha test on the Olin students) Real Soon Now(TM).

I’ve been experimenting with the new firmware for the Surveyor SRV-1b that Carl and Jon have done a lot of excellent work on. Here’s something wacky: what used to be the “firmware” can now be executed as “user code”.

I’ll get there in a few steps. Consider this piece of test code that Jon wrote:

PROC tests (CHAN BYTE in?, out!, CHAN P.LASER lasers!,
            CHAN P.LED leds!, CHAN P.MOTOR motors!)
  SEQ
    out.string("SRV-1 Test Program (of Doom)*n", 0, out!)
    out.string("Testing death lasers*n", 0, out!)
    test.lasers(lasers!)
    out.string("Testing less deadly LED*'s*n", 0, out!)
    test.leds(leds!)
    out.string("Testing harmless motors*n", 0, out!)
    test.motors(motors!)
:

This is is the main process from Jon’s test program. The parameters coming into the process header are channels out to the environment. In this case, the environment is the Surveyor, so we have channels for the serial communications over the WiFi radio (those are the channels `in’ and `out’), and there are output channels for the lasers, the LEDs, and the motors. Each of these are named as you might expect. To run this on the Surveyor, we compile it, upload the bytecode, and things just go.

But what’s great is that this program, although it doesn’t do much, is not really different than the firmware that used to be written in C. The original firmware would handle commands from the SRV Console, and then spit images back, drive around, and do whatever else you commanded your mobile wireless camera platform to do. That is, all the default firmware did was respond to commands over a textual protocol. (There was a C interpreter, to. But my point is that you used a textual protocol to initiate all kinds of things.) Given that we have a channel of bytes representing the textual input coming over the radio, it seems like we could implement the old protocol completely in occam-pi.

And that’s what Carl and Jon have done. They’ve implemented what used to be firmware as a program that any user can now write and upload to the Surveyor. The program srv1.occ can be compiled, uploaded to the Surveyor, and executed as a user program, even though it is implementing the entire SRV-1 protocol. It is now a “user program” that implements what I’ve referred to as the “old firmware.” If we want to kill this “new firmware,” we issue a ‘!’, and it is shut down cleanly. Now, we can upload a newer version, or perhaps a completely different program.

This has its tradeoffs. For example, it means that my SRV-1 doesn’t wake up ready to send me images. On the other hand, it does mean that I can easily modify and extend the firmware, including extending the protocol or (because occam-pi is a parallel-safe language) running my own additional code along with the original firmware. I can filter the channel carrying the commands (perhaps ignoring every other request for an image, or drawing on the images to add data to them), and so on. Over time, we’ll probably end up refactoring the “firmware” into a bunch of reusable components that program authors can selectively include in their programs to get parts of the original firmware’s behavior in their programs. (We’ll see… I haven’t given this much thought yet, but perhaps Carl and Jon have.)

This is still evolving rapidly, but it is an absolute joy to be able to easily modify my occam-pi code, send it over the WiFi to the Surveyor, and get completely new, low-level behavior without having to go through a lengthy reflashing of the bot over JTAG (or, worse, WiFi).

As can be seen above (sorta), I’ve begun to explore drawing on the images before shipping them from the SRV-1 back to the user. I believe this is important for students exploring robotic vision, as they need a way to indicate what they are looking for; drawing onto the image strikes me as a very simple way for their code to communicate that “this looks important!”. It might be that they’re doing edge detection, or blob finding, or any of a host of other things. My code doesn’t do anything exciting yet, but tomorrow is another day; more excitement will ensue.

Nicely done to the UK team! Wootness.

I just wanted to say that Roborealm looks like a very nicely done set of tools for exploring robotics and vision. I haven’t had a chance to play with it yet, but it is something I’d like to do.

The team is nearing completion of a new firmware for the SRV-1 that provides low-level, low-latency parallelism on the Blackfin-based Surveyor. Some excellent work has been done by Carl and Jon on this port, and I’m very excited to see it in the hands of students this semester.

When we release that documentation, I’ll also release the tutorial booklet that students used on the LEGO Mindstorms at the start of the term. It provided a questions-first introduction to occam-pi on the Mindstorms RCX. These documents, taken together (and iterated/revised a bit) will provide, I think, an excellent introduction to the design of concurrent and parallel algorithms for robotic control on some very cool platforms.

Watch this space!

Michael opens up on ICT education in the UK in PC Pro:

It’s like giving children a calculator but never explaining the underlying principles of multiplication. “Teaching children office-automation skills borders on child abuse,” says Dr Michael Kölling, the University of Kent academic, who’s attempting to get the Greenfoot software we’re using for our tutorial into schools across the country. “The main problem for us is they call it computing. Kids think that’s computer science. It isn’t productive, it doesn’t stimulate interest. Children should be creative, they should get the joy and satisfaction that comes from seeing their ideas take shape.”

His point is that Greenfoot is a much more engaging environment for students to be programming in then, say, doing spreadsheets in Excel. He’s right: forcing 13-year-old students to learn Excel macros is a form of child abuse.

So, tomorrow I will be speaking to a group of Olin students who have put together a self-directed course titled “Vital Ideation.” Apparently, because I have a notebook, I’ve been invited to speak about the role of notebooks in the ideation process.

Mind you, I have a lot of notebooks. I had no idea how many I had, until I started putting together a presentation about my notebook practice. At Olin, I have all of my notebooks from my entire PhD process. Kinda cool.

But what made me laugh was their description of me on the course homepage on the Wikiversity.

Matt Jadud is a visiting professor at Olin College in electrical engineering and computer science. He has interests in concurrency in robotics and computing, human factors interface design, and software design. Although he is a visiting professor and still alive, students already have hopes of establishing a Matt Jadud Professor of Computer Science chair and awarding it to him.

I think I need to let students write my bio all the time. I especially like the bit “Although he is a visiting professor and still alive“. I mean, it makes me wonder how long I’ve got until they do me in.