Sub Ubi

Jon and Carl have really pulled out all the stops on this one. It could make you weep it’s so beautiful. What you see here is a screenshot of the typical jEdit interface. We use jEdit because it is open source, lightweight, and runs on Windows, Mac, and Linux without too much fuss. The code in the window is actually a near-copy of the original Surveyor firmware, translated into occam-pi. (Click on the images to see big versions…)

The first thing we need to do is fire up the occPlug. This is our plugin for compiling and running occam programs on a number of different platforms. Currently, the Windows release supports the desktop, the Mindstorms RCX, and the Surveyor SRV-1.

Once the occPlug is started, we can hit the compile button, and as you can see, our code is compiled for the Surveyor SRV-1 target. This isn’t the real magic, though…

When I run this code after compiling it, it is uploaded to the Surveyor and kicked off. Note that this is how all user programs are treated; we’ve turned what used to be the only firmware on the Surveyor into just another program. This is important, because it means it is easy for users to extend the default firmware (so to speak) to work with existing tools on the market, as well as incorporate their own algorithms for processing images, or navigation, and so on.

When I hit run, I get connected up to the Surveyor automatically. You see that “Command” area down below? I can type messages in and send them to my SRV-1 with no fuss whatsoever. Type, hit return (or the “Send” button), and my characters are whisked away into the aether through the magic of 802.11g. One of the commands I can send is the I command (meaning, I typed the letter I and hit return). Why “I”? I don’t know; we’d have to ask Howard. The important thing is that the I command sends me an image. And Jon and Carl have done a wonderful job here; when the terminal catches the header for the JPEG-compressed image coming back, it opens up an image pane, and displays what the Surveyor sees. We don’t have to do anything!

Here, I’ve sent the a command followed by I, and the image pane automatically resizes. Because the a command is interpreted to mean to set the image size to 160 x 128, I get a smaller image back. I then sent an A followed by an I…

And you can’t tell exactly, but that’s a really big picture of my kitchen floor in the background. It took a little longer to send than the tiny image, but we love those big pictures for their general awesomeness. The last thing I did was dial the size of the image back, and fired up my laser cannons. Those things are awesome.

How long does it take to compile that firmware? Let me see… I’m running Windows as a VMWare virtual machine on my MacBook Amateur (2×2GHz, 2GB RAM), and it takes roughly “one Mississippi.” How long does it take to upload the program? About as long. That alone has made developing on the Surveyor under occam-pi an absolute joy: no JTAG, no long compilation, just edit, compile, run, explore, tweak, try again… nice and easy.

Now that things are falling into place, we can start doing some very cool things, I believe. Earlier today, I was exploring the YUV colorspace, and wondering how hard it was to see an orange. Well, my first attempt was pretty lame:

As you can see, I’m highlighting pixels that I believe should be orange-colored. Oops. Well, given that I was squinting at a two-dimensional YUV colorspace plot, it wasn’t a bad guestimate. Note that I apparently have a better orange detector than a baseball detector:

Don’t read too much into this, though. I mean, I was mostly figuring out what Carl and Jon had done, and taking the occasional screenshot.

So, what’s the punchline? Jon and Carl have assembled an excellent IDE for exploring occam-pi and robotics on the Surveyor SRV-1. We’re going to be releasing this to students this weekend, and it goes live for laboratories in the Robotics class at Olin on Monday. We’re using the Surveyor to explore challenges of vision-based navigation in conjunction with simple state machines and behavior engines. What’s more, they’ll be learning how to architect these solutions in parallel-safe ways from day one. I think this laboratory (this is only one 2-week lab of four) is part of what makes Olin’s robotics class completely unique in the world today.

And yeah, I’m pretty jazzed that the software that we’ve all worked on for so long is being put to use for the purposes it was intended. That, really, is the awesome part.

Update: Does it work over the wireless? Yes. This is part of what we like about the Blackfin Surveyor—no wires. You write your code, hit compile, hit run, and the code goes through the air to your robot. You can then talk back and forth completely over the WiFi. And to answer the second qusetion: does it work on the Mac?

You tell me? Because everything in the IDE is WiFi and Java, there’s no real platform dependency. We’ve done a Windows build that will be distributed to Olin students momentarily, and want to discover any obvious problems. Once we’ve done that, we’ll do Mac and Linux builds. Doing releases is not a process that we’ve sufficiently streamlined—something that perhaps needs to be improved.

Regardless, yes, the Mac is a full-peer in this enterprise. Given that the Transterpreter was written entirely on Macs, we’re happy to support it.

(That’s a shot from Jon’s Surveyor, which lives in England and gets nice views out over Canterbury.)

I’ve discovered that programming while helping students and working with colleagues (both in the room and across the ocean) is actually distracting enough to prevent you from writing good code.

By coming in around 6AM this morning, I got some good quiet time that I was able to use productively, and in a short amount of time wrapped my head around YUV 4:22 (UYVY), and got some drawing routines ported into our Surveyor package. I have some questions for Carl and Jon about this, but for the moment, I’m glad my explorations are (mostly) over.

Last night, I got as far as wondering why I was able to make my images green:

This morning, I quickly isolated the luma component (Y) of the image:

Then, it was on to drawing. I think he sub-sampling for JPEG conversion is confusing me, but we can reliably draw shapes over the image:

Now, when students start doing image processing, they can circle or box in regions they are interested in, and see if their algorithms are returning values that make sense. (They can also print debugging information back to the PC, which is useful as well.) Between these two modes of interaction, I’d say we have a reasonably rich platform/API developing upon which students can develop parallel-safe robotics applications on the SRV-1 using occam-pi.

NOTE: We will be unleashing the things you read about here on students at Olin College very soon. We’ll bundle up a release for general consumption (as well as the documentation that we alpha test on the Olin students) Real Soon Now(TM).

I’ve been experimenting with the new firmware for the Surveyor SRV-1b that Carl and Jon have done a lot of excellent work on. Here’s something wacky: what used to be the “firmware” can now be executed as “user code”.

I’ll get there in a few steps. Consider this piece of test code that Jon wrote:

PROC tests (CHAN BYTE in?, out!, CHAN P.LASER lasers!,
            CHAN P.LED leds!, CHAN P.MOTOR motors!)
  SEQ
    out.string("SRV-1 Test Program (of Doom)*n", 0, out!)
    out.string("Testing death lasers*n", 0, out!)
    test.lasers(lasers!)
    out.string("Testing less deadly LED*'s*n", 0, out!)
    test.leds(leds!)
    out.string("Testing harmless motors*n", 0, out!)
    test.motors(motors!)
:

This is is the main process from Jon’s test program. The parameters coming into the process header are channels out to the environment. In this case, the environment is the Surveyor, so we have channels for the serial communications over the WiFi radio (those are the channels `in’ and `out’), and there are output channels for the lasers, the LEDs, and the motors. Each of these are named as you might expect. To run this on the Surveyor, we compile it, upload the bytecode, and things just go.

But what’s great is that this program, although it doesn’t do much, is not really different than the firmware that used to be written in C. The original firmware would handle commands from the SRV Console, and then spit images back, drive around, and do whatever else you commanded your mobile wireless camera platform to do. That is, all the default firmware did was respond to commands over a textual protocol. (There was a C interpreter, to. But my point is that you used a textual protocol to initiate all kinds of things.) Given that we have a channel of bytes representing the textual input coming over the radio, it seems like we could implement the old protocol completely in occam-pi.

And that’s what Carl and Jon have done. They’ve implemented what used to be firmware as a program that any user can now write and upload to the Surveyor. The program srv1.occ can be compiled, uploaded to the Surveyor, and executed as a user program, even though it is implementing the entire SRV-1 protocol. It is now a “user program” that implements what I’ve referred to as the “old firmware.” If we want to kill this “new firmware,” we issue a ‘!’, and it is shut down cleanly. Now, we can upload a newer version, or perhaps a completely different program.

This has its tradeoffs. For example, it means that my SRV-1 doesn’t wake up ready to send me images. On the other hand, it does mean that I can easily modify and extend the firmware, including extending the protocol or (because occam-pi is a parallel-safe language) running my own additional code along with the original firmware. I can filter the channel carrying the commands (perhaps ignoring every other request for an image, or drawing on the images to add data to them), and so on. Over time, we’ll probably end up refactoring the “firmware” into a bunch of reusable components that program authors can selectively include in their programs to get parts of the original firmware’s behavior in their programs. (We’ll see… I haven’t given this much thought yet, but perhaps Carl and Jon have.)

This is still evolving rapidly, but it is an absolute joy to be able to easily modify my occam-pi code, send it over the WiFi to the Surveyor, and get completely new, low-level behavior without having to go through a lengthy reflashing of the bot over JTAG (or, worse, WiFi).

As can be seen above (sorta), I’ve begun to explore drawing on the images before shipping them from the SRV-1 back to the user. I believe this is important for students exploring robotic vision, as they need a way to indicate what they are looking for; drawing onto the image strikes me as a very simple way for their code to communicate that “this looks important!”. It might be that they’re doing edge detection, or blob finding, or any of a host of other things. My code doesn’t do anything exciting yet, but tomorrow is another day; more excitement will ensue.

Nicely done to the UK team! Wootness.

I just wanted to say that Roborealm looks like a very nicely done set of tools for exploring robotics and vision. I haven’t had a chance to play with it yet, but it is something I’d like to do.

The team is nearing completion of a new firmware for the SRV-1 that provides low-level, low-latency parallelism on the Blackfin-based Surveyor. Some excellent work has been done by Carl and Jon on this port, and I’m very excited to see it in the hands of students this semester.

When we release that documentation, I’ll also release the tutorial booklet that students used on the LEGO Mindstorms at the start of the term. It provided a questions-first introduction to occam-pi on the Mindstorms RCX. These documents, taken together (and iterated/revised a bit) will provide, I think, an excellent introduction to the design of concurrent and parallel algorithms for robotic control on some very cool platforms.

Watch this space!

The last post on this blog was in October. Since then, things have been busy. Over the next few posts, I’ll try and catch up a bit.

For the last three months, we’ve been working hard on the new Blackfin Surveyor SRV-1.

Blackfin SRV-1, image stolen from the Surveyor webpages

We first encountered the Surveyor at AAAI 2007. When Howard “Surveyor Daddy” Gordon updated the Surveyor from a 60MHz ARM with a few KB of RAM to a 500MHz Blackfin with 32MB of RAM, we took notice. And, even better, when it sprouted that oh-so-excellent WiFi tail, we really took notice. (It is so much easier to work with WiFi than the ZigBee radio that was there. At least for us it is.)

This semester, I’m co-teaching ENGR 3390: Robotics with Dave Barrett at Olin College, and we’re using the Surveyor as the platform for the mobile portion of the course. We’ll be using the Surveyor as a platform as a demonstrator where we can introduce students to architectures for cognition on robotic platforms in a parallel-safe way using the Transterpreter and occam-pi. And what we realized very quickly is that the SRV-1 looks like it is just begging to be beaten all to hell in the classroom.

See that processor on top, all nice and exposed? After a team of students shuffles in from the dry, Boston winter—BZZAPT!. No more Blackfin. One good static discharge, and the processor or RAM are gone. Or, those little laser diode wires? Oops! A careless gesture, and they’re yanked out. Or what about a drop from the counter-top? I’m not confident the board would handle the fall very gracefully.

Dave threw together a quick Solidworks model, and dumped it out to Olin’s rapid prototyping machine. This thing is incredible: it prints in 3D in ABS plastic. Send the model, step back, and 12 hours later you have this:

That is a battle-ready SRV-1. The laser mount was removed, and the diodes inserted into the body, which was simply printed with the diode mounts in the right place. The camera hole is oversize, and easily gives the camera a full field of view.

As you can see, the rapid prototyping machine has enough resolution to render depressed text in the body of the bot. This one clearly declares itself as property of Olin’s ENGR 3390: Robotics course.

What you may or may not be able to tell is that the entire top of the box has mounting holes at 1/2″ spacing perforating it; it is amazing what the RP machine can do. This means we can easily screw-mount sensor towers or anything else we might like to the top of the bot quickly and easily, running wires straight down through the body to the SRV-1 board.

Generally, the pictures are low quality because they were taken with the webcam built into my Macbook. I’ll get better pictures at a later point.

The first thing we’re going to have the students in the course do is take the basic Solidworks model and design bodies for their team’s bots. This way, each bot will have a unique body, and the students will have some ownership of the process. However, we already have a few things we want to improve in the basic body before we turn them loose:

• The body doesn’t have anywhere to grab on to in the back; as you can see, it is just resting on the SRV-1’s aluminum body. Our plan is to wrap around inside of the treads, where there is just enough clearance for us to sneak the body down. This way, we provide more protection for the boards from grit and the like, and can attach across the base of the bot on the underside.
• We need blinkenlight mounting holes build into the body. This way, you can just push an LED up from underneath, and it will sit flush with the housing. We can wire them into the GPIO pins inside, and have all the wires hidden from view, and safe from snags on obstacles.
• A view hole needs to be left for viewing the state of the three LEDs on the Blackfin’s motherboard.
• A mounting point for a header board could be built into the body, so sensors could be attached and detached outside; this depends on whether we expect to be doing much of this kind of prototyping with different sensors or not.
• Mount points flush with the body wall for IR and other sensors on the front/back/sides could be very useful. We have a number of interesting sensors for the students to characterize and use, and mounting them safely inside the body would be a good idea.
• The antenna should probably be above the ground plane; we could run a cable from the current point, under the body, and up to the top, where we could easily grow an antenna mount.
• We have blue and yellow plastic coming in, which will produce a lasting color. Otherwise, we have to paint, which is less durable and a mess to work with. (Well, as messy as spray paint ever is, anyway.)
• I want my robot’s body covered in dimples (and maybe some through mounting holes) that are compatible with a certain plastic building toy that rhymes with Tierra del Fuego.

Those were the immediate improvements that we saw could be made.

We’re really excited about the course, and I think both Dave and I are pretty excited about how cool the bodies for the Surveyorcould be. What’s really awesome is that they are printed in ABS plastic, and have a wall thickness of roughly 3/16″ of an inch. (I haven’t measured; that’s just my eyeball estimate from memory.) I pounded on it with my fist (off the Surveyor), and I hurt my knuckles. In short, the body is tough. The ultimate goal is to make it possible for a student, in their excitement, to give the Surveyor a kick on the field of combat (or drive it off a table), and have the bot be in a good condition to survive the fall.

Over the next two weeks, this body will evolve a lot. We’re going to be playing with it, and then 24 of Olin’s energetic, creative engineers-to-be are going to rapid-fire evolve it to their hearts content. I think we’ll end up with something very, very cool. I’ll post pictures and Solidworks models as the body evolves.