The odometry post ended on a promise. Odometry tells you how the duck has moved since it booted. It is good, but it drifts. On the real robot we measured it over 47.5 m of walking, cut into twenty walks of half a metre to five metres. At the end of a walk it was 6 cm off on median, and 12 cm at worst: about 2.4 cm of error for every metre walked. Its heading also ended about 2° off, and that matters more than it sounds. With a 2° error, every metre walked afterwards puts the duck 3.5 cm to the side of where odometry thinks it is. So the longer the duck walks, the further its estimate drifts from the truth. After a few minutes of play, it has little to do with the room.
It gets worse when somebody picks the duck up, because odometry cannot feel it. Carry the duck to the other side of the room, and odometry carries on from where it was lifted, a metre or more off. Nothing it measures will ever tell it so.
To know where it is, the robot has to look at the room and remember what it saw. It has to build a map, and find itself in that map, at the same time. This is called SLAM: Simultaneous Localization and Mapping. This post is about our SLAM, and about how we learned to measure it instead of tuning it by eye.
Microduck has a single depth sensor: a VL53L5CX time-of-flight sensor in its head, about 25 cm above the floor. Fifteen times a second, it measures 64 distances, in an 8×8 grid that covers a 45° cone in front of the head. Only the distances under about a metre are reliable. That is all the robot sees for mapping.
From these distances and the leg odometry, the robot builds a map of the room. It keeps the map
when it is switched off, and finds itself in it again when somebody puts it down somewhere else.
Everything runs on the robot, on the four small Cortex-A55 cores of its Rockchip RK3566, next to
the walking policy. There is no laptop and no cloud involved. The code is a small service called
mapd.
Seven and a half minutes, about 7× faster. The camera in the corner of the room and the tags on the floor are only there to measure where the duck really is: the duck itself uses neither. The overlay is drawn on the floor from the robot's own data. Magenta is where the mapper thinks the duck is, blue is odometry alone, green is the truth. Red cells are obstacles in the map the duck is building, cyan is floor it has seen empty. Each time the duck is carried, odometry has no way to know, and the mapper has to find itself again.
What 64 distances can see
A camera sees a whole room at once. Our sensor sees a small patch of it. At 50 cm, the whole 8×8 grid covers a patch about 40 cm wide, and its lower rows mostly see the floor. So one measurement shows a piece of wall, or a chair leg. It says almost nothing about where along that wall the robot is. For comparison, the lidar on a robot vacuum measures all around it, several times a second.
Half a minute through the duck's eyes, in real time. Left, its head camera, which the mapper does not use, with the sensor's 64 zones drawn where they measured. Right, the 64 numbers the mapper actually gets, in metres. Hollow cells came back with nothing usable: facing the sofa two metres away, almost the whole grid is empty.
The mapper works at stops
While the duck walks, the mapper simply follows odometry. When the duck has stood still for half a second, the mapper starts collecting every frame the sensor sends, until the duck moves again. All these frames together make one keyframe: a small local map of what the duck saw from that spot. If the head moves during the stop, the keyframe covers more of the room.
Then the mapper places the keyframe in the map. It tries every position and heading in a small window around where odometry says the duck is. For each one, it scores how well the keyframe's obstacles land on the obstacles already in the map. The best score is the answer. How quickly the score drops around it says how sure that answer is, and in which directions. In front of a single flat wall, for example, the scan pins the distance to the wall, but not the position along it. The mapper then trusts the match for the distance only, and keeps odometry's guess for the rest.
One real stop from the dataset, step by step. While the duck stands, its head turns, and every frame of the sensor adds points to the keyframe. The keyframe is then placed where odometry says the duck is, tried at nearby poses, and kept at the one that fits the map best: here 9 cm and 3° away. The score shown is a simplified one; the mapper's own is a likelihood over every pose in the window.
This worked in our simulator. On the real robot, the map did not look like the room.
Measuring before tuning
When a map looks wrong, the temptation is to stare at it and change things until it looks better. We did that for a while. The problem is that a map that looks a little better proves nothing. So we stopped, and built a way to measure.
Eight ArUco tags taped to the floor define the room's coordinates. An Orbbec Femto Bolt camera, fixed high in a corner, films every session in 4K. In each frame, it finds a 6 cm tag stuck on top of the duck's head. A small tag seen from four metres away gives its direction well, but its distance badly, so we do not use the distance at all. We use the height of the head instead, which the robot knows from its joint angles. The head is where the camera's line of sight to the tag reaches that height. This places it to about a centimetre.
The camera and the robot each have their own clock, so we have to line them up. Both see the head turn: the camera through the tag, the robot through its gyroscope and head joints. Matching the two turning speeds aligns the clocks to within 10 ms over a seven-minute session, including the 130 ppm by which they drift apart.
Before the sessions, we also scanned the empty room with the same camera, carried by hand. Each frame is placed using the floor tags it sees, or by matching it to the frames around it when it sees none. Together, the frames make a 3D point cloud of the room. We only keep the points between 3 and 40 cm above the floor, the heights the duck's sensor can see. Seen from above, these points are our reference map, and we compare the duck's own map with it.
With all this, we can replay a recorded session through the mapper on a laptop, and check every decision it makes against where the duck really was. A replay takes a couple of seconds, so we can measure each idea instead of guessing. The recordings are public, in the SLAM_JJ_Salon_with_GT dataset on the Hugging Face Hub, with the head camera's video too, if you want to test your own SLAM against the same ground truth.
What the truth found
The first thing we learned had nothing to do with the mapper. If we place the sensor's 64 measurements using the robot's true position, they should land on the real walls and furniture. They did not, quite. Three small corrections to how we read the sensor fixed most of it. First, the VL53L5CX reports the distance along its optical axis, not along each zone's own direction, which is up to 11 % shorter in the corners. Second, each frame is about 60 ms old by the time we get it, not 10 ms. Third, the sensor sits about 3° lower in the head than the robot's model says.
The second finding was about trust. The matcher was confident, and often wrong. A narrow scan of a wall fits the same wall 30 cm further along just as well, and it scored 95 %. Meanwhile, the leg odometry that these matches were correcting turned out to be good to a few centimetres between two stops. So now a match is only believed if it stays close to odometry: within 10 cm and 3° of where odometry put the stop. A match further away than that is not a correction. It is the wrong wall.
With this limit, the matches are not much better than odometry alone on this session. Their real value is elsewhere: the map they build is what lets the duck find itself again.
Picked up, or bumped?
The robot has a learned pick-up detector, which stops the walking when somebody lifts the duck. For the mapper, a pick-up means the feet have left the floor. Odometry's position means nothing until the duck is put down, and the duck is lost until it recognises where it is. But not every pick-up is a carry. A stumble triggers the detector for a fraction of a second, and so does a quick lift to straighten the robot. Getting lost over those, and having to find itself from scratch, would be worse than ignoring them.
The fixed camera in the corner of the room showed us what a real carry looks like:
Every bump lasted about 0.4 s and moved the duck less than 40 cm. Every carry lasted 1.5 s or more, and moved it more than a metre. Our first threshold was 2 s, and one quick 1.25 m carry slipped under it: the mapper took it for a bump, and kept tracking from the old place, over a metre off, for a whole minute. With the threshold at 1 s, a bump keeps tracking and only widens the search window for the next match. A carry makes the duck lost.
Sitting down is a special case. The pick-up detector only runs while the walking policy runs, so a duck that sits, is carried, and stands up again reports nothing. So when the duck stands up after sitting, the mapper treats it as a possible carry. It searches near where the duck sat first, then the whole map if that fails.
Finding itself again
When the duck is lost, the mapper searches the whole map at every stop. It keeps the heading, which the gyroscope still knows. With only 64 distances, a single stop's best match is often wrong: a straight wall fits anywhere along itself. So one stop is never enough. Two stops, a few seconds apart, must each agree on where the duck is. When a stop's best match has a close rival somewhere else, the mapper keeps both, and a later stop that agrees with exactly one of them decides.
A seated carry, about 2× faster. The duck sits, is carried 1.5 m, and stands up. Odometry (blue) never noticed, and carries on from the old place. The mapper goes lost when the duck stands up, searches the map at each stop, and finds itself 11 cm from the truth.
You can scrub through the whole seven-minute session below. On the timeline, the orange stretches are when the mapper says it is lost, the black ticks are carries, and the green lines mark each moment it found itself again.
Seven minutes of mapping, seen from above
How good is it?
On the two sessions in the dataset:
| session 2 (4 min) | session 3 (7.5 min, many carries) | |
|---|---|---|
| stop position error, median | 2.7 cm | 4.8 cm |
| worst stop | 6.6 cm | 15 cm |
| time it says where it is | 60 % | 63 % |
| …and is within 15 cm | 99 % of that | 94 % of that |
| relocalized to a wrong place | never | never |
In session 3, it found itself 2 cm off after a 1.46 m carry, 11 cm off after the 1.54 m seated carry, and 3 cm off after a fall. Most of the time it spends lost is not spent failing to recognise the room. It only searches at stops, and some carries came faster than it could stop twice. As for the map, 75–80 % of its obstacle cells are within 5 cm of a real obstacle.
We then ran the same code on the duck in a completely different room, which none of this tuning had ever seen. It found itself correctly three times.
It has its limits. The floor under a sofa is drawn as free, because the sensor's lower beams see under it. And since it only looks at stops, a duck that keeps walking stays lost longer than it needs to.
Now that it works, we want to keep it working. Every change to the mapper is replayed on both sessions, and on a dozen carries cut out of them, and compared with the scores of the version described here. A change does not go in if it loses a single recovery, or sends a single relocalization to the wrong place.
What comes next
Today the duck maps one room. The next challenge is knowing when it is somewhere new. Switch it off in the living room and on in the kitchen, and it searches its living-room map for a place it has never seen. In a room the map does not contain, two stops can even agree on a wrong place: we have seen it happen once. We want the duck to notice that the room is unknown, start a new map there, and in the end keep a map of each room it knows. It should also be able to map a room on its own: walk around, look, and go and fill in the parts it has not seen yet. That is what we are working on now.
A map also opens the door to much more interesting behaviours. A duck that knows where it is can go back to a favourite spot, come and find you in the room, explore the corners it has never visited, or wander around without bumping into the same sofa twice. Odometry alone could never keep that up for more than a few minutes.