The Hand That Assembles Without Looking

In 2023 a robot at the Robotics Institute at Carnegie Mellon inserted pegs into holes with 0.1 millimetre clearance using tactile feedback alone. No vision in the loop at all.\\n\\nThe number that makes that interesting is not the clearance. It is the comparison. A vision-guided assembly cell has to hold its own positioning accuracy tighter than the tolerance it is aiming at, so a 0.01 millimetre fit demands a fixture, a calibration routine and a repeatable arm that cost more than the task is worth. A hand that can feel the hole only needs to get within about 0.5 millimetre and then search. The precision requirement falls by a factor of fifty, and the search is the part that was always going to be cheap.\\n\\nVision cannot make that trade. It has to know, before it moves, where the hole is. Touch is allowed to find out during the move. That is the whole difference between the two senses, and it shows up as a bill of materials.

The demonstrations that followed in 2024 and 2025 were chosen well. A robot followed a cable through its own fingers and routed it through guides. A robot folded and smoothed cloth with sensorized grippers.\\n\\nCable and cloth and peg are three different problems, and they share one property: the moment the hand closes on the object, the camera loses the contact point. A cable tip disappears into a grip. A cloth layer is inside the pinch. A peg is in the hole, behind the wrist. Vision gives global context, what the object is and roughly where, and touch gives contact detail, the surface, the compliance, the force, the onset of slip. In biology the split is the same. The two channels are not redundant, and they are not interchangeable.\\n\\nThe transferable reading for anyone buying this equipment is that the demonstrations are the specification. Three contact-rich tasks, no camera in the decisive loop, and each one a case where the sense that had to do the work was the one in contact with the world.

A person detects the onset of slip 60 to 80 milliseconds before the object actually moves. Conscious reaction time is around 200 milliseconds. The gap is not skill. The grip tightens at the spinal level before the conscious mind is aware anything is wrong, driven by RA-I afferents reporting the 5 to 50 hertz micro-vibrations that precede macroscopic movement.\\n\\nMatch that with hardware and the requirements are unforgiving: detection latency under 10 milliseconds, grip response under 50, sensitivity to sub-micrometre vibration at the contact surface. The physics gives you four signals to work with. A partial slip front propagates inward from the contact edge. Asperities breaking and re-forming emit 50 to 500 hertz vibrations. The ratio of tangential to normal force climbs toward the friction coefficient. Static contact area shrinks.\\n\\nThe fastest route is a piezoelectric film at the surface, bandpassed to the slip band, which detects in 0.5 to 5 milliseconds at a threshold of 0.01 to 0.1 metres per second squared with a false positive rate of 1 to 5 per cent. The robust route is force-ratio tracking, watching the tangential to normal ratio approach a static friction coefficient that sits somewhere between 0.3 and 1.0 and is not known in advance for an unfamiliar object. The best published result in 2024 combined both and reported detection under 5 milliseconds at better than 95 per cent accuracy across 50 objects. The fastest reflex on record is 15 milliseconds from slip onset to increased grip force, on dedicated hardware rather than a central processor.\\n\\nNo off-the-shelf slip detection module exists for any major robot hand. Every group builds its own. That is the gap in the market, and it is not the one the sensor sales decks describe.

Vision runs at 30 to 60 hertz. Touch runs at 100 to 1,000. Fusing them is not a nice-to-have; it is the part that decides whether the numbers on the sensor matter.\\n\\nFour architectures exist and each one charges a price. Fuse the inputs early and the tactile image and the video frame have to share a representation despite running at wildly different resolutions. Fuse in the middle and each encoder gets its own processing, but someone has to pick the layer and cross-attention between a vision transformer and a taxel array is expensive. Fuse only at the decision and each channel can be developed alone, but the system can never learn that tactile roughness agrees with visual texture. Cross-attention, where visual and tactile tokens attend to each other, is the current best and it is also the most compute-hungry.\\n\\nThe returns are large enough to justify it. A policy trained on both senses reached 94 per cent success on cable insertion in 2025, against 72 per cent with vision alone and 68 per cent with touch alone. Neither sense on its own gets close to what the two together get. Temporal alignment is the practical headache underneath: either downsample the touch stream to the camera, interpolate between video frames when slip is in play, or trigger a frame only when touch reports an event. The last option is the most efficient and the least used.

Vision needs precision better than the tolerance. Touch does not need precision at all. It searches.

The sensor lineage is short and its milestones are dated. GelSight was invented at MIT in 2009. GelSlim compressed it into a fingertip in 2014. DIGIT, OmniTact and the soft bubble arrived open-sourced between 2019 and 2020, mostly from Meta AI. Commercial sensors were on sale by 2022, integrated into humanoid hands through 2023 and 2024, and in a first industrial deployment on an assembly line in 2025.\\n\\nAgainst that, the performance figure that matters is not resolution. Hands fitted with vision-based tactile sensors complete contact-rich tasks at around 90 to 95 per cent success where non-tactile hands manage 60 to 75 per cent. The sensor is being judged by the tasks its hand completes, not by the numbers on its own sheet. The groups that understood this early stopped optimising the transducer and started building complete hands, then let the demonstrations set the specification.\\n\\nThere is a second reason that order works. A manipulation system has to solve a fragile-and-firm trade-off on every grip: too little force and the object drops, too much and it crushes. The answer is the minimum force that prevents slip, and that force changes with the weight, the friction and the fragility of each object. Only a closed loop on contact can find it. A datasheet cannot.

The honest reading of this field is that the sensing problem is close to finished and the integration problem is not. A coarse pressure array with a working slip reflex outperforms a millimetre-resolution taxel map with none, and that sentence has been true since Howe and Cutkosky in 1989.\\n\\nSo the probability that a tactile system works in production is set far less by which transducer a team selects than by whether it builds the whole hand, wires the reflex locally, and picks its three demonstration tasks before it picks its sensor. Teams that start from the datasheet end up with a hand that can feel everything and assemble nothing. Teams that start from the task end up with a peg in a hole nobody can see.\\n\\nThe pattern generalises past robotics. A component curve improves, the component gets bought, and the constraint quietly relocates to the layer where nobody is selling anything. The only way to catch that is to test the whole task, not the part.