There Is No Tactile ImageNet

Benchmarking in touch splits into two questions and the field answers one of them. Task-level evaluation asks whether the robot finished the job: the Yale-CMU-Berkeley object set scores grasping success rate, DARPA ARM scores time to complete assembly, the NIST assembly task board scores clearance achieved on peg insertion and gear placement. Sensor-level evaluation asks how sensor A compares with sensor B on the same number, and it is rare in the literature. The first question tells you the task can be done. The second tells you which hardware to buy. Only the second is procurement, and it is the one the papers leave open. A task benchmark is built on one machine. The machines are assembled by one lab around one sensor, so a success rate carries the hardware with it as an unstated variable. Swap the fingertip and the number loses its meaning. Nobody re-runs the whole board. That is the gap. Vision closed it with a dataset and a metric. Touch has not.

Vision moved fast because of one artefact: a standard dataset with a standard metric that every group reported. ImageNet-21K holds 14 million images and top-5 classification accuracy is the number in every abstract. Touch is missing the same three pieces. No shared dataset with multiple objects and multiple sensors in the open. No shared hardware platform with the same hand and the same sensor layout. No shared metric for what performance even means. What exists instead is five datasets in five years, each on its own instrument. Penn Haptic Texture Toolkit, 2019, on BioTac, 100 textures, about 100,000 samples, open. GelSight Texture Dataset, 2020, on GelSight, 50 surfaces, about 100,000 samples, open. Touch-and-Go, 2021, on DIGIT, 100 objects, about 1,000,000 samples, open. A visuo-tactile set in 2023 pairing GelSight with cameras, 80 objects, about 100,000 samples, open. TacBench in 2024, across multiple sensors, 50 objects, about 10,000 samples, research access only. Add all five together and the total is under 10,000,000 samples. One vision benchmark holds 14 million images. The scale gap is two to three orders of magnitude and it is not closing.

An image can be scraped. A tactile sample has to be taken, one contact at a time, with the instrument that will be sold. That is the cost structure nobody budgets for, and it explains why the corpora stay small while the papers multiply. It gets worse. A resistance map from BioTac and a camera image from GelSight are not the same modality, so a corpus collected on one instrument cannot be replayed on another. Every new sensor design restarts its dataset at zero. The corpus has a half-life set by the hardware cycle, and in this field the hardware cycle is somebody's doctorate. Vision never paid that price. A photograph of a cat stays a photograph of a cat when the camera changes. A touch recording stops existing when the fingertip does.

Two-point discrimination has measured spatial resolution in biology for a century. In electronic skin papers it is rare. Braille character recognition, which folds resolution and classification into one task, appears in some GelSight work. Texture discrimination across 10 surfaces is used by some groups and standardised by none. Force accuracy checked against NIST-traceable weights is rare, because most groups weigh their own. Cyclic loading has no standard protocol. Slip threshold has no standard protocol. Read that list again and notice which two are missing a protocol. Durability and slip detection. Those are the two failure modes that end a deployment, the one that stops a glove surviving a season and the one that drops the thing you were holding. The result is a field that can report a success rate for a task and cannot report a life expectancy for a component.

A task benchmark tells you the job can be done. A sensor benchmark tells you which hardware to buy.

Three levels have been proposed and every rung is measurable. Sensor level is the transfer function, force in and reading out, measured three ways: static, which is range, resolution, hysteresis, linearity and repeatability; dynamic, which is bandwidth, step response and slip detection latency; and environmental, which is temperature coefficient, humidity sensitivity and drift. Perceptual level asks whether the sensor can separate N levels of force, N surface textures, incipient slip at a perturbation measured in millinewtons, and place a contact to within millimetres. Task level is pick and place over N items, peg insertion by clearance and time, fragile handling by damage rate on an egg or a fruit, cable routing by completion rate, in-hand manipulation by success over degrees of rotation. The ladder is written down. What does not exist is a lab that runs all three levels on one sensor and publishes the sheet. Until one does, every comparison is assembled by hand out of two papers with different test rigs, and the buyer pays for that ambiguity.

None of this stops a purchase. It changes what you ask for. Ask which instrument produced the number and on which surfaces. Ask for the transfer function rather than the sensitivity headline. Ask how the sensor was loaded and for how many cycles, because a durability claim without a protocol is one lab's habit. Ask for slip detection latency, since that single figure separates a gripper that drops things from a gripper that does not. Companies ask for the sensitivity chart and receive a chart measured on a surface that is not theirs. Then run the oldest test in the field on your own sample: two points, closing until the reading merges into one. It takes an afternoon and it is the only number in the conversation that belongs to you. The probability that a single tactile benchmark arrives and settles the comparison is not zero. The field is pricing it at zero, which is why every quote you receive is a comparison between two sensors that has never been made.