What makes a demonstration usable
A teleoperated demonstration looks like success on video. Whether a policy can learn from it depends on details that are invisible in the recording.
Match the action space
If the policy outputs end-effector deltas, record end-effector deltas, not only the operator's joint commands. Converting afterward introduces errors that look like noise to the model. Decide the action representation first and record it directly.
Log the latency
There is a delay between the operator moving a controller and the robot moving. If it varies, the observation the operator reacted to is not the observation the policy sees when it predicts the action. Measure the delay and store it with each episode.
Keep the failures
Failed grasps, recoveries and retries are some of the most useful data you will collect. Label them as failures and keep them. A dataset of only clean successes teaches a policy nothing about what to do when something goes wrong.
Vary the start, not the instruction
- Change object positions, orientations and distractors between episodes on purpose.
- Keep the operator protocol consistent, and record who performed each episode.
- Vary instruction wording deliberately if the policy will take language input, and store the exact text.
Define success before recording. If it is decided afterward, the labels will quietly reflect whatever was convenient.
None of this is complicated. It is a matter of deciding these things before the first episode and writing them down.