sBot-Datasets: dataset guide

Load one episode

Start with the croissant dataset: 98 demonstrations of moving a croissant onto a plate. Download its metadata and robot data first, about 34 MB. Adding all the videos brings the download to about 2.5 GB. No Hugging Face token is needed.

1. Download the data

Run these commands in a terminal, inside your Python environment:

python -m pip install huggingface_hub pyarrow

hf download L5vel/base4-plate-croissant-eef-merged-v30 \
    --repo-type dataset \
    --include 'meta/**' --include 'data/**' \
    --local-dir ./plate-croissant

2. Read episode 47

Save this as inspect_episode.py beside the downloaded plate-croissant/ directory, then run python inspect_episode.py. It reads one episode’s robot data without decoding video.

import json
from pathlib import Path

import pyarrow.dataset as ds

root = Path("plate-croissant")
info = json.loads((root / "meta/info.json").read_text())
episode_id = 47
frames = ds.dataset(root / "data", format="parquet").to_table(
    columns=["frame_index", "timestamp", "observation.state", "language_persistent"],
    filter=ds.field("episode_index") == episode_id,
).sort_by([("frame_index", "ascending")])

print("Episodes:", info["total_episodes"])
print("Declared rate:", info["fps"], "Hz")
print("State dimensions:", len(frames["observation.state"][0].as_py()))

For this release, the output is:

Episodes: 98
Declared rate: 50 Hz
State dimensions: 10

Continue with the annotation example below to print this episode’s subtask timeline.

python -m pip install 'lerobot[dataset]'

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset(
    "L5vel/base4-plate-croissant-eef-merged-v30",
    episodes=[47],
)
sample = dataset[0]  # First frame of the selected episode.
print(sample["observation.state"].shape)
print(sample["observation.images.wrist"].shape)

Choose a dataset

All ten datasets use LeRobot v3.0 and Apache-2.0. Each has three camera views and robot state and action data. Pick a shorter task to inspect a grasp, or a longer sequence to work with several steps of manipulation and movement. The links below open the full repositories on Hugging Face.

Typical length is the median episode duration. Sizes include videos and are rounded. Across the collection: 1,499 episodes, 17.2 hours and 54.9 GB. The detailed table also includes frame counts, duration ranges and annotation sources.

Find the files and fields

These are LeRobot v3.0 datasets. Episode data is packed into shards; one Parquet or video file can contain more than one episode.

data/chunk-*/file-*.parquet          Robot states, actions and language
videos/<camera>/chunk-*/file-*.mp4   Camera recordings
meta/info.json                     Format, features and dataset counts
meta/tasks.parquet                 Task text, looked up by task_index
meta/episodes/                     Episode metadata and success flags
meta/eef_kinematics.json            End-effector representation metadata

The common 50 Hz rate describes the stored streams. Physical synchronization error between cameras and other sensors has not been measured.

Choose a camera view

These frames show episode 47 of the croissant dataset at the same recorded timestamp, 15.8 seconds. Use the scene views for the arm’s approach and the wrist view for a closer look at the gripper and target.

Read subtask annotations

Read the annotation list once per episode. Every frame stores the same complete list in language_persistent. Select messages with style="subtask", then sort them by timestamp. Append this code to inspect_episode.py:

import ast

messages = frames["language_persistent"][0].as_py() or []
messages = [ast.literal_eval(m) if isinstance(m, str) else m for m in messages]
subtasks = sorted(
    (m["timestamp"], m["content"])
    for m in messages if m.get("style") == "subtask"
)

for start, label in subtasks:
    print(f"{start:6.2f} s  {label}")
if not subtasks:
    print("This episode has no subtask annotations.")

You will get one line per subtask: its start time in seconds and its label. Depending on the reader, a message can arrive as a decoded object or a Python-literal string; the example handles both. Read another episode by changing episode_id in the first example.

A subtask lasts until the next subtask’s start; the last one lasts until the episode ends. For example, a step starting at 3.94 seconds followed by a step at 9.00 seconds covers [3.94, 9.00). Do not merge repeated labels: two wiping passes are two separate spans.

{
  "role": "assistant",
  "content": "move the green towel towards the mess",
  "style": "subtask",
  "timestamp": 3.940000057220459,
  "camera": null,
  "tool_calls": null
}

Check annotation sources

The collection combines existing annotations with model-generated annotations. Importing records where labels entered the pipeline; it does not identify who originally wrote or reviewed them.

Independent annotation agreement and downstream training gains have not been measured for this release. The timing study evaluates fixed-label predictions against source references; it does not measure the quality of the generated labels published here.

Prepare your experiment

• Split by episode. The published datasets define one train split containing every episode. Create your own training and evaluation split before sampling frames.
• Choose whether to include unsuccessful episodes. Read the success flag in meta/episodes/; five of the 1,499 episodes are marked unsuccessful.
• Check labels against video. Wording and timing conventions vary. In base4-mobile-cup, episode 82’s task says “blue cup” while a subtask says “black cup”.
• Handle missing subtasks. Two episodes have empty tracks. The parsing example above handles this case.
• Read task text by index. base4-mobile-place-cup contains two canonical task strings. The other nine datasets contain one each.
• Use repository metadata for the schema. The snapshot’s dataset cards contain boilerplate and some visualization badges point to an older namespace. Use the repository links on this page and inspect meta/.

These recordings come from a working laboratory, with the teleoperator and colleagues sometimes visible. When inspecting or sharing clips, keep that recording context in mind.

Explore collection statistics

Open the sections below for the complete counts and charts. They describe the 9 September 2026 snapshot; recompute them if the repositories change.

← Back to the blog