The desktop app for building gesture datasets

What it does

Gesto Labeller is a desktop app that turns your webcam into a gesture-dataset recorder. You define the gestures you care about, capture examples of each, and it saves clean, training-ready landmark data to your disk. Think "Teachable Machine for gestures" — but offline, with no account and no upload.

The data it produces feeds directly into the gesto Python package, which trains and runs the actual recognition models.

The workflow

Projects

The Projects screen is your home base. Each project is a self-contained dataset with its own region, classes, and samples.

• New project — name it, add an optional description, and choose the region (one hand, two hands, pose, legs, or full body).
• Edit project — rename or change the description later.
• Copy path — grabs the project folder path, ready to paste into a gesto train command.
• Sort & filter — order projects by name or last updated, and filter by region.
• Each card shows the region, class count, annotation count, and when it was last updated.
• Delete project when you're done with it.

Annotate — capturing samples

The Annotate tab is where recording happens. Select a class, start the camera, perform the gesture, and press Capture (or the space bar).

What you control

• Static or Sequence — whether each sample is one frame or a short motion clip.
• Camera picker — choose which webcam to use, with a refresh button.
• Open video — capture from a video file instead of a live camera.
• Normalise (on by default) — makes samples position- and scale-independent, so a gesture is recognised wherever it appears in frame.
• Max frames — the clip length for sequence mode, adjustable with − / + or by typing the number.
• Live readouts: current gesture, samples for this label, and the feature dimension.

Static & sequence

The most important choice when capturing — it must match the nature of your gesture.

Regions

A region decides which body landmarks are tracked. You pick it when creating the project.

Explore & edit

The Explore tab shows every sample you've captured as a grid of skeleton previews, filtered by class. It's how you check the quality of your data and clean it up.

Per-sample tools

• View — each captured sample is rendered as its landmark skeleton, including a clear side-by-side view for two-hand samples.
• Edit — opens the sample editor to inspect it frame by frame.
• Delete — permanently removes a bad or accidental sample from disk.

In the sample editor

• Relabel — move a sample into a different class with one dropdown; the file moves with it.
• Remove frames — for sequence samples, tick the bad frames and save; the sample is rewritten with only the frames you kept.
• Delete the whole sample — including a prompt if you remove every frame.

Counts update live across the app as you add, relabel, or remove samples.

Classes

The Classes tab manages the gestures in your project — one class per gesture.

• Add a class for each gesture you want to recognise.
• Rename a class — samples move with it automatically.
• Delete a class and its samples.
• Duplicate names are prevented, so your labels stay clean.

Export

The Export tab packages your whole project — classes, region, and every annotation's landmark data — into a single re-importable .json file. One file to back up, move, or share a dataset.

For training, you don't even need to export: gesto reads the project folder directly.

License

Gesto Labeller is released as a free Community Edition. It's built with PySide6 (Qt for Python) under the LGPL, and ships with a third-party notices file listing its components. The companion gesto Python package is MIT-licensed.

Then what? Train a model

Once you've captured a dataset, the gesto Python package turns it into a working model:

pip install gesto
gesto train static hands_one "C:\...\gesto_projects\my-signs"
gesto detect static hands_one

See the full documentation for training options, live detection, and running models in your own code.