Understanding the Pre-Annotation JSON Structure for CV
Updated at August 27th, 2026
Every task is described by a single JSON file. That file can include annotations known as pre-annotations, or it can represent a task with no annotations as well. This guide walks through that structure so you can read an existing file and build your own from scratch.
 |
|
Before diving into a full example, it helps to know the three pieces you'll be reading. Section 1 covers the top-level structure, the outer fields every file starts with. Section 2 covers the shapes array, the actual geometry, described by four shape types. Section 3 covers the objects array, the labels and attributes that give each shape its meaning. Together, sections 2 and 3 are the core of the file; everything else is metadata around them.
Top-level structure
The outer file is always an array containing one task record. That record carries task metadata plus the two annotation sections described above.
| Field | Description |
priority |
Internal queue priority for the task. Not related to the annotation content itself. |
primary_keys |
The unique reference(s) for the asset being annotated (e.g. an image or clip path). Client-facing, placeholder in examples. |
data |
Metadata about the asset itself: file name, dataset ID, and asset URL. Client-facing, placeholder in examples. |
output_annotation (also seen as output_Video or answers) |
The array holding the actual shapes. The key name varies slightly by project configuration, but the contents follow the same structure, see Section 2. |
output_objects (also seen as objects) |
The array describing what each shape represents, its class and attributes. See Section 3. |
The shapes array
Each entry in output_annotation represents one frame group and contains a shapes list. For a single image, there's one frame (frame_number: 0). For a video, the same shape repeats across multiple frame_number entries to track movement over time.
Every shape object shares the same fields, regardless of type:
| Field | Description |
index |
The shape's position in the list. Used for ordering only. |
type |
One of four values: point, line, rectangle, or polygon. See the tabs below. |
tags |
Attributes attached directly to this specific shape occurrence (usually empty, most attributes live on the object instead; see Section 3). |
object_id |
Links this shape to its label metadata in output_objects the connection described above. |
key_locations |
An array with one entry per frame. Each entry has frame_number, the points array (pixel coordinates), and visibility (1 = visible, 0 = hidden/occluded on that frame). |
Shape points inside shapes array
Here's what points looks like for each shape type, along with its matching object entry:
Point
Marks a single reference location. The points array holds exactly 1 coordinate pair. Typical use: a keypoint or landmark.
{
"index": 1,
"type": "point",
"tags": {},
"object_id": 3,
"key_locations": [
{
"frame_number": 0,
"points": [[320, 240]],
"visibility": 1
}
]
}Matching entry in output_objects:
{
"id": 3,
"sort_index": 1,
"tags": { "landmark_type": "reference_point" },
"is_class_read_only": false,
"class_name": "Landmark"
}Line
Marks an edge, boundary, or path. The points array holds 2 or more coordinate pairs, in order. Typical use: a lane marking.
{
"index": 2,
"type": "line",
"tags": {},
"object_id": 4,
"key_locations": [
{
"frame_number": 0,
"points": [[150, 400], [300, 380], [450, 390]],
"visibility": 1
}
]
}Matching entry in output_objects:
{
"id": 4,
"sort_index": 2,
"tags": {},
"is_class_read_only": false,
"class_name": "LaneMarking"
}Rectangle (bounding box)
A bounding box. The points array holds exactly 4 coordinate pairs, in this order: top-left, top-right, bottom-left, bottom-right.
{
"index": 3,
"type": "rectangle",
"tags": {},
"object_id": 5,
"key_locations": [
{
"frame_number": 0,
"points": [[500, 200], [650, 200], [500, 350], [650, 350]],
"visibility": 1
}
]
}Matching entry in output_objects:
{
"id": 5,
"sort_index": 3,
"tags": { "color": "red", "occluded": "no" },
"is_class_read_only": false,
"class_name": "Vehicle"
}Polygon
A precise, irregular-shaped outline. The points array holds 3 or more coordinate pairs tracing the outline. Typical use: a road surface or any organic shape a box can't capture cleanly.
{
"index": 4,
"type": "polygon",
"tags": {},
"object_id": 6,
"key_locations": [
{
"frame_number": 0,
"points": [[700, 300], [750, 280], [800, 300], [790, 350], [730, 360]],
"visibility": 1
}
]
}Matching entry in output_objects:
{
"id": 6,
"sort_index": 4,
"tags": { "surface_type": "road" },
"is_class_read_only": false,
"class_name": "RoadSurface"
}The objects array
Where shapes describe geometry, objects describe meaning. Every entry in output_objects corresponds to one shape (matched by id ↔ object_id) and carries the label information a human or model actually cares about.
| Field | Description |
id |
Matches a shape's object_id. This is the join key between the two arrays. |
class_name |
The label/class assigned to this shape (e.g. "Vehicle", "LaneMarking"). This is what shows up as the annotation's category. |
tags |
Key-value attributes describing this object (e.g. color: "red", occluded: "no"). This is where most descriptive detail lives. |
sort_index |
Display order in the annotation panel. Not related to geometry. |
is_class_read_only |
Whether the annotator is allowed to change the class after it's created. |
parent_id / class_name: "scene"
|
Most files include one root "scene" object and one media object (class_name: "image" or "video") that all other objects are implicitly grouped under. Copy this pattern as-is — it rarely needs editing. |
Putting it together: a complete example
Below is a minimal, fully valid file containing one of every shape type a point, a line, a rectangle, and a polygon each linked to its own labeled object. All IDs, URLs, and identifiers have been replaced with generic placeholders; use this as a template rather than copying real project data.
Full Example
[
{
"priority": 0,
"client_batch_id": "example_batch_001",
"primary_keys": ["example_project/000123"],
"data": {
"dataset_id": "example_dataset",
"file_name": "example_project/000123",
"url": "https://example-assets.com/api/v1/assets/example-asset-id"
},
"output_annotation": [
{
"group_type": null,
"frame_count": 1,
"shapes": [
{
"index": 1,
"type": "point",
"tags": {},
"object_id": 3,
"key_locations": [
{
"frame_number": 0,
"points": [[320, 240]],
"visibility": 1
}
]
},
{
"index": 2,
"type": "line",
"tags": {},
"object_id": 4,
"key_locations": [
{
"frame_number": 0,
"points": [[150, 400], [300, 380], [450, 390]],
"visibility": 1
}
]
},
{
"index": 3,
"type": "rectangle",
"tags": {},
"object_id": 5,
"key_locations": [
{
"frame_number": 0,
"points": [[500, 200], [650, 200], [500, 350], [650, 350]],
"visibility": 1
}
]
},
{
"index": 4,
"type": "polygon",
"tags": {},
"object_id": 6,
"key_locations": [
{
"frame_number": 0,
"points": [[700, 300], [750, 280], [800, 300], [790, 350], [730, 360]],
"visibility": 1
}
]
}
]
}
],
"output_objects": [
{ "id": 1, "tags": {}, "class_name": "scene" },
{ "id": 2, "parent_id": 1, "sort_index": 0, "tags": { "name": "Image" }, "class_name": "image" },
{ "id": 3, "sort_index": 1, "tags": { "landmark_type": "reference_point" }, "is_class_read_only": false, "class_name": "Landmark" },
{ "id": 4, "sort_index": 2, "tags": {}, "is_class_read_only": false, "class_name": "LaneMarking" },
{ "id": 5, "sort_index": 3, "tags": { "color": "red", "occluded": "no" }, "is_class_read_only": false, "class_name": "Vehicle" },
{ "id": 6, "sort_index": 4, "tags": { "surface_type": "road" }, "is_class_read_only": false, "class_name": "RoadSurface" }
]
}
]