Working with Datasets

A dataset contains the data that participants will annotate in a Task Builder Batch. This page covers dataset creation, upload, and advanced configuration options.

For the complete batch workflow, see Working with Batches.

Dataset versions

There are two dataset versions:

V4 datasets are created by passing a schema when calling POST /datasets. See Defining a dataset schema for details.

Creating a dataset

POST /api/v1/data-collection/datasets

V3 dataset (no schema):

{
  "name": "Product reviews Q4 2024",
  "workspace_id": "6278acb09062db3b35bcbeb0"
}

V4 dataset (with schema):

{
  "name": "Product reviews Q4 2024",
  "workspace_id": "6278acb09062db3b35bcbeb0",
  "schema": {
    "strict": false,
    "fields": {
      "review_text": { "type": "text", "label": "Review" },
      "product_name": { "type": "text", "label": "Product" },
      "source": { "type": "metadata" }
    }
  }
}

Uploading data

V4 datasets (JSONL or CSV)

V4 upload uses a three-step flow with import job tracking.

Step 1: Request a presigned URL

GET /api/v1/data-collection/datasets/{dataset_id}/upload-url/{filename}

For example:

GET /api/v1/data-collection/datasets/0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d/upload-url/reviews.jsonl

The response includes a presigned S3 URL, the content_type to use for the upload, and an import_id to track this upload:

{
  "upload_url": "https://s3.amazonaws.com/bucket/...",
  "http_method": "PUT",
  "content_type": "application/x-ndjson",
  "import_id": "01935c2d-1a2b-3c4d-5e6f-7a8b9c0d1e2f"
}

Step 2: Upload to S3

Use the content_type value from the response as the Content-Type header.

curl -X PUT \
  -H "Content-Type: {content_type}" \
  --data-binary @reviews.jsonl \
  "{upload_url}"

Step 3: Poll for import status

GET /api/v1/data-collection/datasets/{dataset_id}/imports/{import_id}

See Monitoring an import for status details.

V3 datasets (CSV or ZIP)

Step 1: Request a presigned URL

GET /api/v1/data-collection/datasets/{dataset_id}/upload-url/{filename}

Step 2: Upload to S3

curl -X PUT \
  -H "Content-Type: text/csv" \
  --data-binary @reviews.csv \
  "{upload_url}"

Step 3: Poll dataset status

GET /api/v1/data-collection/datasets/{dataset_id}/status

Wait until status reaches READY before creating a batch.

JSONL format

For V4 datasets, upload data as newline-delimited JSON. Each line is one datapoint. Field names must match the keys defined in the dataset schema.

{"review_text": "Great product!", "product_name": "Widget Pro", "source": "amazon"}
{"review_text": "Arrived damaged", "product_name": "Widget Pro", "source": "trustpilot"}
{"review_text": "Works as expected", "product_name": "Basic Widget", "source": "amazon"}

In strict mode ("strict": true), every record must include all schema fields. In non-strict mode, missing fields are allowed and treated as absent.

CSV format

Both V3 and V4 datasets accept CSV uploads. Each row is one datapoint.

id,review_text,product_name,rating
1,"Great product, exactly what I needed!",Widget Pro,5
2,"Arrived damaged, very disappointed",Widget Pro,1
3,"Works as expected, nothing special",Basic Widget,3

For V4 CSV uploads, column names must match schema field keys.

Metadata columns

Columns prefixed with META_ are not displayed to participants. Use these for internal data you need in your results but don’t want participants to see.

id,review_text,META_source,META_timestamp
1,"Great product!",amazon,2024-01-15T10:30:00Z
2,"Not worth it",trustpilot,2024-01-16T14:22:00Z

In V4 datasets, use fields of type metadata instead of the META_ prefix.

Custom task grouping

By default, tasks are grouped randomly when you set up a batch (using the tasks_per_group parameter). To define your own groupings, include a META_TASK_GROUP_ID column in your CSV (V3) or a field of type task_group_id in your schema (V4).

Rows with the same group ID value are grouped together into a single task group. Participants complete all tasks within a group in one submission.

id,review_text,product_name,META_TASK_GROUP_ID
1,"Great product!",Widget Pro,widget_pro_reviews
2,"Excellent quality",Widget Pro,widget_pro_reviews
3,"Not worth the price",Basic Widget,basic_widget_reviews
4,"Does the job",Basic Widget,basic_widget_reviews

Monitoring an import

For V4 datasets, each upload is tracked as an import job.

GET /api/v1/data-collection/datasets/{dataset_id}/imports/{import_id}

A partial response includes per-record errors:

{
  "import_id": "01935c2d-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
  "status": "partial",
  "accepted_count": 498,
  "rejected_count": 2,
  "errors": [
    { "record_index": 47, "field": "review_text", "reason": "Value exceeds maximum length" },
    { "record_index": 312, "field": null, "reason": "Record missing required field: product_name" }
  ]
}

Accepted records are available immediately even if some records were rejected.

Dataset status (V3 only)

Poll the status endpoint to check processing status for V3 datasets.

GET /api/v1/data-collection/datasets/{dataset_id}/status

Wait for the status to reach READY before creating a batch with this dataset.