Working with Datasets

A dataset contains the data that participants will annotate in an AI Task Builder Batch. This page covers dataset creation, upload, and advanced configuration options.

For the complete batch workflow, see Working with Batches.

Dataset versions

There are two dataset versions:

V4 datasets are created by passing a schema when calling POST /datasets. See Defining a dataset schema for details.

Creating a dataset

$
POST /api/v1/data-collection/datasets

V3 dataset (no schema):

1
{
2
  "name": "Product reviews Q4 2024",
3
  "workspace_id": "6278acb09062db3b35bcbeb0"
4
}

V4 dataset (with schema):

1
{
2
  "name": "Product reviews Q4 2024",
3
  "workspace_id": "6278acb09062db3b35bcbeb0",
4
  "schema": {
5
    "strict": false,
6
    "fields": {
7
      "review_text": { "type": "text", "label": "Review" },
8
      "product_name": { "type": "text", "label": "Product" },
9
      "source": { "type": "metadata" }
10
    }
11
  }
12
}

Uploading data

V4 datasets (JSONL or CSV)

V4 upload uses a three-step flow with import job tracking.

Step 1: Request a presigned URL

$
GET /api/v1/data-collection/datasets/{dataset_id}/upload-url/{filename}

For example:

$
GET /api/v1/data-collection/datasets/0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d/upload-url/reviews.jsonl

The response includes a presigned S3 URL, the content_type to use for the upload, and an import_id to track this upload:

1
{
2
  "upload_url": "https://s3.amazonaws.com/bucket/...",
3
  "http_method": "PUT",
4
  "content_type": "application/x-ndjson",
5
  "import_id": "01935c2d-1a2b-3c4d-5e6f-7a8b9c0d1e2f"
6
}

Step 2: Upload to S3

Use the content_type value from the response as the Content-Type header.

$
curl -X PUT \
>
  -H "Content-Type: {content_type}" \
>
  --data-binary @reviews.jsonl \
>
  "{upload_url}"

Step 3: Poll for import status

$
GET /api/v1/data-collection/datasets/{dataset_id}/imports/{import_id}

See Monitoring an import for status details.

V3 datasets (CSV or ZIP)

Step 1: Request a presigned URL

$
GET /api/v1/data-collection/datasets/{dataset_id}/upload-url/{filename}

Step 2: Upload to S3

$
curl -X PUT \
>
  -H "Content-Type: text/csv" \
>
  --data-binary @reviews.csv \
>
  "{upload_url}"

Step 3: Poll dataset status

$
GET /api/v1/data-collection/datasets/{dataset_id}/status

Wait until status reaches READY before creating a batch.

JSONL format

For V4 datasets, upload data as newline-delimited JSON. Each line is one datapoint. Field names must match the keys defined in the dataset schema.

{"review_text": "Great product!", "product_name": "Widget Pro", "source": "amazon"}
{"review_text": "Arrived damaged", "product_name": "Widget Pro", "source": "trustpilot"}
{"review_text": "Works as expected", "product_name": "Basic Widget", "source": "amazon"}

In strict mode ("strict": true), every record must include all schema fields. In non-strict mode, missing fields are allowed and treated as absent.

CSV format

Both V3 and V4 datasets accept CSV uploads. Each row is one datapoint.

1
id,review_text,product_name,rating
2
1,"Great product, exactly what I needed!",Widget Pro,5
3
2,"Arrived damaged, very disappointed",Widget Pro,1
4
3,"Works as expected, nothing special",Basic Widget,3

For V4 CSV uploads, column names must match schema field keys.

Metadata columns

Columns prefixed with META_ are not displayed to participants. Use these for internal data you need in your results but don’t want participants to see.

1
id,review_text,META_source,META_timestamp
2
1,"Great product!",amazon,2024-01-15T10:30:00Z
3
2,"Not worth it",trustpilot,2024-01-16T14:22:00Z

In V4 datasets, use fields of type metadata instead of the META_ prefix.

Custom task grouping

By default, tasks are grouped randomly when you set up a batch (using the tasks_per_group parameter). To define your own groupings, include a META_TASK_GROUP_ID column in your CSV (V3) or a field of type task_group_id in your schema (V4).

Rows with the same group ID value are grouped together into a single task group. Participants complete all tasks within a group in one submission.

1
id,review_text,product_name,META_TASK_GROUP_ID
2
1,"Great product!",Widget Pro,widget_pro_reviews
3
2,"Excellent quality",Widget Pro,widget_pro_reviews
4
3,"Not worth the price",Basic Widget,basic_widget_reviews
5
4,"Does the job",Basic Widget,basic_widget_reviews

Monitoring an import

For V4 datasets, each upload is tracked as an import job.

$
GET /api/v1/data-collection/datasets/{dataset_id}/imports/{import_id}

A partial response includes per-record errors:

1
{
2
  "import_id": "01935c2d-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
3
  "status": "partial",
4
  "accepted_count": 498,
5
  "rejected_count": 2,
6
  "errors": [
7
    { "record_index": 47, "field": "review_text", "reason": "Value exceeds maximum length" },
8
    { "record_index": 312, "field": null, "reason": "Record missing required field: product_name" }
9
  ]
10
}

Accepted records are available immediately even if some records were rejected.

Dataset status (V3 only)

Poll the status endpoint to check processing status for V3 datasets.

$
GET /api/v1/data-collection/datasets/{dataset_id}/status

Wait for the status to reach READY before creating a batch with this dataset.