Appending data to a dataset

Once a dataset has batches attached, you often need to add more datapoints — new reviews arrive, a labelling round surfaces extra cases, or you want to grow a live task. The append endpoint adds datapoints to an existing V4 dataset synchronously, in a single request, and (when configured) propagates them straight into the tasks of any attached batch.

For the full dataset lifecycle, see Working with Datasets. To learn how appended datapoints reach existing tasks, see Syncing a batch.

When to append vs re-upload

Append and file upload solve different problems. Pick based on how much data you’re adding and whether the dataset is already in use.

Append is V4-only and JSONL-only. For the file-upload flow (which also supports CSV and V3 datasets), see Uploading data.

Appending datapoints

$
POST /api/v1/data-collection/datasets/{dataset_id}/datapoints

Send your datapoints as JSONL in the request body — one JSON object per line. Field names must match the keys defined in the dataset schema, exactly as for a JSONL file upload.

$
curl -X POST \
>
  -H "Authorization: Token <API_TOKEN>" \
>
  -H "Content-Type: application/x-ndjson" \
>
  --data-binary @new-reviews.jsonl \
>
  "https://api.prolific.com/api/v1/data-collection/datasets/<dataset-id>/datapoints"

Where new-reviews.jsonl contains:

{"review_text": "Fast delivery, works great", "product_name": "Widget Pro", "source": "amazon"}
{"review_text": "Stopped working after a week", "product_name": "Widget Pro", "source": "trustpilot"}
{"review_text": "Exactly as described", "product_name": "Basic Widget", "source": "amazon"}

Requirements

The append is rejected before any writes if:

Response

The append is synchronous. The response reports the outcome directly:

1
{
2
  "accepted": 3,
3
  "rejected": 0,
4
  "errors": []
5
}

Accepted datapoints are persisted immediately and sort after every existing datapoint in the dataset.

Handling partial outcomes

Valid records are written even if others in the same request fail. When any record is rejected, the successful ones are still accepted and the response lists what went wrong per record:

1
{
2
  "accepted": 2,
3
  "rejected": 1,
4
  "errors": [
5
    { "record_index": 2, "field": "rating", "reason": "Value is not a valid number" }
6
  ]
7
}

To recover, fix the rejected lines and re-send them. Appends are idempotent per row: a datapoint identical to one already stored is counted as a duplicate (included in accepted), not written twice — so it’s safe to re-send a corrected batch that overlaps rows you already appended. Key order within each JSON object doesn’t affect deduplication.

Where appended data appears

Each append is also recorded as an api_append import job so it shows up alongside file uploads in the dataset’s imports projection:

$
GET /api/v1/data-collection/datasets/{dataset_id}/imports/{import_id}

Unlike a file upload, this job is created already in its terminal state (complete or partial) — you don’t need to poll it, since the append response already told you the outcome. See Monitoring an import for the job shape.

Propagating appends to live tasks

Appending adds datapoints to the dataset. Batches built from that dataset don’t automatically gain tasks for them unless a sync runs.

• If an attached batch has auto-sync enabled, a successful append automatically triggers a sync for it — the new datapoints become tasks with no further action.
• Otherwise, trigger a sync manually once you’ve finished appending.

See Syncing a batch for both flows.

By using AI Task Builder, you agree to our AI Task Builder Terms.