Working with Batches

An AI Task Builder Batch allows you to collect human annotations on your existing data. You provide a dataset, define instructions, and participants evaluate each datapoint according to your instructions.

This guide covers the workflow for creating, configuring, and publishing a Batch.

Workflow overview

Creating a dataset

Create a dataset to hold your data.

$
POST /api/v1/data-collection/datasets

1
{
2
  "name": "Product reviews Q4 2024",
3
  "workspace_id": "6278acb09062db3b35bcbeb0"
4
}

Response

1
{
2
  "id": "0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d",
3
  "name": "Product reviews Q4 2024",
4
  "status": "UNINITIALISED"
5
}

Uploading your data

Upload your dataset as a CSV file using presigned URLs.

Step 1: Request a presigned URL

$
GET /api/v1/data-collection/datasets/{dataset_id}/upload-url/{filename}

For example:

$
GET /api/v1/data-collection/datasets/0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d/upload-url/reviews.csv

Step 2: Upload to S3

Use the presigned URL from the response to upload your CSV file directly to S3.

$
curl -X PUT \
>
  -H "Content-Type: text/csv" \
>
  --data-binary @reviews.csv \
>
  "{presigned_url}"

CSV format

Your CSV should contain one row per datapoint. Each column is displayed to participants alongside the instructions.

1
id,review_text,product_name,rating
2
1,"Great product, exactly what I needed!",Widget Pro,5
3
2,"Arrived damaged, very disappointed",Widget Pro,1
4
3,"Works as expected, nothing special",Basic Widget,3

For advanced options including metadata columns and custom task grouping, see Working with Datasets.

Monitoring dataset status

Poll the dataset endpoint to check when processing is complete.

$
GET /api/v1/data-collection/datasets/{dataset_id}

Wait for the status to change to READY before proceeding.

Dataset status

Creating a batch

Once your dataset is ready, create a batch that attaches the dataset and defines your task layout in batch_items. Instructions are defined inline within batch_items — this is how every Batch specifies what participants see and do, whether the layout is a simple sequential list or a multi-column comparison.

$
POST /api/v1/data-collection/batches

The example below is the simplest layout: a single page with one row and one column, stacking the dataset field to annotate followed by two instructions. Participants see each item in order, top to bottom.

1
{
2
  "workspace_id": "0192a3b4-c5d6-7e8f-9a0b-1c2d3e4f5a6b",
3
  "name": "Product review sentiment analysis",
4
  "dataset_id": "0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d",
5
  "task_details": {
6
    "task_name": "Review Sentiment Classification",
7
    "task_introduction": "<p>Read each product review carefully and classify its sentiment.</p>",
8
    "task_steps": "<ol><li>Read the review text</li><li>Consider the overall tone</li><li>Select the appropriate sentiment</li></ol>"
9
  },
10
  "batch_items": [
11
    {
12
      "rows": [
13
        {
14
          "columns": [
15
            {
16
              "items": [
17
                { "type": "dataset_field", "field": "review_text" },
18
                {
19
                  "type": "multiple_choice",
20
                  "description": "What is the overall sentiment of this review?",
21
                  "answer_limit": 1,
22
                  "options": [
23
                    { "label": "Positive", "value": "positive" },
24
                    { "label": "Neutral", "value": "neutral" },
25
                    { "label": "Negative", "value": "negative" }
26
                  ]
27
                },
28
                {
29
                  "type": "free_text",
30
                  "description": "Briefly explain why you chose this sentiment rating",
31
                  "placeholder_text_input": "e.g. The reviewer uses positive language and expresses satisfaction..."
32
                }
33
              ]
34
            }
35
          ]
36
        }
37
      ]
38
    }
39
  ]
40
}

Task details

The optional task_details object provides context to participants:

All three fields support basic HTML formatting.

Instructions

Instructions are the input items inside batch_items — the questions participants answer for each datapoint. Add them as column items alongside your dataset fields and content, in the order you want them displayed.

Response

1
{
2
  "id": "0192a3b4-d6e7-7f8a-0b1c-2d3e4f5a6b7c",
3
  "workspace_id": "0192a3b4-c5d6-7e8f-9a0b-1c2d3e4f5a6b",
4
  "name": "Product review sentiment analysis",
5
  "dataset_id": "0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d",
6
  "status": "UNINITIALISED",
7
  "total_task_count": 0
8
}

Batch status

A batch transitions through the following states:

Advanced layouts

The example above uses a single column, but batch_items can arrange content into multiple rows and up to two columns per row — for example a side-by-side comparison where two dataset fields sit next to each other, each with its own rating, followed by a shared follow-up question.

See Configurable Layout for the full structure, all column item types (dataset fields, content blocks, and instructions), constraints, and a side-by-side example.

Setting up the batch

Once your batch is created, trigger task generation. Each datapoint in your dataset is paired with your task layout to create a task. Tasks are then organized into task groups — participants complete one task group per submission.

$
POST /api/v1/data-collection/batches/{batch_id}/setup

1
{
2
  "tasks_per_group": 5
3
}

The tasks_per_group parameter controls how many tasks are randomly assigned to each group. If omitted, each task group contains a single task.

Participants complete all tasks within their assigned group in a single submission. No participant will be assigned the same task group twice, even if they complete multiple submissions.

For custom task grouping based on your own criteria, see Working with Datasets.

This triggers task generation. The batch status will change to PROCESSING and then to READY once complete.

Monitoring batch status

Poll the batch endpoint to check when task generation is complete.

$
GET /api/v1/data-collection/batches/{batch_id}

Wait for the status to change to READY before creating a study.

1
{
2
  "id": "0192a3b4-d6e7-7f8a-0b1c-2d3e4f5a6b7c",
3
  "workspace_id": "0192a3b4-c5d6-7e8f-9a0b-1c2d3e4f5a6b",
4
  "name": "Product review sentiment analysis",
5
  "status": "READY",
6
  "total_task_count": 150
7
}

The total_task_count reflects the number of datapoints in your dataset.

Publishing a batch

To make your batch available to participants, create a Prolific study that references it.

$
POST /api/v1/studies/

When creating the study, set data_collection_method to AI_TASK_BUILDER_BATCH and provide your batch ID:

1
{
2
  "name": "Product Review Sentiment Analysis",
3
  "internal_name": "sentiment-analysis-q4-2024",
4
  "description": "<p>Help us understand the sentiment in product reviews by classifying each review and explaining your reasoning.</p>",
5
  "estimated_completion_time": 15,
6
  "maximum_allowed_time": 45,
7
  "reward": 300,
8
  "data_collection_method": "AI_TASK_BUILDER_BATCH",
9
  "data_collection_id": "0192a3b4-d6e7-7f8a-0b1c-2d3e4f5a6b7c",
10
  "data_collection_metadata": {
11
    "annotators_per_task": 3
12
  }
13
}

Then publish the study:

$
POST /api/v1/studies/{study_id}/transition/

1
{
2
  "action": "PUBLISH"
3
}

Retrieving responses

After participants have completed their tasks, download the annotated data as a CSV.

$
GET /api/v1/data-collection/batches/{batch_id}/report/

This returns your original CSV with additional columns containing participant responses for each instruction.

By using AI Task Builder, you agree to our AI Task Builder Terms.