Defining a Dataset Schema

A dataset schema lets you describe the structure of your data before uploading it. Schemas are a feature of V4 datasets and enable:

• Named, typed fields that drive what participants see
• dataset_field items in batch_items that pull dataset values directly into your task layout
• Per-record validation during import (with strict mode)
• Structured metadata and custom task grouping

Creating a dataset with a schema

Pass a schema object when creating a dataset.

POST /api/v1/data-collection/datasets

{
  "name": "AI response evaluation",
  "workspace_id": "6278acb09062db3b35bcbeb0",
  "schema": {
    "strict": true,
    "fields": {
      "prompt": { "type": "text", "label": "Prompt" },
      "source_audio": { "type": "audio_url", "label": "Audio clip" },
      "source_video": { "type": "video_url", "label": "Video clip" },
      "response_a": { "type": "text", "label": "Response A" },
      "response_b": { "type": "text", "label": "Response B" },
      "category": { "type": "metadata" }
    }
  }
}

Field types

Audio URL fields

Use audio_url for externally hosted source audio that participants should listen to during a Batch task.

• Prolific does not host source media. audio_url fields must point to audio files hosted outside Prolific.
• URLs must remain valid for the full study duration. The audio URL should stay reachable and accessible from dataset import through participant completion.
• Use direct-file HTTPS URLs only. Manifest-based streaming formats such as .m3u8 and .mpd are not supported.
• Supported audio formats: .mp3, .wav, .aac, and .m4a.
• Access-Control-Allow-Origin (CORS): without it the audio can still play, but the player shows a generic placeholder waveform instead of the decoded waveform preview because waveform generation uses a cross-origin fetch.
• Cache-Control and/or ETag: without cache headers the same file may be downloaded twice — once to decode the waveform and once again for playback.

Video URL fields

Use video_url for externally hosted source video that participants should watch during a Batch task.

• Prolific does not host source media. video_url fields must point to video files hosted outside Prolific.
• URLs must remain valid for the full study duration. The video URL should stay reachable and accessible from dataset import through participant completion.
• Use direct-file HTTPS URLs only. Manifest-based streaming formats such as .m3u8 and .mpd are not supported and are rejected during dataset validation.
• Supported video formats: .mp4, .mov, .webm, and .m4v.
• Access-Control-Allow-Origin (CORS): without it the video can still play, but the player shows a generic placeholder poster frame instead of the real one, because the poster frame is decoded via a cross-origin fetch.
• Cache-Control and/or ETag: without cache headers the same file may be downloaded twice — once to decode the poster frame and once again for playback.

Frame controls for video fields

To let participants refer to a specific frame in a video, give each video its frame rate in your dataset. A video with a frame rate gets a frame counter and frame-step buttons in the participant’s video player.

The frame rate belongs to each video file, so it goes in your data rather than in the task layout, and different rows can hold videos at different rates.

1. Add a column for the rate and declare it in the schema as a metadata field. You can name it anything.
2. On the video_url field, bind the frame_rate attribute to that column with attributes.

{
  "fields": {
    "video_a":     { "type": "video_url", "attributes": { "frame_rate": "video_a_fps" } },
    "video_a_fps": { "type": "metadata" },
    "prompt":      { "type": "text" }
  }
}

Each row then supplies its own rate, for example { "prompt": "…", "video_a": "https://…/clip.mp4", "video_a_fps": 25 }.

• frame_rate is the attribute name Prolific defines. The column name is yours.
• The rate is a number from 1 to 480 (frames per second). Fractional rates such as 29.97 are allowed, and a numeric string is accepted.
• A row with an invalid rate is rejected at import, and the error names the rate column. A missing or blank rate is not an error: that video simply has no frame controls. In a strict schema the column must still be present.
• The rate column is metadata, so participants do not see it. Several video fields can share one column.
• attributes is only allowed for supported attributes on the field types that support them, and each column must be a declared metadata field in the same schema. Anything else is rejected with 400.
• Frames are numbered from 1.
• To add, change or remove a binding on a dataset that already has data, update the schema: existing rows are reprocessed from the original upload, so the column must have been in that upload. Uploading the same file again does not apply the change.

Strict mode

The strict flag controls how missing fields are handled during import.

Schema constraints

• Maximum 200 fields per schema.
• Field keys: 1–128 characters.
• Field labels: maximum 255 characters.
• At most one field of type task_group_id per schema.

Referencing schema fields in the batch layout

Once a dataset with a schema is attached to a batch, you can use dataset_field items in batch_items to display dataset values to participants.

{
  "batch_items": [
    {
      "rows": [
        {
          "columns": [
            {
              "items": [
                { "type": "dataset_field", "field": "prompt" },
                { "type": "dataset_field", "field": "response_a" },
                { "type": "dataset_field", "field": "response_b" },
                {
                  "type": "multiple_choice",
                  "description": "Which response is more helpful?",
                  "answer_limit": 1,
                  "options": [
                    { "label": "Response A", "value": "a" },
                    { "label": "Response B", "value": "b" },
                    { "label": "Neither", "value": "neither" }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Retrieving a dataset with its schema

GET /api/v1/data-collection/datasets/{dataset_id}

The response includes the current schema and all import jobs:

{
  "id": "0192a3b5-e8f9-7a0b-1c2d-3e4f5a6b7c8d",
  "name": "AI response evaluation",
  "schema_version": 4,
  "total_datapoint_count": 1000,
  "schema": {
    "strict": true,
    "fields": {
      "prompt": { "type": "text", "label": "Prompt" },
      "source_audio": { "type": "audio_url", "label": "Audio clip" },
      "source_video": { "type": "video_url", "label": "Video clip" },
      "response_a": { "type": "text", "label": "Response A" },
      "response_b": { "type": "text", "label": "Response B" },
      "category": { "type": "metadata" }
    }
  },
  "imports": [
    {
      "import_id": "01935c2d-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
      "status": "complete",
      "accepted_count": 1000
    }
  ]
}