Context

This doc explains how to use the Prolific API to create a study with AI Taskers, a group of qualified participants who excel at model evaluation, testing, and alignment. To learn more about how we qualify AI Taskers, check out this article.

We’ve added a new filter (ai-taskers) which allows you to select AI Taskers as participants for your study. We recommend using them for model evaluation and benchmarking, side-by-side performance comparisons, factuality and accuracy assessments, and generative tasks.

Changes to GET filters api

A researcher can get the list of filters by making a GET filters request as specified in our API doc

The response will now include a new filter:

{
   "filter_id": "ai-taskers",
   "title": "AI taskers",
   "description": "",
   "question": "AI tasker?",
   "researcher_help_text": "",
   "participant_help_text": "",
   "category": "AI taskers",
   "tags": [],
   "type": "select",
   "data_type": "ChoiceID",
   "choices": {
   "0": "Qualified AI taskers"
  }
}

Create a study for our AI Taskers

Using the response above, you can create a study with a new filter id “ai-task-experts” following our existing create study API doc

You’ll need to update the filters attribute in the request body of the create study API as such:

{
  "internal_name": "test",
  "name": "khoov test",
  "description": "test",
  "external_study_url": "<<https://www.test.com>",
  "reward": 15,
  "total_available_places": 100,
  "publish_at": null,
  "prolific_id_option": "not_required",
  "audience": "standard_sample",
  "device_compatibility": [
    "mobile",
    "desktop",
    "tablet"
  ],
  "peripheral_requirements": [],
  "url_prefix": "<https://test.prolific.com/submissions/complete?cc=>",
  "selected_ai_tasker_group": "0",
  "location": true,
  "estimated_completion_time": 1,
  "maximum_allowed_time": 13,
  "content_warnings": [],
  "study_labels": [],
  "pii": {
    "enabled": false
  },
  "is_custom_screening": false,
  "filter_set_id": null,
  "filter_set_version": null,
  "has_credentials": false,
  **"filters": [
    {
      "filter_id": "ai-taskers",
      "selected_values": [
        "0"
      ]
    }
  ],**
  "study_type": "SINGLE",
  "completion_codes": [
    {
      "code": "C1FF2ZY9",
      "code_type": "COMPLETED",
      "actions": [
        {
          "action": "MANUALLY_REVIEW"
        }
      ]
    }
  ],
  "submissions_config": {
    "max_submissions_per_participant": 1,
    "max_concurrent_submissions": -1
  }
}

Please note that the selected_values in the filters attribute is “0” which match General AI Taskers choices from the get filters response above.

The response will be 200 OK with an example payload of:

{
  "id": "60d9aadeb86739de712faee0",
  "name": "Study about API's",
  "internal_name": "WIT-2021 Study about API's version 2",
  "description": "This study aims to determine how to make a good public API",
  "external_study_url": "<https://eggs-experriment.com?participant={{%PROLIFIC_PID%}>}",
  "prolific_id_option": "url_parameters",
  "completion_codes": [
    {
      "code": "ABC123",
      "code_type": "COMPLETED",
      "actions": [
        {
          "action": "AUTOMATICALLY_APPROVE"
        }
      ]
    },
    {
      "code": "DEF234",
      "code_type": "FOLLOW_UP_STUDY",
      "actions": [
        {
          "action": "AUTOMATICALLY_APPROVE"
        },
        {
          "action": "ADD_TO_PARTICIPANT_GROUP",
          "participant_group": "619e049f7648a4e1f8f3645b"
        }
      ]
    }
  ],
  "total_available_places": 30,
  "estimated_completion_time": 5,
  "maximum_allowed_time": 25,
  "reward": 100,
  "device_compatibility": [
    "desktop"
  ],
  "peripheral_requirements": [],
  **"filters": [
    {
      "filter_id": "ai-taskers",
      "selected_values": [
        "0"
      ]
    }
  ],**
  "filter_set_id": null,
  "filter_set_version": null,
  "status": "UNPUBLISHED",
  "study_labels": [
    "interview"
  ],
  "content_warnings": [
    "sensitive"
  ],
  "content_warning_details": "Experiences with hateful activities, experiences with self-injury and harmful behaviour",
  "is_custom_screening": false
}