Skip to content

Need solution to get target # of ratings per hit #4

@rosscutler

Description

@rosscutler

The current implementation may not give the requested number of ratings per hit because hits may fall into the "accept but not use". In a recent run 34% of hits fell into this bucket. This adds inconsistency to the ratings (not a constant number) and is also inefficient (wasted $). There are at least two solutions:

  1. Add a real-time failure for the pair comparison environment check which was 72% of failures in a recent run.
  2. Use the MTurk API to create a controller that monitors and updates the hits to meet the target goals.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions