OCS2007 Session 2: Crowdsourcing

Topic introduction by Peter Cohen from Amazon.com

Overview of Amazon's Mechanical Turk, a reference to their "artificial artificial intelligence." This is a software interface that simply allows humans to interact with each other. Amazon has established a market for workers who largely do pattern-matching for online information.

Data-oriented problems that uniquely benefit from human interactions of high volume. The software allows for many people to work on 'human intelligence tasks.' "What problems could I solve if I could get tens of thousands of people to contribute to the task?"

200,000 people have signed up and completed tasks under Mechanical Turk. Last week, they had participants from 146 countries. These are tasks that are inherently difficult or impossible for a machine to do. Use cases include:

  • Data augmentation for Amazon catalog, to get richer information for their products or de-duplicate
  • Improve search results, using humans to establish relevance. Search doesn't give answers, it gives results. The humans on this task are helping to provide answers. This reminds me of a "best bets" search capability in enterprise portals – where a human points to the right document when an employee is searching for things like HR policy items.
  • There is a company who provides transcription services using Mechanical Turk.
  • A firm uses MT to look at and identify 'tone' in online articles or forum postings.
  • Verify quote attribution.
  • Write product reviews.

The most active users are working to supplement income, these are not just technical experts. One high-visibility task involves the searching for aviator Steve Fosset in September, where humans searched online imagery for signs of his missing aircraft.

Great question about how to avoid underage users on MT, you must be 18 years old to "work" here. Every online commerce with any aspect of anonymity will invite mischief, and there was a concern that someone would exploit children to work these often rote tasks.

One goal is to create a global democratic marketplace for labor.

Q: What of communities? Communities have emerged among MT users/workers, including one called "Turker Nation." They built a lot of anonymity into the MT system, perhaps they will add a "real identity" option for people who are willing to reveal who they are.