a companion discussion area for blog.codinghorror.com

Is Amazon's Mechanical Turk a Failure?


#41

“Let’s not forget that the person inside the mechanical Turk was a slave.”

Actually, that’s not true. Tou can read the Wikipedia article or any of several books about the chess-playing Turk to learn more about it.

Keep in mind that the hidden person had to be not just a capable chess player, but a phenomenal one; it often beat the best players around. This wasn’t cheating–this was real chess playing. That’s hardly a skill that a “slave” would develop. The people who were the secret players were talented co-conspirators with the Turk’s inventor, never “slaves.”


#42

Don’t tie in money with the work done? Sounds like socialism to me! Lol Love your blog, even though I’m not a software developer!


#43

I actually blogged about mechanical turk when it first came out, figuring that it had the potential to change the nature of how we work, but had my doubts about that actual process … and also wondered about the notion that one could find people who were low paid enough that the kind of work involved would be worthwhile and who nonetheless had regular internet access. (I also raised the issue with the good folks at Amazon about the joys of taxes …)

As to intrinsic vs. extrinsic rewards - I remember when I was about ten and my dad made an offer to me of one penny for each dandelion weed I could get out of the back yard. After about twenty five dandelions I found that the motivation for the reward just wasn’t there. On the other hand, I suspect that if he had just gone out with me and we had both pulled dandelions without any discussion of reward, we’d probably have taken care of the entire lawn.

I’ve actually found that I prefer for those projects that I have the biggest personal emotional stake in that I don’t have a dollar value tied up in with it, though certainly if along the way I make some money I won’t argue. Once you start tying a job to a paycheck it ceases being an endeavor to master and simply becomes x amount of dollars per hour, and you then do anything you can to avoid having to actually do any more work than you absolutely have to for that amount.

Good post.


#44

I guess I came to mTurk after all the hullabaloo about “changing the way people work,” because I just saw it as a chance to earn enough to buy a couple DVDs.


#45

Mturk does have the potential to change how we work, unfortunately it is based on a bizarre premise that is often touted by people in favour of globalisation, that there is a whole lot of quite literate and skilled people in developing countries who are willing to work for peanuts. This is not the case. If you’re educated enough to write properly, you probably are being paid more for your work than what mturk offers, no matter where you live.

I have been turking for 3 weeks now and though I’ve done many interesting things and some of the repetitive HITs feel like certain video games, I just want to get paid and that is the part that apparently takes the longest.

The majority of my hits are still pending and I think that over time, I will probably stop turking altogether because the novelty is wearing off. Could mturk help by being scrupulously honest about payment and offer intrinsic rewards to keep us going? Of course. But companies have spent the last twenty years deliberately avoiding meeting our emotional needs because they think it’s cheaper and easier to operate that way. It’s demanding and expensive to keep people happy enough to do a good job. Mturk is a short-cut around that, just like outsourcing is.


#46

I’ve been watching Mechanical Truk for awhile and also thought it was a failure. I’ve changed my mind. I think they were just using us for beta testing. Powerset is the real goal.

http://whatis.blogs.techtarget.com/2007/05/11/jeff-bezos-mechanical-turk-and-powerset/


#47

If you want to earn with Google. Join Google Adsense Programme today. Google charges no fee to join this service. You can start earning in dollars instantly. For more information visit: http://magotras.googlepages.com/earnwithgoogle

If you face any problem in creating Google Adsense account or ur webpage, just click on post a comment on my blog and I will be haappy to guide you through the process.


#48

I’ve been reading a lot about Mechanical Turk because I have a transcribing business. I pay a good rate to people in the Philippines. This is what one of them had to say…

“My reason from not continuing to work at Mechanical Turk is because of the tasks available. When there is a task, it is not assigned to you, it is available. When you start working on that task, somebody is also doing that task so it’s like a race to the finish. Sometimes you spend an hour working, then when you finish it somebody had just finished it and that work is already done. Besides the pay is too low. You will need to work on a lot of tasks just to earn $10 and that is for already a week or so work.”

Seems the Amazon engineers are quite proud of their Turk. But they’ve created a monster. Outsourcing is one thing. Pitting low-paid workers in foreign countries against one another is just plain ruthless.


#49

I second the opinion of tim.

What is cheaper than all labor force in the world?
Pay 10,000 “turkers” a cent for a ridiculous “micro-task” that insults the human brain! Just make a “game” out of it, promise rewards or bonuses (which doesn’t necessarily need to exist) for good work and rake in the added value. Just wrap “at most one idiot gets paid a bit” or “mass-exploitation” into euphemistic jabber about “competition” and “community” and noone will hear the shot.

But I don’t care if bored housewives spend their whole day comparing images of products for a cent. What makes crowdsourcing really evil is the devaluation of work beyond the already dramatic negative effects of outsourcing, the devaluation of knowledge, expertise and education. They break into the translation and transcription business, marketing research and consulting business… everything that can be done via the internet and everything that has to do with brainwork is threatened to be devaluated to a few cent. What a luck that brain-surgery can’t be done @home. (“Is this brain convolution needed or not? Click ‘no’ to remove it”)

But this is only one half of the mess, the other half is that enough people are sufficiently brain-damaged to offer their manpower, knowledge and skills to be exploited, not for charity, not for other good reasons but for a few bucks of which Amazon gets even its share back because most turkers order stuff off Amazon.com for their money. A transfer to a bank account costs $4 and many have to work a whole week to earn that much.

Is MT a failure? If you see it from a idealistic viewpoint characterized by the above mentioned jabber, yes. If you see it from the viewpoint of Amazon.com, surely not.
It’s capitalist’s dream of a global galley on which everybody rows voluntarily and shares bread and water with the owner of the galley. Tim, you’re invited to join them when they ruined your business explains why they are so wonderfully brainless: If you ever plan to visit Seattle, Toronto or Kirkland (wherever that might be), don’t check the “I love Seattle/I Love Toronto” etc. blogs for restaurant or hotel recommendations. At least most of the euphoric comments on the menu or the rooms on these blogs are not real and have been written by people who doesn’t even know where Seattle is or who write things like “Canada is a Wonderfull city”. One of the poor guys who was lurked to lie on these blogs by “Data Services” could not completely overcome his honesty and called himself “Michael Turk”…


#50

Oh, I forgot to rant about the abusive possibilities of MT: Populating forums and blogs artificially is not bad per se, I know that it’s hard to get a community going, but to do it with turkers is even cheaper than the existing services. And it explains why many posts are so wonderfully brainless. The funniest thing I’ve found so far (and I’m watching MT merely for a few weeks) was a completely boring dating agency website (“lowered expectations”) trying to push their popularity on StumbleUpon.com. Need a vote? Only 1 cent @ MTurk.com

This comment was not paid by Amazon - Mechanical Turk…or was it? :smiley:


#52

fuck google answers and uclue, all these assholes are are just another fucking clique. why should anybody reward 0.000000000000000001% of the internet when there are so many willing and able people able to give much better indepth answers then these sets of assholes?

mechanical turk? it’s not a social community, just a virtual sweatshop full of lonely assholes on the internet, and the motherfuckers and greedy assholes willing to exploit them. who the fuck wants to write a 500word essay for 20cents? i wouldn’t, but i’d be willing to hunt down these people and blow their heads off for 20cents.


#53

The mechanical turk offers some very interesting avenues to consumer opinions, provided you are able to ask the right questions.


#54

I agree Larry, perhaps the best thing to do with MT is to use it as a very cheap focus group.


#55

it is a take it or leave it mturk. what i can say is everyone is clever enough to make decision whether they want to continue work in mturk. whether they want to “continue” be abused by mturk requesters it is entirely their choice. however if it does prove earn more than their current job, why not?


#56

i just watched the video about the on Human Computation…
it’s sad that people who take a lot of things for granted can mock
income such as $2-3 per hour from a third world country… Instead of just letting the less fortunate get a chance to earn such small amounts to help for a living…someone chose to make a game so the work can be done by the people for free… we are all interconnected in this web world…as one…yet we all live in different countries…so different, specially the economy…to 1st world country standards…the income form a third world employee is a laughing matter… but for us, it puts food on the table…milk for our young…i have been searching the net for online work for sometime because I need it to survive since my regular job is simply not enough… whether be it cents or a few dollars for several hours work…if i have time…i’d do it… in the process encountering countless spams…unpaid work…etc…i can’t help but say that the video i watched hurt so much…specially when people laughed about a 2 dollar per hour added income…something which to me would be mean a lot to add up for our daily expenses. that’s the way its is i guess… just have to accept it.a lot of us work hard…its not our fault we were brought to this world in a third world country…but we have to cope up… with just dreams of the benefits of belonging to rich countries.


#57

I got on the MTurk bandwagon way early, way back when it was new and the automated programs hadn’t quite taken over yet. Back when it was all about Click the image that best illustrates this address. I spent a few hours a day for a few weeks on that.

It bought Christmas presents for my whole family in 2005, just clicking things on my off time while watching TV. At the time, I was extolling it as the Next Big Thing… but then the easy HITs dried up completely. For a while, I was able to do $.03 HITs fast enough that I was making over $10 an hour, sometimes more. But then it dried up hard. I haven’t touched it in a while, but my guess is the HITs can no longer be completed fast enough to match an actual wage, and at that point, they lose me.

Also, I’m guessing the newer ones aren’t nearly as fun as the old days, loading up an Opera script and clicking the address.


#58

At least you’re honest enough to leave up your wrong predictions. There’ve been lots of very nice applications using the Mechanical Turk.

Check out these on the why and how-much it pays:

http://behind-the-enemy-lines.blogspot.com/2008/09/how-much-turking-pays.html

http://behind-the-enemy-lines.blogspot.com/2008/09/why-people-participate-on-mechanical.html

and some links to articles analyzing how effective the Turk is:

http://behind-the-enemy-lines.blogspot.com/2008/09/links-to-mechanical-turk-articles.html


#59

I think, rather than a failure, the mtrurk is ahead of it’s time.

Imagine what mturk morphs into as the interface between human and computer becomes increasingly seamless and these small tasks can be done faster and faster and faster. The actual thought process that the system is trying to harness is a small portion of the time spent on these tasks today. Most of it seems to be wasted on IO.


#61

Mturk also does art: www.thesheepmarket.com. The same guy also combined 2000 voices singing the Daisy song.


#63

Eerie - I’ve been scratching my head over the same topic (why does the bonus they’re dangling in my face just not do it for me?) and came to some articles by Alfie Kohn on the topic - http://www.alfiekohn.org/business.htm
What I took away is that people want to work with a good team on a good product and be compensated properly for it. This is probably why things like goofy t-shirts with in-jokes on them are better rewards (excuse me, external inducements) than money: they’re tangible proof that someone out there understands and appreciates what you’re doing, and that makes perfect sense to me.
When it comes to birthdays, do you give loved ones a wad of twenty dollar bills or do you give them a present that shows that you know what they want? Which would you rather get?