Institution Profiles have been generated for hundreds of colleges & universities across the United States, Canada, the United Kingdom, and Australia.
They're built up from a variety of statistics TurkerView has collected from worker reviews and are meant to give a high level overview of how an institution is represented.
Confident workers do better work & make more money
New workers are often unsure of which tasks to accept, and have an extra difficult time understanding if work is safe before it has been reviewed by other workers.
By checking our Institution Profiles, workers can feel confident when they see consent forms from established institutions even if the individual account has yet to be reviewed. Outside of searching for work, returning tasks because a worker didn't have confidence in the requester is one of the largest losses of wages in online crowd work.
We want workers to feel comfortable & confident on MTurk, and academic research is by far one of the best entry experiences most workers could have. Academics pay higher wages, have much higher ethical standards, and are more receptive to feedback than traditional businesses. Highlighting their contributions to crowd work at a higher level, we hope, will allow workers to feel more confident working with all researchers from well respected & established institutions.
On top of the Institutional Profiles themselves, we'll also mark requester accounts with their top institution as well as collaborating institutions (if applicable)
Accountability
For years we've heard calls for Amazon to step in and mediate disputes between requesters & workers. From a practical standpoint we feel like this is the wrong approach to framing some of the problems online crowd workers face. In a scenario where Amazon acts as a mediator a researcher who does not put together properly run studies can simply move elsewhere. Most of our workers also exist on other similar platforms so shoving that work off onto them elsewhere is not a solution.
Most institutions, however, already have existing infrastructure to ensure researchers are conducting high quality, ethical research. That system (IRB/ERB) is one we're hoping to give tools to make better decisions and offer more constructive guidance. By offering institutions an overview of the work their researchers are uploading we're giving them a tool to take participant's experiences into account.
This also opens avenues in the future where TurkerView can begin to take collective complaints to a requester or institution automatically instead of requiring dozens of workers to waste time & effort doing the same task. We are aiming to streamline what is usually a frustrating, time consuming process for both parties. Workers get understandably upset when they need to take time out of their work to chase down a rejection or debug a broken task, and no researcher enjoys the sometimes rude inquiries in cases that are often caused by simple human error / misunderstanding.
Institution Profiles are a step in helping us better understand where to direct our efforts to support workers & the work they do on MTurk. Coming next week we're already planning on releasing a new metric that these profiles helped us generate that directly (and more objectively) measures how "fair" a requester (academic or otherwise) treats workers.
Service Accounts
We love TurkPrime (SurveyComet TurkerView Profile) & Positly (TurkerView Profile). They do amazing work to make Mechanical Turk more accessible to researchers, and they generally treat workers well. We've designated both of their accounts with special tags to explain to workers that while they are not necessarily directly related to Institutions, they are important parts of the platform & can generally be treated with the same regard as the organizations/institutions they work with.
New Feature: Consent Form Storage/Retrieval
MTurk communities have always shared information to help get rejections overturned, which sometimes involves asking an IRB to step in and help researchers fix mistakes. In light of that, and because we prefer to help keep contact information discreet (posting them to public forums often meant private contact information was cached by search engines) we're offering this information to qualified workers to retrieve right from our website.
In order to retrieve a consent form (which includes researcher / IRB contact information) a worker must meet the following criteria:
- A reviewer account in good standing (no TOS violations such as rude, abusive, or inflammatory reviews - this is in accordance with MTurk's Participation Agreement)
- A high approval rating across reviews & a minimum of 10 submissions in the last 7 days (with our quick submission feature submitting information to TV should take no more than 1 click and our goal response times are <200ms for submissions)
- Have submitted a review for the HIT a worker wants to retrieve a consent form for (we will not hand out contact data without this as it verifies you've completed the HIT & should have access to the consent form for it).
You can head to your profile and search for any HITs to retrieve contact information, or use the "My Reviews" option on a requester profile to display reviews and, if we have a consent form available, the link will be placed at the bottom of your review.
Awesome! What else is there?
For starters, you can visit the Institutions Listings on our website! Feel free to familiarize yourself with the layout or shoot us any feedback or improvements you'd like to see. They're a work in progress and we'll be workshopping them heavily as we find out what works and what doesn't for you so please send us feedback!
We also recommend you grab our free TamperMonkey script, TurkerViewJS, which will be updating to bring you this data live on Mechanical Turk's worker interface. It also has a wealth of information available about requesters, HITs, a worker-only heads up display which features warnings to help you avoid trouble and some general improvements to the MTurk interface/functionality.
MTurk Suite, a third party extension for MTurk, is also an amazing, simple to use tool that allows access to our data and provides a great set of enhancements to the default worker experience.
We also encourage you to contribute by sharing your experiences with various HITs/Requesters on the platform. Leave a review, let others know why you're returning a HIT, or just stop by a worker forum & say hello! You can check out our full guide to reviews as well as a quick how-to video on this page.
And if you feel like we're helpful please also consider upgrading your account to a paid plan to support us. You'll get access to some cool stats on HITs that can help you prioritize what to work on and preview access to new, upgraded features as we work on them. We can't continue to do what we do without your help, so we appreciate everyone who contributes their experiences & reviews with us!
Happy Turking!
