Search Relevance Rating, Online Data Rating
Online Data Rating and Validation for Multinational Technology Company
The Challenge
Our client, a multinational technology company, was looking for a large, in-country workforce of raters to validate and review search queries and results across audio, image, text, and video data. Since the project spanned globally, our client requested the team be reactive and available around the clock, with both high flexibility and scalability. Our client also outlined detailed guidelines and quality standards to ensure that the team be up to date with new trends, comfortable with search, and have sharp eyes for detail. With the various data types in scope, this large-scale rating and validation initiative spread across 150-plus projects globally.
• • • •The Solution• • • •
In the initial stages of the project, DataForce assembled a group of fewer than 50 raters with a growth of 20% daily, climbing to a total of 1,500 active and trained raters within six months. As the project grows with advanced specifications, DataForce continues to source an in-country workforce, providing a variety of project-specific training programs. Aside from onboarding, the team is in constant communication with the raters, providing a ticketing system for feedback and issues should they arise. Adhering to the guidelines set by our client, an intensive quality assurance (QA) process was established,aligned with our client’s expectation. The guidelines include several different layers of QA to assess the raters’ speed and fraud prevention measures (IP checks, resume authentication, etc.), all while ensuring bi-weekly payments and performance reporting for the community members.
Some examples of the projects include:
- Point of Interest: the confirmation of location-specific data (operating hours, address, etc.)-Search: the validation of search results, confirming both accurate and up-to-date results.
- Product Identification: the validation and comparison of various products such as clothing, accessories, and food, ensuring the search results match the link, product description, color, style, etc.
- Auto Complete: the validation of system prediction when writing a query involving an abbreviation or finishing a statement, confirming the system understands what it is being asked.
- Sentiment Analysis: reviewing and tagging text data with positive, negative, and neutral classification to train the AI system to the emotion behind the text.
- Podcast: the rater is provided a text-based synopsis of the audio file and they must review the text and listen to the podcast to confirm the copy matches what is said on the podcast.
- Audio Quality Rating: the evaluation of sound quality for text-to-speech models, confirming if the output sounds natural (high quality) or robotic (low quality).
- Offensive Content Moderation: mitigating unsuitable online content such as adult material, hate speech, and other detrimental motives through diligently identifying and assessing image, video, audio, and text data that should appear in various search results.
As this project advances, DataForce will continue to facilitate the enhancement of our client’s model by meticulously following the specific guidelines provided, all while assembling and training a network of geographically diverse raters across the world.
DataForce has a global community of over 1,000,000 members from around the globe and linguistic experts in over 250 languages. DataForce is its own platform but can also use client or third-party tools. This way, your data is always under control.