For the Slicethepie team, the challenge was clear. How to enhance the credibility of the website by ensuring that users were unable to abuse the system, without constantly increasing overhead costs?
Slicethepie use their 70,000 users (they call them “Scouts”) to help identify the very best of the 10,000 artists currently using the site. The “Scout Rooms” are a highly effective filter to ensure that the best artists are put forward for financing on the site. The Scout Room is where Scouts are fed tracks anonymously and randomly and are paid to review and rate the tracks. The artist and track name are only revealed after the review is submitted.
The Scouts agree to submit considered and constructive reviews to help artists learn about their work. The problem was that there was a small number of Scouts who were using techniques to abuse the system, but with reviews being submitted at a rate of up to 19,000 reviews per day, how to identify them quickly and efficiently?
The techniques being used to cheat the system included copying and pasting the same review repeatedly, pasting reviews from other sources, operating multiple accounts or simply submitting random characters or irrelevant material.
“We were checking the reviews manually. This was hugely time consuming, ineffective (at best we identified 5-10%) and did little to support the integrity of the site as the review had already been received by the artist.” says Steve Cox, COO of Slicethepie Limited. “The Scout Room administrators not only had to identify the bad Scouts, they then had to lock their accounts, identify the extent of the abuse, contact the customer, remove earnings and then resolve the issue. No small task. This process was taking up the equivalent of 40 hours per week and was growing fast.”
With the site attracting hundreds of new Scouts and artists on a daily basis, this problem would only get worse and something had to be done - and quickly.
In January 2008, CFL met with the Slicethepie team and quickly identified that their software could have a major impact. CFL quickly reviewed sample data, used it to identify the main abuses and then proposed ways to identify and prevent them.