The Solution
In February 2008, a system was put in place which exported reviews to a CFL program, which checks the similarity between a Scout’s reviews and thus highlights any Scouts who has copied and pasted or used templates. The system takes around 5-10 seconds to complete a simultaneous and cross referencing analysis of up to 20,000 reviews and the administrators can then easily identify the abusers and take the appropriate action. This reduced the total workload by over 60% to around 3 hours per day and reduced account closures by around 50%.
This initial solution was a great success and Slicethepie decided that what they wanted was an integrated solution which would prevent these reviews ever reaching the artists (and also prevent the scout getting paid). CFL and Slicethepie determined what the tests would be and how to manage them.
In September 2008, the second solution incorporated CFL computational forensic linguistic systems into the Scout Room page itself. This was written in ActionScript 3 to sit inside the Flash player which handles the playing of the tracks and the submission of the review. Now when a Scout submits a review, it is checked against their previous reviews for copying and pasting or templating before it is accepted. The degree of similarity can also be preset enabling Slicethepie to fine tune the sensitivity of the system. It also checks against a lexicon of musical terminology to ensure that it actually relates to music. Finally, it checks the length of the review compared to the time taken to create it and hence calculates the Scout’s Words Per Minute rate, providing another indicator of copying and pasting. If any of these tests fail, the user is shown a light hearted message pointing out the error of their ways and given the opportunity to correct them. Only when the review passes all of these tests is it posted to the system, sent to the artist and the Scout payment made.
“The effect on workload was significant. What had been an 8 hours per day role was now down to just 1 hour per day, plus we identify 100% of bogus reviews”. Steve Cox, again. “Of course, the qualitative effects were just as important to us. The quality of the reviews is visibly better and the team received a lot of very positive feedback from artists and Scouts alike. The credibility of the Scout Rooms and Slicethepie has been reinforced.”
Development
CFL is now working with Slicethepie on a range of other developments including the ability to auto extract and weight the mood and the emotions that a track elicits in the listeners by ‘reading’ and analysing all reviews for a particular track.