Google and its early-stage incubator Jigsaw have launched a new tool that uses machine learning to help publishers combat online abuse.
With Perspective, Google is offering online publishers an API that they can integrate into any of their platforms that facilitate user comments. Perspective taps a human-generated database of comments that have already been tagged as abusive or toxic. Through the API, publishers can connect their own comments with the hundreds of thousands on Google’s database, and Perspective then rates them based on how “similar” they are to the ones already flagged as “toxic.” Perspective facilities corrections from users too, so that it should improve as it receives more feedback from people using it.
The Perspective API essentially provides the data, while publishers can elect to use this data in different ways. Comments could be flagged by publishers to be manually reviewed by moderators or the community, alternatively a publisher could decide to illustrate to a commenter in real time that what they’re writing is abusive:
Online abuse has been a problem since the advent of the web, with many publishers ultimately ditching the comments sections due to the difficulty in policing them — earlier this month, Amazon-owned IMDb shuttered its comments section after years of battling toxicity. The IMDb board has now resurfaced on Reddit (good luck with that).
While most comments sections have a community-led approach for flagging abuse, and many are already able to filter out comments that contain profanities, being able to tap a gargantuan database of pre-flagged comments should help proceedings. And as more publishers pick up on it, Perspective will gain access to more comments and thus it should improve — Perspective is also now part of Google’s open-source software library TensorFlow, and its cloud Machine Learning platform.
Google’s latest initiative comes at a time when technology companies are increasingly investing in anti-abuse smarts. Just last week, Twitter unveiled a slew of new tools to battle trolls, while back in August Microsoft announced a similar initiative.
Google says it has been testing Perspective with the The New York Times, which has a team dedicated to sifting through comment before they go live — this amounts to 11,000 comments daily.
“This problem doesn’t just impact online readers,” explained Jigsaw president Jared Cohen, in a blog post. “News organizations want to encourage engagement and discussion around their content, but find that sorting through millions of comments to find those that are trolling or abusive takes a lot of money, labor, and time. As a result, many sites have shut down comments altogether. But they tell us that isn’t the solution they want. We think technology can help.”
Though the focus of Perspective is on abuse for now, the technology could also be appropriated for other comment-managing means. “Over the next year we’re keen to partner and deliver new models that work in languages other than English as well as models that can identify other perspectives, such as when comments are unsubstantial or off-topic,” added Cohen.