The Facebook-owned network first announced the concept last year, saying it had been training a tool to detect and suppress “offensive and spammy” comments in at least nine languages. The filter, powered by machine learning, is turned on by default, but users can opt out to let comments people leave on their posts go unfiltered. They can also enter specific keywords they want to use to filter comments they receive.
Humans at Instagram trained the system, which is built on a Facebook-developed text-processing system called DeepText, by sorting negative comments into categories, including bullying, racism, and sexual harassment, the New York Times reports.
DeepText uses deep neural networks and other state-of-the-art machine-learning tools to try to parse language in context, Facebook has said. In theory, that should reduce the chance of the filters blocking too much innocuous content (which could lead users to disable it on their profiles ) or making it too easy for bullies and trolls to circumvent the filters by, say, tweaking the spelling of insults and slurs.A survey released last fall by the British anti-bullying group Ditch the Label found that 42% of young people had experienced bullying on Instagram, the highest of the networks the survey inquired about. By comparison, 37% reported bullying on Instagram’s sibling network Facebook, and 31% on Snapchat.
Since teens and even younger kids often deliberately use social media to communicate away from the eyes of parents and teachers, adults can sometimes be unaware of the bullying that goes on on the platforms. Ideally, a sufficiently powerful AI system can help curb bad behavior without kids needing to tattle, feeling like their profiles are being reviewed by human censors, or having to abandon the platforms where all their friends socialize.
“Because Snapchat, Instagram, and texting are where teens hang out all day, being bullied in these very public online spaces can become an emotional torture chamber for a teen,” warned McAfee family safety evangelist Toni Birdsong in a blog post last year.
Even celebrities have experienced Instagram bullying: Selena Gomez, who has one of the photo-sharing service’s most popular accounts with roughly 136 million followers, told the New York Times last year that insulting comments led her to delete the app from her phone at least once a week.
“We are also expanding our policies to guard against bullying young public figures on our platform,” wrote Instagram cofounder and CEO Kevin Systrom in a blog post. “Protecting our youngest community members is crucial to helping them feel comfortable to express who they are and what they care about.”
Originally Published by FastCompany, continue reading here.
