The Center for Data Innovation spoke with Allon Mason, CEO of UserWay, a company based in Delaware that creates innovative website accessibility solutions. Mason discussed the company’s new AI-powered content moderator that helps websites reduce the use of biased, discriminatory, and racially-charged language.
Hodan Omaar: Many of the terms that would be flagged by Userway’s AI Content Moderator such as “blackmail” or “chairman” have only recently been understood to be prejudicial. How does UserWay’s AI work to stay atop of the ever evolving complexities of sensitive language?
Allon Mason: We are approaching this carefully. New words or phrases are added for inspection only after vetting by our internal process and advisors, which include Dr. Israel Charny, a world leading expert in inciting language. New words for moderation are then offered to site administrators to accept or edit as they wish. Their responses are then reported back to our aggregator in a privacy-preserving manner, but these responses aren’t attributed to specific users. Instead, they are generalized and used by the moderator to improve the systems suggestions.
In this way we see how the moderator is being applied, and where it’s being ignored. We build that back into our engine, and that will then slowly, over time, build more intelligent analysis. As more data is collected, the system is better able to identify bias and sentiments and offer more appropriate recommendations. In short, every person using the moderator improves it for future users.
Since we are focusing on English terms and have committee members and contributors across the globe, we are confident that our list will evolve with the times without making reactionary missteps in what’s included.
Omaar: How dynamic is the tool? How often will it scan a website for new sensitive material and is it able to scan material that was already posted on a site?
Mason: This is all happening in real-time. The initial scan will evaluate all content on all pages selected by the site owner, and that can range from minutes to a few hours depending on the size of the site. It’s important to note here that it can take a bit longer because we are not just scanning for offensive content, but also accessibility violations on those pages. However, once that initial scan is completed, the site owner will receive real-time updates when a new term is flagged on their site, just the same way they are currently alerted to an accessibility violation.
Omaar: Does the tool work in other languages and if not, are there plans to include other languages in the future?
Mason: At the moment, we are focusing on English. We do however, distinguish between different English-speaking countries. So a user in Australia might have different alerts than one in Scotland. Since local slang and certain terms are considered questionable in one place and not the other, we take that into consideration.
Currently, there aren’t plans to expand into other languages. We will consider that expansion once the current version has been tested and refined to our satisfaction.
Omaar: Some of the platforms UserWay supports, like WordPress, may have content where it is contextually acceptable to use certain language, for instance a blog that uses African-American Language (AAL). How does AI-powered content moderation work to perceive and safeguard such nuances?
Mason: Context is everything, and that’s one of the biggest benefits of our moderator. Decisions are never made for the user—they have full control. Where so many companies are trying to tackle this issue with a search and replace approach, we allow for more thought and consideration. If they disagree with a term being flagged, the user can just ignore it.
Automatic language detection can be added in the future to detect such settings and apply them.
Omaar: Contrary to the last question, there has also been more conversation about companies using cultural lexicons to sell products, such as IHOP’s use of AAL in their social media campaign on Twitter. Do you think there is a role in the future for AI-powered solutions to help companies or brands review their material for contextual harm, or will context always be the place of humans?
Mason: It’s an interesting question that touches on how the latest innovations in AI, machine learning, and deep learning can help determine the best recommendation in a given context, including language, tone, mood and motivations.
Natural language processing has made huge leaps in the last year alone. Transformer-based models such as the new GPT-2 and GPT-3 can recommend edits to text in a way that previous vector models simply cannot. They act within a specific context in seemingly miraculous ways. However, these less-understood machine learning algorithms pose an algorithmic risk, and their errors are unpredictable. These are the challenges that UserWay continues to tackle.
For now, we use more traditional context detection based on classical AI decision trees and word stemming.