Hate Speech Detector
Challenge
Abusive content in social media is pervasive and poses a serious problem. Due to the users’ anonymity on the Internet, people feel unreachable when it comes to the punishment. As a result, many cybernauts may suffer from being insulted. The number of such insults and even threats is overwhelming. It is impossible to handle all of these cases individually and the automated methods have to be very sophisticated as the issue is much more complicated than simply banning the list of forbidden words.
Solution
The most effective solution is a computational and intelligent method able to automatically detect offensive language in various types of utterances posted on the web and identify their target. The system was created using multiple deep learning models based on state of the art Transformer architectures, which are able to extract rich features from online utterances and use them to identify offensiveness, agression, toxicity etc. The method also leverages extensive text pre-processing in order to appropriately prepare input data before feeding it into the complex artificial neural system. Since abusive content occurs relatively rarely comparing to civil posts, advanced dataset balancing methods were employed in order to easily capture the real nature of offensiveness.
Result
The main achievement is the significant improvement in recognition of hostile online utterances. Up to 90% of hateful statements have been correctly recognized along with minimizing the number of false alarms. It made an important contribution to increasing safety in cyberspace and reducing the churn rate.
The members of our team had a significant share in planning and development of the final solution.
