Social media sites are challenged by both the scale and vari- ety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guide- lines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is of- ten regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and vi- sual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and un- derstanding of this approach to challenging, deviant content. We discuss how automation might impact community modera- tion, and the ethical and social obligations of this area

, , , , , ,
doi.org/10.1145/3025453.3025985
SIGCHI Conference on Human Factors in Computing Systems

Chancellor, S., Kalantidis, Y., Pater, J. A., & De Choudhury, M. (2017). Multimodal classification of moderated online pro-eating disorder content. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3213–3226). doi:10.1145/3025453.3025985