On Utilizing Nonstandard Abbreviations and Lexicon to Infer Demographic Attributes of Twitter Users

Abstract

Automatically determining demographic attributes of writers with high accuracy, based on their texts, can be useful for a range of application domains, including smart ad placement, security, the discovery of predator behaviors, enabling automatic enhancement of participants’ profiles for extended analysis, and various other applications. It is also of interest from the perspective to linguists who may wish to build on such inference for further sociolinguistic analysis. Previous work indicates that attributes such as author gender can be determined with some amount of success, using various methods, such as analysis of shallow linguistic patterns or topic, in authors’ written texts. Author age appears more difficult to determine, but previous research has been somewhat successful at classifying age as a binary (e.g. over or under 30), ternary, or even as a continuous variable using various techniques. In this work, we show that word and phrase abbreviation patterns can be used toward determining user age using novel binning, as well as toward determining binary user gender, and ternary user education level. Notable results include age classification accuracy of up to 83% (67% above relative majority class baseline) using a support vector machine classifier and PCA extracted features, including n-grams. User ages were classified into 10 equally sized age bins and achieved 51% accuracy (34% above baseline) when using only abbreviation features. Gender classification achieved 75% accuracy (13% above baseline) using only abbreviation features, PCA extracted, and education classification achieved 62% accuracy, 19% above baseline with PCA extracted abbreviation features. Also presented is an analysis of the evident change in author abbreviation use over time on Twitter.