Detecting Different Forms of Semantic Shift in Word Embeddings via Paradigmatic and Syntagmatic Association Changes

Abstract

Automatically detecting semantic shifts (i.e., meaning changes) of single words has recently received strong research attention, e.g., to quantify the impact of real-world events on online communities. These computational approaches have introduced various measures, which are intended to capture the somewhat elusive and undifferentiated concept of semantic shift. On the other hand, there is a longstanding and well established distinction in linguistics between a word’s paradigmatic (i.e., terms that can replace a word) and syntagmatic associations (i.e., terms that typically occur next to a word). In this work, we join these two lines of research by introducing a method that captures a measure’s sensitivity for paradigmatic and/or syntagmatic (association) shifts. For this purpose, we perform synthetic distortions on textual corpora that in turn induce shifts in word embeddings trained on them. We find that the Local Neighborhood is sensitive to paradigmatic and the Global Semantic Displacement is sensitive to syntagmatic shift in word embeddings. By applying the newly validated paradigmatic and syntagmatic measures on three real-world datasets (Amazon, Reddit and Wikipedia) we find examples of words that undergo paradigmatic and syntagmatic shift both separately and at the same time. With this more nuanced understanding of semantic shift on word embeddings, we hope to analyze a similar concept of semantic shift on RDF graph embeddings in the future.