Constructing VeSNet: Mapping LOD Thesauri onto Princeton WordNet and Polish WordNet

Abstract

Lexical resources are crucial in many modern applications of Natural Language Processing and Artificial Intelligence. We present VeSNet – a network of lexical resources resulting from the merge of Polish-English WordNet (PEWN) with several existing large electronic thesauri from the Linked Open Data cloud (DBpedia, Wikipedia, GeoWordNet, Agrovoc, Eurovoc, Gemet and MeSH). We describe the procedure of making the resource and depict its elementary properties, as well as, evaluate its quality. The created lexical network is characterised both by great coverage and high precision: nearly 1.3M new exactMatch links were created, including 85K to PEWN, with the estimated precision of 94%.

6 Appendix

1.1 6.1 PSA as the Cohen’s Kappa Limit

Positive specific agreement could be derived from the definition of Cohen’s kappa with the assumption that the negative class is infinitively large. We start from the definition of $\kappa$ (with letters’ meaning established in the confusion Table 2):

$$\begin{aligned} \begin{aligned} \kappa =\frac{A_0-A_e}{1-A_e},~A_0=\frac{a+d}{N},\\ A_e=p_Y+p_N=(\frac{a+b}{N}\cdot \frac{a+c}{N})+(\frac{d+b}{N}\cdot \frac{d+c}{N}), \end{aligned} \end{aligned}$$

(3)

where $N=a+b+c+d$ is the total number of annotations, $A_0$ is the observed relative agreement, $A_e$ is agreement expected by chance, and $p_Y$ as $p_N$ are joint probabilities of choosing “Y” and “N”, respectively [15]. Hence,

$$\begin{aligned} \begin{aligned} \kappa = \frac{\frac{a+d}{N}-(\frac{a+b}{N}\cdot \frac{a+c}{N}+\frac{d+b}{N}\cdot \frac{d+c}{N})}{1-(\frac{a+b}{N}\cdot \frac{a+c}{N}+\frac{d+b}{N}\cdot \frac{d+c}{N})}=\frac{\frac{N\cdot (a+d)}{N^2}-\frac{(a+b)\cdot (a+c)+(d+b)\cdot (d+c)}{N^2}}{\frac{N^2-((a+b)\cdot (a+c)+(d+b)\cdot (d+c))}{N^2}}\\ =\frac{N\cdot (a+d)-((a+b)\cdot (a+c)+(d+b)\cdot (d+c))}{N^2-((a+b)\cdot (a+c)+(d+b)\cdot (d+c))}. \end{aligned} \end{aligned}$$

(4)

In the numerator after elementary calculations we obtain:

$$\begin{aligned} \begin{aligned} (a^2+ab+ac+2ad+bd+cd+d^2)\\ -(a^2+ab+ac+2bc+bd+cd+d^2)=2(ad-bc), \end{aligned} \end{aligned}$$

(5)

while in the denominator we get:

$$\begin{aligned} \begin{aligned} a^2+2ab+b^2+2ac+2bc+2ad+2bd+c^2+2cd+d^2\\ -(a^2+ab+ac+2bc+bd+cd+d^2)=\\ =ab+b^2+ac+2ad+bd+c^2+cd. \end{aligned} \end{aligned}$$

(6)

Substituting the results 5 and 6 into the fraction 4 we obtain:

$$\begin{aligned} \kappa = \frac{2\cdot (ad-bc)}{ab+b^2+ac+2ad+bd+c^2+cd} = \frac{d\cdot (2a-\frac{2bc}{d})}{d\cdot (\frac{ab+b^2+ac+c^2}{d}+2a+b+c)} \end{aligned}$$

(7)

As d aproaches infinity kappa approaches PSA:

$$\begin{aligned} \lim _{d \rightarrow \infty }\kappa = \lim _{d \rightarrow \infty } \frac{2a-\frac{2bc}{d}}{\frac{ab+b^2+ac+c^2}{d}+2a+b+c} = \frac{2a-0}{0+2a+b+c} = PSA.~~\square \end{aligned}$$

(8)

1.2 6.2 PSA as the Harmonic Mean of Annotator Percentage Agreements

Lets define annotator percentage agreement (APA) as the number of agreed cases divided by the total number of each annotator decisions. For the confusion Table 2 we define APA as follows:

$$\begin{aligned} \begin{aligned} APA_1=\frac{a}{a+b},~APA_2=\frac{a}{a+c}, \end{aligned} \end{aligned}$$

(9)

“1” stands for the first annotator, while “2” for the second one. The PSA index could be then defined as the harmonic mean of $APA_1$ and $APA_2$:

$$\begin{aligned} PSA=\frac{2\cdot APA_1\cdot APA_2}{APA_1+APA_2}. \end{aligned}$$

(10)

Through elementary transformations we obtain from the definition 10:

$$\begin{aligned} \begin{aligned} PSA=\frac{2\cdot \frac{a}{a+b}\cdot \frac{a}{a+c}}{\frac{a}{a+b}+\frac{a}{a+c}}=\frac{\frac{2a^2}{(a+b)\cdot (a+c)}}{\frac{a(a+c)+a(a+b)}{(a+b)\cdot (a+c)}}=\frac{2a^2}{a(a+c)+a(a+b)}=\frac{2a}{2a+b+c}.~~\square \end{aligned} \end{aligned}$$

(11)