Location identification for real estate investment using data analytics

Abstract

The modeling and control of complex systems, such as transportation, communication, power grids or real estate, require vast amounts of data to be analyzed. The number of variables in the models of such systems is large, typically a few hundred or even thousands. Computing the relationships between these variables, extracting the dominant variables and predicting the temporal and spatial dynamics of the variables are the general focuses of data analytics research. Statistical modeling and artificial intelligence have emerged as crucial solution enablers to these problems. The problem of real estate investment involves social, governmental, environmental and financial factors. Existing work on real estate investment focuses predominantly on the trend predictions of house pricing exclusively from financial factors. In practice, real estate investment is influenced by multiple factors (stated above), and computing an optimal choice is a multivariate optimization problem and lends itself naturally to machine learning-based solutions. In this work, we focus on setting up a machine learning framework to identify an optimal location for investment, given a preference set of an investor. We consider, in this paper, the problem to only direct real estate factors (bedroom type, garage spaces, etc.), other indirect factors like social, governmental, etc., will be incorporated into future work, in the same framework. Two solution approaches are presented here: first, decision trees and principal component analysis (PCA) with K-means clustering to compute optimal locations. In the second, PCA is replaced by artificial neural networks, and both methods are contrasted. To the best of our knowledge, this is the first work where the machine learning framework is introduced to incorporate all realistic parameters influencing the real estate investment decision. The algorithms are verified on the real estate data available in the TerraFly platform.

Appendices

Appendix A

Procedure 1

Let $\mathscr {F}=\{f_{1},f_{2},f_{3}\ldots f_{n}\}$ be the set of features (attributes) $\forall \mathscr {F}\in \mathbb {R}^{n}$, A feature $f_{*}$ is called a root of D, if the information gain $\mathrm{IG}|_{f_{*}}=\mathrm{sup}(\mathrm{IG}|_{f_{*}},f_{j}\in \mathscr {F}.$

Steps. Let $\mathscr {F}=\{f_{1},f_{2},f_{3}\ldots f_{n}\}$ be the set of features $\forall \mathscr {F}\in \mathbb {R}^{n}.$ Let the randomness in any variable be defined by entropy:

$$\begin{aligned} H=-p\mathrm{log}_{2}p \end{aligned}$$

(10)

where p is the probability of occurrences of instances in the column of a truth table.

Let the target be $\tau =\{p_{1},p_{2}\ldots p_{c}\}$ ,where c is the number of class.¹⁴

Let D be the decision tree $\ni D:\mathscr {F}\rightarrow \tau$; we find the root of D.

We find the information before split of a parent node (in our case the output column) by $I_{BS}=-p_{1}\mathrm{log}_{2}p_{1}-p_{2}\mathrm{log}_{2}p_{2} -\cdots -p_{c}\mathrm{log}_{2}p_{c}=$

$$\begin{aligned} \sum \limits _{d=1}^c-p_{d} \mathrm{log}_{2} p_{d} \end{aligned}$$

(11)

Consider the feature $f_{i}\in \mathscr {F}$ having two classes (1 or 0). The net information of the children nodes is given by

$$\begin{aligned} I_{AS}= p_\mathrm{t}\bigg (\sum \limits _{j=1}^c -p_{j} \mathrm{log}_{2}p_{j}\bigg )+ p_\mathrm{f}\bigg (\sum \limits _{k=1}^c -p_{k}\mathrm{log}_{2}p_{k}\bigg ) \end{aligned}$$

(12)

Let the truth occurrences in the children (the split probability of truths) be $p_\mathrm{t}$ and that for the falses be $p_\mathrm{f}$. Let $p_{j}$ and $p_{k}$ be the probabilities of the target accompanied with the truths and the falses, respectively.¹⁵ Every instances in Eq. (12) are written according to the entropy of (10). The total information gain is obtained by subtraction of (12) from (11). Therefore, $I_\mathrm{g}=I_\mathrm{BS}-I_\mathrm{AS}$

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d} \mathrm{log}_{2} p_{d}+p_\mathrm{t}\bigg (\sum \limits _{j=1}^c p_{j} \mathrm{log}_{2}p_{j}\bigg )\nonumber \\&+p_\mathrm{f}\bigg (\sum \limits _{k=1}^c p_{k}\mathrm{log}_{2}p_{k}\bigg ) \end{aligned}$$

(13)

The following conditions are applied throughout the root identification process.

• $0\le p_\mathrm{t}\le 1,0\le p_\mathrm{f}\le 1\ni p_\mathrm{t}+p_\mathrm{f}=1$

• $\Bigg \{ 0\le \sum \nolimits _{j=1}^cp_{j}\le 1, 0\le \sum \nolimits _{j=1}^cp_{k}\le 1\Bigg \} \ni \Bigg \{\sum \nolimits _{j=1}^cp_{j}+\sum \nolimits _{j=1}^cp_{k}= \sum \nolimits _{d=1}^c p_d\Bigg \}$

• $\sum \nolimits _{d=1}^cp_{d}=1$

Let us identify the root node (with the highest information gain by induction in Eq. (12) for five cases and its variants (totally eleven in the following) discussed prior):

Case 1: When $p_\mathrm{t}=1,p_{j}=0 \bigvee p_\mathrm{t}=0,p_{j}=p_{d}$with$p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$.

If $p_{j}=0$, we have $p_{k}=p_{d}$; substituting in (13) and changing the limits, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+(1-p_\mathrm{t}) \sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}\nonumber \\= & {} -\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(14)

Case 2: When $p_\mathrm{t}=1,p_{j}=P_{d} \bigvee p_\mathrm{t}=0,p_{j}=0$ , with$p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$.

If $p_{j}=p_{d}$, we have $p_{k}=0$; substituting in (13) and changing the limits we have:

$$\begin{aligned} I_\mathrm{g}=-\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} +\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}=0 \end{aligned}$$

(15)

Case 3: When $p_\mathrm{t}=1,p_{j}=p_{k}\bigvee p_\mathrm{t}=0,p_{j}=p_{k}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$.

If $p_{j}=p_{k}$, we have $p_{k}=\frac{p_{d}}{2}$; substituting in (6) and on further simplification, we have:

$$\begin{aligned} I_\mathrm{g}=-\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} +\sum \limits _{d=1}^c\bigg (p_{d}\mathrm{log}_{2}p_{d}-p_{d}\bigg ) {=}-\sum \limits _{d=1}^c p_{d}\nonumber \\ \end{aligned}$$

(16)

Case 4: When $0<p_\mathrm{t}<\frac{1}{2},p_{j}=p_{d}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=p_{d}$, then $p_{k}=0$; substituting in (13) and on further simplification, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+p_\mathrm{t} \sum \limits _{d=1}^c\bigg (p_{d}\mathrm{log}_{2}p_{d}\bigg )\nonumber \\= & {} (p_\mathrm{t}-1)\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}\succ -\frac{1}{2} \sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(17)

Case 5: When $0<p_\mathrm{t}<\frac{1}{2},p_{j}=0$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=0$, then $p_{k}=p_{d}$; substituting in (13) and on further simplification, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+(1-p_\mathrm{t}) \sum \limits _{d=1}^c\bigg (p_{d}\mathrm{log}_{2}p_{d}\bigg )\nonumber \\= & {} (-p_\mathrm{t})\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(18)

Case 6: When $0<p_\mathrm{t}<\frac{1}{2},p_{j}=p_{k}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=p_{k}$, then $p_{k}=\frac{p_{d}}{2}$; substituting in (13) and on further simplification, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} +\frac{1}{2}\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} -\sum \limits _{d=1}^c p_{d}\nonumber \\= & {} -\frac{1}{2}\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} -\sum \limits _{d=1}^c p_{d}\prec 0 \end{aligned}$$

(19)

Case 7: When $p_\mathrm{t}>\frac{1}{2},p_{j}=0$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=0$, we have $p_{k}=p_{d}$; substituting in (13) and changing the limits, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+(1-p_\mathrm{t}) \sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}\nonumber \\= & {} -p_\mathrm{t}\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}> -\frac{1}{2} \sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(20)

Case 8: When $p_\mathrm{t}>\frac{1}{2},p_{j}=p_{d}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=0$, we have $p_{k}=0$; substituting in (13) and changing the limits, we have:

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+p_\mathrm{t} \sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}\nonumber \\= & {} (1-p_\mathrm{t})\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}< -\frac{1}{2} \sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(21)

Case 9: When $p_\mathrm{t}>\frac{1}{2},p_{j}=p_{k}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

If $p_{j}=p_{k}$, then $p_{k}=\frac{p_{d}}{2}$; substituting in (13) and on further simplification, we have:

$$\begin{aligned} I_\mathrm{g}=-\frac{1}{2}\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} -\sum \limits _{d=1}^cp_{d}<0 \end{aligned}$$

(22)

Case 10: When $p_\mathrm{t}=\frac{1}{2},p_{j}=p_{d} \bigvee p_\mathrm{t}=\frac{1}{2},p_{j}=p_{d}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

$$\begin{aligned} I_\mathrm{g}= & {} -\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d}+(1-\frac{1}{2}) \sum \limits _{d=1}^c\bigg (p_{d}\mathrm{log}_{2}p_{d}\bigg )\nonumber \\= & {} -\frac{1}{2}\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(23)

Case 11: When $p_\mathrm{t}=\frac{1}{2},p_{j}=p_{k}$ with $p_{j}+p_{k}=p_{d},p_\mathrm{t}+p_\mathrm{f}=1$

$$\begin{aligned} I_\mathrm{g}=-\frac{1}{2}\sum \limits _{d=1}^cp_{d}\mathrm{log}_{2}p_{d} -\frac{1}{2}\sum \limits _{d=1}^cp_{d}<0 \end{aligned}$$

(24)

For remaining conditions, we can apply (13) to obtain information gain, which gives the maximum information gain of a tree. Let us analyze the above cases; the conditions used to obtain (14) are in contradiction to one another, i.e., $p_\mathrm{t}=1$ and $p_j=0$ cannot happen at the same time. Hence, this case can never happen in a decision tree. $I_\mathrm{g}$ in Eqs. (17) and (20) are the optimal for the information gain and best suited for the decision tree operation. In the rest of the cases, the probability conditions do not occur due to contradiction or they do not lead to maximum information gain.

Relation between information gain$I_\mathrm{g}$and entropy$H_{s}$: (a general result)

Let us denote the overall entropy (combined entropy of parent and children) as $H_{s}$. We find a relation between $H_s$ and $I_\mathrm{g}$.

$$\begin{aligned} H_{s}= -\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d}-p_\mathrm{t} \sum \limits _{j=1}^c p_{j}\mathrm{log}_{2}p_{j}-p_{j} \sum \limits _{k=1}^c p_{k}\mathrm{log}_{2}p_{k} \end{aligned}$$

(25)

When we add Eqs. (6) and (17), we get:

$$\begin{aligned} I_\mathrm{g}+H_{s}=-2\sum \limits _{d=1}^c p_{d}\mathrm{log}_{2}p_{d} \end{aligned}$$

(26)

In (26), the R.H.S is a constant because the class probabilities in the target column will not change. Hence, we can conclude that

$$\begin{aligned} I_\mathrm{g}+H_{s}= \mathrm{constant} \end{aligned}$$

(27)

This follows the notion of a straight line with a negative slope.¹⁶

Simulation results of Procedure-1: We simulated the equations in MATLAB 2014. The simulation parameters were as follows: number of classes=3 (nevertheless in our work, it is a 9 class problem, because cluster has 9 landmarks, for the analysis of the theorem and simulations, let us choose number of classes as 3), the probability of classes: $p_{1}=0.1,p_{2}=0.1$ and $p_{3}=0.8$. Let the truth occurrences in the children (the split probability of truths) be $p_\mathrm{t}$ and that for the false be $p_\mathrm{f}$.¹⁷ Let $p_{j}$ and $p_{k}$ be the probabilities of the target accompanied with the truths and the falses, respectively. The graphs are plotted for the different conditions of $p_\mathrm{t}$—$p_\mathrm{t}=0,p_{t}=1,p_\mathrm{t}=\frac{1}{2},p_\mathrm{t}=0.3,p_\mathrm{t}=0.7$. The value $p_\mathrm{t}=0.3$ is a representative of the condition $0\le p_\mathrm{t}<\frac{1}{2}$, and $p_\mathrm{t}=0.7$ is a representative of the condition $p_\mathrm{t}>\frac{1}{2}$. Since the information gain is always positive, the iteration on $p_\mathrm{f}$ will give the same outputs/results, since $p_\mathrm{t}+p_\mathrm{f}=1$. Let the terms associated with $p_\mathrm{t}$ be $p_{k1},p_{k2},p_{k3}$ and with $p_\mathrm{f}$ be $p_{f1},p_{f2},p_{f3}$. The information gain in (13) can be written as: $I_\mathrm{g}=-p_{1}\mathrm{log}_{2}p_{1}-p_{2}\mathrm{log}_{2}p_{2}-p_{3}\mathrm{log}_{2}p_{3} +p_\mathrm{t}\{p_{k1}\mathrm{log}_{2}p_{k1}+p_{k2}\mathrm{log}_{2}p_{k2}+p_{k3}\mathrm{log}_{2}p_{k3}\} +p_\mathrm{f}\{p_{f1}\mathrm{log}_{2}p_{f1}+p_{f2}\mathrm{log}_{2}p_{f2}+p_{f3}\mathrm{log}_{2}p_{f3}\}$.

This equation can be rewritten as: $I_\mathrm{g}=-p_{1}\mathrm{log}_{2}p_{1}-p_{2} \mathrm{log}_{2}p_{2}-p_{3}\mathrm{log}_{2}p_{3}+p_\mathrm{t}\{p_{k1}\mathrm{log}_{2}p_{k1} +p_{k2}\mathrm{log}_{2}p_{k2}+p_{k3}\mathrm{log}_{2}p_{k3}\}+(1-p_\mathrm{t}) \{(1-p_{k1})\mathrm{log}_{2}(p-p_{k1})+(1-p_{k1})\mathrm{log}_{2}(p-p_{k2}) +(1-p_{k1})\mathrm{log}_{2}(p-p_{k2})\}$. since $p_{j}+p_{k}=p_{d}$ and $p_\mathrm{t}+p_\mathrm{f}=1$.

We vary the $p_{k1},p_{k2},p_{k3}$ probabilities such that $0\le p_{k1}\le p_{1}, 0\le p_{k2}\le p_{2}$ and $0\le p_{k3}\le p_{3}$. The obtained graphs are shown in Fig. 7.

Fig. 10

a Plot of information gain versus possible combinations of $p_{t},p_{k},p_{j}$ where $p_\mathrm{t}=0$. b Plot of system entropy versus possible combinations of $p_{t},p_{k},p_{j}$,where $p_\mathrm{t}=0$. c Plot of information gain versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=1$. d Plot of system entropy versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=1$. e Plot of information gain versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=0.5$. f Plot of system entropy versus possible combinations of $p_{t},p_{k},p_{j}$,where $p_\mathrm{t}=0.5$. g Plot of information gain versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=0.3$. h Plot of system entropy versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=0.3$. i Plot of information gain versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=0.7$. j Plot of system entropy versus possible combinations of $p_{t},p_{k},p_{j}$, where $p_\mathrm{t}=0.7$

In Fig. 10a, we have fixed the truth occurrences $p_\mathrm{t}=0$ (meaning the feature has only false occurrences and there are no truths), and probability of class-1 occurrences is 0.1, probability of class-2 occurrences is 0.1 and that of class-3 is 0.8 in the target column. In Fig. 10a, the information gain reaches maximum when $p_{k}=p_{d}$ (meaning that all the classes of the target are associated with the truths). This is a contradiction, since there are no truth occurrences in the feature; the classes cannot associate with the truths of the children nodes. Hence, we can omit this condition and the system configuration (set of probabilities used), though the $I_\mathrm{g}$ obtained is 0.9219 which is the maximum of all the probability configurations, and if we move along x-axis, we can see 11 lobes in the information gain plot. Each main lobe has 11 sublobes, and each sublobe has 11 points which runs vertically. This is because of the possible combinations of $p_{1},p_{2},p_{3}$ each having 11 instances (i.e., 0 to 0.1 in steps of 0.01). Also, there is a decreasing slope between $I_\mathrm{g} and H$ which goes according to Eq. (26).

In Fig. 10c, we have repeated the simulations with $p_\mathrm{t}=1$ (meaning that all are truths in the considered feature column). The maximum information gain happens to be when $p_\mathrm{t}=0$ with the gain value of 0.9219. This implies that the feature column has only truths, and no classes are associated with the truths. This is a contradiction, and this will not happen at the same time. Hence, the system with the probability conditions aforementioned is neglected.

In Fig. 10e, we can notice that the information gain is symmetric when $p_\mathrm{t}=\frac{p_{d}}{2}$, where the information gain reaches exactly the half of the maximum of its value. The information gain reaches to its maximum value 0.4610 that happens when $p_\mathrm{t}=p_{d}$. It can be seen that the maximum value is exactly the half of the information gain obtained according to (13). This is not a point of operation for a decision tree because the information gain goes slightly negative at its minimum point $p_\mathrm{t}=\frac{p_{d}}{2}$ or we can assume it as 0. This is because the uncertainty in the system is beyond zero, which is a contradiction in the present scenario. But we can call the point $p_\mathrm{t}=p_{d}$ as the equilibrium point of operation. There is no gain; neither there is loss. The information of parent gets split among the children nodes equally.

Figure 10g is the case when $p_\mathrm{t}=0.3$, an instance where $0<p_\mathrm{t}<\frac{1}{2}$; we get the maximum information gain of 0.6453 when $p_\mathrm{t}=p_{d}$. It was also found that the information gain is always > 0.4610, which is according to Eq. (16). It is clear that the information gain has a hard threshold where it stays always above. The feature with $0<p_\mathrm{t}<\frac{1}{2}$ has maximum gain when all the classes are associated with truths itself. Less truth probability with all classes associated with it gives the optimal information gain.

Figure 10i is the case when $p_\mathrm{t}=0.7$, an instance where $p_\mathrm{t}>\frac{1}{2}$; we get the maximum information gain of 0.6453 when $p_\mathrm{t}=0$. It was also found that the information gain is always > 0.4610, which is according to Eq. (20). It is clear that the information gain has a hard threshold, and it always stays above that. The feature with $p_\mathrm{t}>\frac{1}{2}$ has maximum gain when all the classes are associated with false, meaning that none are associated with the truths. Even though the classes are associated with the false, the parent can get the maximum information gain in this case as well. We conclude that the probability conditions mentioned in Case 4 and Case 7 are the best conditions to choose an attribute as the root node. In other words, whichever attributes satisfies the conditions of Case 4 and Case 7 are placed as the root node of a tree.

Figure 10b, d, f, h, j are the plot of system entropy versus system probabilities, and is according to Eq. (26).

Appendix B

See Tables 6, 7 and 8.

Table 6 Validation of decision tree (layer-1 classification)

Table 7 Validation of PCA$+$K-means (layer-2 classification)

Table 8 Validation of ANNs$+$K-means (layer-2 classification, cluster centers match error)