A cost model for random access queries in document stores

Abstract

Document stores have become one of the key NoSQL storage solutions. They have been widely adopted in different domains due to their ability to store semi-structured data and expressive query capabilities. However, implementations differ in terms of concrete data storage and retrieval. Unfortunately, a standard framework for data and query optimization for document stores is nonexistent, and only implementation-specific design and query guidelines are used. Hence, the goal of this work is to aid automating the data design for document stores based on query costs instead of generic design rules. For this, we define a generic storage and query cost model based on disk access and memory allocation that allows estimating the impact of design decisions. Since all document stores carry out data operations in memory, we first estimate the memory usage by considering characteristics of the stored documents, their access patterns, and memory management algorithms. Then, using this estimation and metadata storage size, we introduce a cost model for random access queries. We validate our work on two well-known document store implementations: MongoDB and Couchbase. The results show that the memory usage estimates have the average precision of 91% and predicted costs are highly correlated to the actual execution times. During this work, we have managed to suggest several improvements to document storage systems. Thus, this cost model also contributes to identifying discordance between document store implementations and their theoretical expectations.

Appendices

Calculating internal B-tree blocks

We calculated the probability of a leaf block of data B-tree in memory being requested $P^{\textit{req}}_d(C)$ as $1-(1-SF(C))^{R_d(C)}$, in Eq. 10. Hence, we continue the calculation of the internal nodes of the data B-tree as follows.

If a document is in the cache, the data block containing the document and the internal block containing the reference entry to that data block must be in the cache. On the contrary, if an internal block is not in the cache, none of the data blocks pointed by the reference entries in it can be in the cache. Therefore, for an internal block not to be in the cache, all of the reference entries of the block should reference blocks, not in the cache. Thus, since the probability of a single reference entry referring to a leaf block not in the cache is $1-P^{\textit{req}}_d(C)$, and there are $R_{\textit{int}}(C)$ reference entries in a single internal block, the probability of an internal block in memory being requested can be defined as follows.

$$\begin{aligned} P^{\textit{req}}_{\textit{int}}(C)=1-(1-P^{\textit{req}}_d(C))^{R_{\textit{int}}(C)} \end{aligned}$$

(28)

Moreover, we estimate the number of internal blocks pointing to the leaves, Inter(C), as follows.

$$\begin{aligned} \textit{Inter}(C)=\Bigg \lceil \frac{\lceil \frac{|C|}{R_d(C)}\rceil }{R_{\textit{int}}(C)}\Bigg \rceil \end{aligned}$$

(29)

Finally, we can state the cached internal blocks $M_{\textit{int}}(C)$ as follows.

$$\begin{aligned} M_{\textit{int}}(C)&=\textit{Inter}(C)*P^{\textit{req}}_{\textit{int}}(C) \end{aligned}$$

(30)

Cost calculation examples for MongoDB

We present the application of our cost model in MongoDB with two examples. First, a single collection is accessed through primary index with complete set of equations and calculations. Second, we present a real-world use case with only the initial calculation of the inputs because the complexity and the number of equations increase in such scenario.

1.1 Single collection with primary index

Let us take an scenario with Test 1 (13 million documents) and average document size of 40 bytes. First, we calculate the average number of documents and index entries in a block, together with the total number of data and index blocks as follows (by applying Eqs.1 and 2).

$$\begin{aligned} \begin{aligned}&|C| = 13*10^6 \quad {\textit{Bsize}}_{d}=32Kb \quad {\textit{Bsize}}_{i}=32Kb\\&{\textit{Size}}_d(C) = 40b \quad {\textit{Size}}_{i_{\_id}}(C) = 22b \quad F=0.7\\&R_d(C) = 0.7\cdot \Big \lfloor \frac{32768}{40}\Big \rfloor = 573\\&R_{{{\_id}}}(C) = 0.7\cdot \Big \lfloor \frac{32768}{22}\Big \rfloor = 1042\\&B_d(C) =\Big \lceil \frac{13*10^6}{ 573}\Big \rceil = 22676 \quad K= 10Mb \quad M=256Mb \\&B_{{{\_id}}}(C)=\Big \lceil \frac{13*10^6 * 1 }{1042}\Big \rceil = 12473 \quad u = 0.80\\ \end{aligned} \end{aligned}$$

Since we have only the primary index, $\textit{Rep}_{\textit{\_id}}=1$, $P(C,\textit{\_id}) = 0.5$, and $P(C)=0.5$. Now, applying Eqs. 7–13 together with Eqs. 4 and 19, we come up with the following set of equations.

$$\begin{aligned} \begin{aligned}&{Req}_{\textit{\_id}}(C) = |Q|\cdot 0.5\\&E_{{\_id}}(C) = 13*10^6*\bigg (1-\Big (\frac{13*10^6-1}{13*10^6}\Big )^{\textit{Req}_{\textit{\_id}}(C)}\bigg )\\&SF_{{\_id}}(C) = \frac{E_{\textit{\_id}}(C)}{13*10^6}= \bigg (1-\Big (\frac{13*10^6-1}{13*10^6}\Big )^{\textit{Req}_{\textit{\_id}}(C)}\bigg )\\&SF(C) = SF_{\textit{\_id}}(C) = \bigg (1-\Big (\frac{13*10^6-1}{13*10^6}\Big )^{\textit{Req}_{\textit{\_id}}(C)}\bigg )\\&P^{{req}}_d(C) = 1-(1-SF(C))^{573} \quad \\&P^{{req}}_{\textit{\_id}}(C) = 1-(1-SF_{\textit{\_id}}(C))^{1042}\\&M^{{sat}}_d(C) =22676*P^{\textit{req}}_d(C) \quad M^{{sat}}_{\textit{\_id}}(C) =12473*P^{\textit{req}}_{\textit{\_id}}(C)\\&\quad M^{{sat}}_d(C)*32768 +M^{\textit{sat}}_{\textit{\_id}}(C)* 32768 = ((0.8*256)-10)*1024^2 \end{aligned} \end{aligned}$$

By solving the above set of equations, we obtain the values for $|Q| = 4242.85$, $M^{\textit{sat}}_d(C) = 3038.98$, and $M^{\textit{sat}}_{\textit{\_id}}(C) =2847.82$. Using these values at the memory saturation point, we can come up with the following set of equations by applying Eqs.  21–27 together with Eqs. 4 and 19.

$$\begin{aligned} \begin{aligned}&P_d^{\textit{in}}(C) = P_d^{\textit{out}}(C) \quad P_{\textit{\_id}}^{\textit{in}}(C) = P_{\textit{\_id}}^{\textit{out}}(C) \\&{\textit{Shots}}_d^{\textit{in}}(C)=3038.98 \cdot \frac{M_d(C)}{22676} \\&{\textit{Shots}}_{\textit{\_id}}^{\textit{in}}(C)= 2847.82\cdot \frac{M_{\textit{\_id}}(C)}{24962} \\&E_d^{}(C)= M_d(C)-{\textit{Shots}}_d^{\textit{in}}(C) \\&E_{\textit{\_id}}^{}(C)= M_{\textit{\_id}}(C)-{\textit{Shots}}_{\textit{\_id}}^{\textit{in}}(C) \\&W_d(C) = \frac{M_d(C)\cdot 32768}{((0.8*256)-10)*1024^2} \\&W_{\_id}(C) = \frac{M_{\_id}(C)\cdot 32768}{((0.8*256)-10)*1024^2} \\&P_d^{\textit{out}}(C)=\frac{W_d(C) \cdot \frac{E_d^{}(C)}{M_d(C)}}{W_d(C) \cdot \frac{E_d^{}(C)}{M_d(C)} + W_{\textit{\_id}}(C) \cdot \frac{E_{\textit{\_id}}^{}(C)}{M_{\textit{\_id}}(C)}} \\&P_{\textit{\_id}}^{\textit{out}}(C)=\frac{W_{\textit{\_id}}(C) \cdot \frac{E_{\textit{\_id}}^{}(C)}{M_{\textit{\_id}}(C)}}{W_d(C) \cdot \frac{E_d^{}(C)}{M_d(C)} + W_{\textit{\_id}}(C) \cdot \frac{E_{\textit{\_id}}^{}(C)}{M_{\textit{\_id}}(C)}} \\&P_d^{\textit{in}}(C) = \frac{3038.98 \cdot (1-0.5)}{3038.98 \cdot (1-0.5)+2847.82 \cdot (1-0.5)} = 0.52\\&P_{\textit{\_id}}^{\textit{in}}(C) = \frac{2847.82 \cdot (1-0.5)}{3038.98 \cdot (1-0.5)+2847.82 \cdot (1-0.5)} = 0.48 \\&M_d(C)* 32768 +M_{\textit{\_id}}(C)\cdot 32768 = ((.8*256)-10)*1024^2 \end{aligned} \end{aligned}$$

By solving the above equations, we obtain $M_d(C) = 2847.82$ and $M_{\textit{\_id}}(C)= 3038.98$. By applying these values on Eqs. 3 and 4, we get a relative cost for a query through $\textit{\_id}$ as follows.

$$\begin{aligned} \begin{aligned}&P_d(C) = \frac{2847.82}{22676} = 0.12 \quad P_d(C) = \frac{3038.98}{12473} = 0.24 \\&\textit{Cost}_{\textit{Rand}} = 2 - (0.24+0.12) = 1.64 \end{aligned} \end{aligned}$$

1.2 Multiple collections

Let us take a use case of storing the data of authors and their books. Let’s assume that we chose to have a reference to the authors inside each of the books (as shown in Listing 2) out of the possible design choices. Moreover, let us also assume that each author has 5 books and each book has 3 authors on average.

In this scenario, we have two collections and three indexes (two primary on each of the collections and one secondary index on $A\_ID$ in Books). We calculate the number of documents/indexes in a block and the total number of blocks for each of these five B-trees as follows.

$$\begin{aligned} \begin{aligned}&|\textsf {Books}| = 4*10^6 \quad |\textsf {Authors}|= 2.5*10^6 \quad \textit{Bsize}_{d/i}=32Kb\\&\textit{Size}_d(\textsf {Books}) = 265b \quad \textit{Size}_d(\textsf {Authors}) = 150b \quad F=0.7\\&\textit{Size}_i(\textsf {Books}/\textsf {Authors}) = 22b \quad R_d(\textsf {Authors})= 152\quad \\&R_d(\textsf {Books})=89 \quad B_d(\textsf {Authors})=16448 \\&B_d(\textsf {Books}) =44944\\&R_{\textsf {\_id}}(\textsf {Books})= R_{\textsf {A\_{ID}}}(\textsf {Books})= R_{\textsf {\_id}}(\textsf {Authors}) =1042\\&B_{\textsf {\_id}}(\textsf {Books})=3843 \quad B_{\textsf {\_id}}(\textsf {Authors}) =2402\\&\textit{Mult}_{{\textsf {A\_ID}}}(\textsf {Books})=5 \quad B_{{\textsf {A\_ID}}}(\textsf {Books})=19213\\ \end{aligned} \end{aligned}$$

The following queries are executed with equal probability (0.25) on our documents store.

Q1:

Find the author name by _id

Q2:

Find the book name by _id

Q3:

Find all the book names with a given _id of an author

Q4:

Find all the author names with a given _id of a book

Since our cost model depends on the access probability on each of the B-tree structures, we calculate them as shown in Table 2. In Q3, we have single access to the secondary index on A_ID and five accesses to the data B-tree. Q4 involves two queries, first, one to retrieve a book through its _id and then on average, there would be 3 A_IDs which need to be retrieved as three independent requests through _id of the authors collection.

Table 2 Calculating the access probability of the B-trees

Now, we have the final input for our cost model, together with $\textit{Rep}_{\textsf {A\_ID}} (\textsf {Books})$ by applying Eq. 6, as follows:

$$\begin{aligned} \begin{aligned}&P(\textsf {Book}) = 0.389 \quad P(\textsf {Book},\textsf {\_id}) = 0.111 \quad \\&P(\textsf {Book},\textsf {A\_ID}) = 0.056 \\&P(\textsf {Author}) = 0.222 \quad P(\textsf {Author},\textsf {\_id}) = 0.222\\&\textit{Rep}_{\textsf {A\_ID}} (\textsf {Books})= \frac{5* 2.5*10^6 }{4*10^6 } = 3.125 \end{aligned} \end{aligned}$$

Now, we can apply Eqs. 7–13 together with Eqs. 4 and 19 on the inputs to obtain the values for memory distribution at the saturation point. Then, using these results on Eqs. 21–27 together with Eqs 4 and 19, we obtain the following final memory distribution. These calculations are similar to the example in Appendix B.1, but we omit listing them out due to their extensiveness.

$$\begin{aligned} \begin{aligned}&M_d(\textsf {Books})= 2532\quad M_{\textsf {\_id}}(\textsf {Books})= 638\quad \\&M_{\textsf {A\_ID}}(\textsf {Books})= 351\quad \\&M_d(\textsf {Authors}) = 1401\quad M_{\textsf {\_id}}(\textsf {Authors}) = 935\quad \end{aligned} \end{aligned}$$

Finally, we calculate the miss rates and the relative cost of each of the queries as follows.

$$\begin{aligned} \begin{aligned} P_d(\textsf {Books})&= \frac{2532}{44944} = 0.056\\ P_{\textsf {\_id}}(\textsf {Books})&= \frac{638}{3843} = 0.166\quad \\ P_{\textsf {A\_ID}}(\textsf {Books})&= \frac{351}{19213} = 0.018\\ P_d(\textsf {Authors})&= \frac{1401}{16448} = 0.08\quad \\ P_{\textsf {\_id}}(\textsf {Authors})&= \frac{935}{2402} = 0.38 \quad \\ \textit{Cost}(Q1)&= 2-(0.38+0.08) = 1.54\\ \textit{Cost}(Q2)&= 2-(0.166+ 0.056) = 1.778\\ \textit{Cost}(Q3)&= 1-0.018 + 5*(1-(0.056)) = 5.702\\ \textit{Cost}(Q4)&= 2-(0.166+ 0.056) + 3*(2-(0.38+0.08)) \\&= 6.398 \end{aligned} \end{aligned}$$