1 Introduction

The use of Web technologies is increasingly becoming a convenient and valuable asset for children’s education (Danby 2013; Knight 2014), because it facilitates learning (Chu 2012), it enhances the class environment and it introduces children, in the early stages of their lives, to today’s information society (Sadaf et al. 2012). However, as Danby (2013) described, incorporating technology into the traditional activities of early childhood education is not a trivial task. As defined in Rieh et al. (2016), searching as a learning tool is often explored from two different perspectives: “searching to learn” and “learning to search”. The former focuses on locating information in the educational setting as a result of an online search process in order to learn, the latter explores users’ search skills, i.e., their ability to know how to formulate queries that best capture their information needs and trigger the retrieval of relevant resources.

Young people today—who have been referred to as “the Google generation”—turn to search engines as their first “port of call for knowledge” (Rowlands et al. 2008). In fact, children use search engines on a daily basis to locate materials that can help them with different academic tasks, from finding information for a class to discovering the meaning of a word (Knight 2014). While the use of search engines for the enhancement of learning tasks is very common, they are not designed with children in mind, and thus a number of issues arise when used by this audience (Gossen et al. 2013).

There is a natural connection between conducting successful information-discovery tasks and the knowledge required to initiate such tasks. Unfortunately, numerous studies have shown a lack of explicit information literacy instruction, even though search engines are used daily by young users (Gossen et al. 2014; Kuhlthau et al. 2015; Rieh et al. 2016; Rowlands et al. 2008; Spink et al. 2010). Another important barrier is showcased by the fact that search engines are not always successful in understanding children’s information needs, particularly because children often express those needs in long natural language or with ambiguous queries (Bilal and Boehm 2013). Another prominent issue is evidenced by the results of the survey conducted by Bilal and Boehm (2013), which identifies that out of 300 retrieved results to satisfy the information needs of seventh graders, only one matched their reading level. This is a concern, as it might not even be possible for children to use resources that are too hard for them to understand because the readability levels of the resources do not match their respective reading comprehension abilities. Furthermore, given that children as web users, “differ widely in their reading proficiency and ability to understand vocabulary, depending on factors such as age, educational background, and topic interest or expertise” (Collins-Thompson et al. 2011), it is imperative to tailor the complexity of results to the specific needs of each child, and not just to generalize these needs based on a label such as age or grade. As reported by Lennon and Burdick (2004), a reader needs to be able to read and comprehend at least three quarters of a text, if learning is to occur as a result of reading. This ratio is meant to set a balance between what the reader understands and the existence of challenging passages that will lead to the improvement of reading comprehension skills (Lennon and Burdick 2004). Therefore, unless the retrieved resources match the reading skills of the user, reading for learning and learning based on the specific information seeking task cannot take place.

As reported by Hew and Brush (2007), using web technologies can help students get higher scores in standardized tests and promotes creativity when solving problems. It is important to note, however, that benefits of incorporating technologies in the classroom are not limited to students. As several educational studies suggest (Hew and Brush 2007; Kepple et al. 2015) teachers could greatly benefit from using Web 2.0 technologies in the classroom environment as those tools contain numerous pedagogical benefits. Even if some teachers use tools such as BiblioNasium.com or Instagrok.com to keep track of their students’ reading logs or locate materials related to the topics they are teaching, the use of information technologies is not widely extended among them (Kepple et al. 2015), a fact that might be due to the lack of breadth offered by these tools, i.e., they focus either on the reading ability progress or on serving materials for learning, but not both. A tool that would integrate both search and reading ability progress tracking could greatly benefit teachers, as they would be able go beyond readability leveled static books, and incorporate up-to-date online articles in the learning process of their students. In addition, they would also be able to actively monitor the learning process of tools such as search engines, providing students with personalized suggestions on how to improve their information discovery strategies.

The main contributions of this article are twofold: (1) an analysis of search environments in relation to meeting children’s (see Study 1 in Sect. 4.1) and teacher’s needs (see Study 2 in Sect. 4.2); and, (2) the presentation of YouUnderstood.Me (YUM) (see Sect. 3), our evolving research environment that focuses on facilitating search as learning from children’s and teachers’ perspectives.

More specifically, with regards to our first contribution, we present a qualitative and quantitative study oriented to analyze the issues both children and teachers face when performing information discovery tasks or locating learning materials. In order to accomplish so, we analyze the behavior of multiple search environments when using queries generated by children. Note that we focus on the K-9Footnote 1 audience comprised of 6–15 year-olds, since these ages refer to children from their initial search experiences through their “graduation” to adult search tools. We also explore tools teachers use in the process of finding adequate class materials that can be tailored to individual users. The results of this analysis allow us to identify and report challenges teachers are facing and further justify the need of a tool that that considers both teachers and students to foster learning.

As a means to frame the possibilities that current information retrieval techniques can provide in closing the loop between both stakeholders, we also include YUM among the tools examined in this study. YUM is the result of our ongoing efforts to develop a web search environment designed to help K-9 students find adequate online materials, while also considering teachers’ needs of tracking the progress of their students and finding adequate materials for preparing their classes. More importantly, as we discuss later on, YUM is meant to address not just one of the aforementioned search as learning perspectives, but both. YUM simultaneously enables searching-to-learn and learning-to-search by aiding children find resources that will help them learn and showcasing their corresponding reformulated queries–following the “showing results for...” feature offered by search engines, such as Google. Learning-to-search is of particular importance, given that by learning how to search as they search, children can ultimately take full advantage of rich search engine capabilities and find resources for both leisure- and education-related tasks.

The remainder of this paper is organized as follows. In Sect. 2, we discuss related work pertaining to search as learning as a new area of study, the need for personalization in children’s search tools, and various search tools designed for children and teachers. Since we include YUM as one of the tools evaluated in our analysis, we describe this tool in Sect. 3. In Sect. 4, we discuss the findings based on the study conducted on popular search tools for both children and teachers, including YUM. We also identify the need for an environment that eases search as learning by taking a dualistic approach that considers both teachers’ and students’ perspectives. Lastly, in Sect. 5, we detail lessons learned, identify problems to be considered in the future, and identify the need for further enhancements of YUM, our proposed environment.

2 Related work

In this section, we first discuss search as learning, as a relatively new field of study. Thereafter, we provide a brief overview of personalization on search environments, search tools used by children and teachers, as well as the importance of reading.

2.1 Search as learning

The information-seeking process and the use of search engines has been studied by the information retrieval community for over a decade. However, as reported by Hansen and Rieh (2016), “researchers have only recently started recognizing search systems as rich online spaces in which people can learn and discover new knowledge while interacting with online content”. This is evidenced by the publication of surveys such as the one compiled by Rieh et al. (2016), which discusses existing work in the area and enumerate possible future directions, as well as the organization of recent “Search as Learning” workshops (Freund et al. 2014; Gwizdka et al. 2016) and panel (Rieh et al. 2014) that aim to congregate researchers with varied expertise (e.g., information retrieval, human computer interaction, and education, to name a few), in order to understand challenges most commonly associated with this new area of study.

As discussed in the latest literature on the topic, optimizing results for educational goals (Syed and Collins-Thompson 2016), search literacy (Chu and Law 2008; Theng et al. 2016; Wilson et al. 2016), metacognition (Crescenzi 2016), and facilitating learning from web resources (Koesten et al. 2016) are a few of the topics of interest for this emerging search as learning community. We focus on searching-to-learn while also supporting learning-to-search for our target user group of children in grades K-9.

2.2 Personalization of children’s search experience

Search engines have become indispensable for all types of users, from novice to experts and from children to scholars, to perform information-related tasks (Huurdeman and Kamps 2015). A number of studies have targeted the issue of search personalization (Collins-Thompson et al. 2011; Eickhoff et al. 2013; Jatowt et al. 2012; Wang et al. 2013). The authors in Eickhoff et al. (2013) and Wang et al. (2013) argue for the need to personalize search results to satisfy diverse users’ needs and preferences. However, while their personalization strategy focuses on parameters such as authority of web pages or atypical search sessions, respectively, we focus on parameters that can aid the learning process, particularly the readability levels of the retrieved results. Personalization based on readability has also been explored (Collins-Thompson et al. 2011; Jatowt et al. 2012). Collins-Thompson et al. (2011) demonstrate, based on the results of an extensive query-log analysis, that readability is a valuable signal for the relevance of retrieved resources. Jatowt et al. (2012) highlight the need for suitable readability levels on resources retrieved as a result of queries formulated by non-experts on complex topics. We concur that the readability of personalized web searches is important, which is why YUM is designed to present users with resources they can read and understand. Our efforts to create a search environment that addresses issues K-9 students encounter while conducting information seeking tasks are further encouraged by the conclusions reported by Huurdeman and Kamps (2015), who argue in favor of the need to connect literacy to the information seeking process using search engines.

2.3 Search environments: tools and engines for children and teachers

Related to search environments specifically designed for children, the author in Gossen (2016) presents an overview of a number of search user interfaces and search tools that aims to enhance the search process. While the book enumerates new search environments, we instead design YUM so that it incorporates modular capabilities that can be applied to improve the functionality of popular search engines preferred by children (Bilal and Boehm 2013), in terms of the needs and expectations of children. The authors’ latest research work, detailed in Gossen et al. (2015), focuses instead on a search user interface that takes the user’s age as a parameter for adaptation. Similarly, YUM focuses on adapting the search environment to the needs of children, from a reading comprehension standpoint—not just age—in order to facilitate the searching-to-learn task. The closest environment to YUM is the one described in Usta et al. (2014). However, the application proposed by Usta et al. (2014) only offers grade level filtering, which is a constraint, since students’ reading abilities differ within the same class and may improve over time at different paces (Shin et al. 2013). In addition, the environment introduced in Usta et al. (2014) is not based on known search engines, which children tend to favor (Bilal and Boehm 2013).

A number of search engines have been developed to aid children, including SearchyPants.com (no longer active), KidRex.org, KidzSearch.com, Kidtopia.info, Kiddle.co, and Mymumka.com. Yet, they are not optimal to conduct information discovery tasks for learning purposes as discussed in Gossen et al. (2013) and Sect. 4. Online search tools have also been developed with teachers in mind. Scholastic.com, Pbslearningmedia.org, Readingglue.com, Instagrok.com, or BiblioNasium.com are just a few examples of tools that teachers can use to locate materials for their classes or keep track of their students’ reading habits. Unfortunately, these tools either do not filter by readability levels, or, if they do, they are only limited to books, as opposed to more general reading materials, such as websites. Furthermore, the tracking of students’ reading abilities is based on reading logs, as opposed to on-the-fly information related to information discovery tasks performed daily, which is the strategy adopted by YUM that models tracking based on students’ feedback.

To the best of our knowledge, YUM is the only education-oriented environment that considers readability levels as well as queries that potentially lead to the retrieval of child-targeted resources to aid K-9 students in completing successful information seeking tasks.

3 YouUnderstood.Me (YUM)

In this section, we briefly describe YUM’s current design, which is the result of our ongoing research efforts to address some of the issues students (and teachers) face searching-for-learning and learning-to-search. These issues include: identifying a child’s search intent, suggesting queries, retrieving resources filtered by readability, and tracking children’s search to infer reading level. YUM, which in its initial stages was first introduced in Azpiazu et al. (2016), is also one of the tools examine in our study in Sect. 4.

YUM is an evolving online environment built around a search engine,Footnote 2 which aims to make the search process valuable for children (see Fig. 1). Opposed to similar environments (Usta et al. 2014), YUM is not meant to be treated as a new, child-oriented search engine, since studies (Bilal and Boehm 2013) show that children prefer popular search engines. Instead, YUM acts as an intermediate layer between the child and an existing search engine, to facilitate the interaction between them. In doing so, YUM puts into practice strategies oriented to address issues children face when using popular search engines as well as strategies that can enhance the search experience to foster learning. Moreover, YUM builds an important bridge and establishes a direct relationship between teachers and students, where teachers can follow the progress of students’ reading abilities and adapt and refine individual and class-level learning plans based on that information.

Fig. 1
figure 1

An overview of YUM and its major goals

Full size image

3.1 Identifying a Child’s Search Intent

To facilitate query formulation and ease the search process that leads to suitable learning resources, YUM leverages our previous research work, QuIK (Dragovic et al. 2016), a search intent module designed for children. QuIK’s goals are to remove noisy elements usually included in a query that can hinder the performance of a search engine, as well as to highlight information that is valuable from a child perspective, so that the search engine can retrieve relevant results. In accomplishing these goals, QuIK addresses common patterns in each query Q written by a child including, but not limited to, diminutives, emphasis, trendy terms used by children, or children specific misspellings. Thereafter, it transforms Q into a new keyword query that captures the information expressed by the child in Q in a way that can be easier for search engines to comprehend. Similarly to Google’s “Did you mean” or “Including results for...” features, and to explicitly encourage learning-to-search, the updated query generated using QuIK is displayed along with the set of results retrieved in response to the query.

As illustrated in Fig. 2, QuIK follows a number of steps to transform the query Sven the amaaazing raindeer of Frozen into a three-keyword query that represents the most likely intent of the query by selecting the most relevant and distinguishing terms from a child’s perspective.

Fig. 2
figure 2

Step-by-step transformations applied by QuIK to the query “Sven the amaaazing raindeer of Frozen”

Full size image

3.2 Suggesting Queries

To further ease query formulation, YUM includes a query suggestion strategy that generates diverse kid-oriented query suggestions. By offering these suggestions, the child can select the query that better fits his needs, even if he has trouble articulating such a need. In order to accomplish this functionality, YUM takes advantage of our previously-developed ReQuIK (Dragovic et al. 2017), a query recommender tailored to children.

To create a set of possible candidate suggestions, ReQuIK uses Ubersuggest.io, a popular online tool for generating alternative keywords given an initial query Q. As previously mentioned, children may struggle with formulating queries that capture the information they meant to search for in a succinct manner (Bilal and Ellis 2011). For this reason, if Q includes more than three query terms, then ReQuIK starts the suggestion process from the most representative keywords in Q, as determined by using QuIK.

The thousands of alternative keyword suggestions that are generated are then ranked by ReQuIK using a multi-criteria strategy that examines traits commonly associated with children. ReQuIK’s ranking strategy aims to prioritize suggested queries that are diverse, associated with children topics, and lead to the retrieval of resources with levels of readability matching those of the K-9 audience. To accomplish this task, ReQuIK depends upon:

  • Categorized resources extracted from popular websites, such as CommonSenseMedia.org. This information is used for determining the degree of child-friendliness of terms in a query, e.g. a term can correspond to a book character intended for child audiences.

  • Kid-friendly corpora, extracted from online resources, including SpaghettiBookClub.org. These extensive collections of documents targeting topics of interest to children is employed to detect commonly-used kid-related phrases.

  • Readability formulas, including Flesch (1948) and Spache (1953), to estimate the degree of text difficulty of the resources potentially retrieved by each candidate query.

3.3 Retrieving resources filtered by readability

Children are not properly trained to identify high-quality, suitable resources from among those retrieved by their favored search engines (Bilal and Ellis 2011), nor their direct connection with the information needs meant to be expressed in their queries. In fact, they tend to systematically go through retrieved resources and rarely judge retrieved information sources (Graham and Metaxas 2003). To explicitly address these constraints, YUM takes advantage of Google Safe Search to leverage the benefits of Google-powered searches, i.e., access to rich and varied resources in an efficient and effective manner. More importantly, YUM explicitly shows its users the “updated” query (if applicable) employed to initiate the search process. In doing so, YUM adopts a non-intrusive strategy that encourages learning-to-search while searching, which emulates the “showing results for...” feature offered by popular search engines, such as Google. This design strategy is meant to emulate the “learning by repetition” process (Cole 2008; PLB 2016) where students eventually “no longer need to consciously think about their participation in the skill-building activity.”

In order to retrieve documents that go beyond adequacy and also match the reading ability of the children, YUM incorporates a filtering strategy based on readability levels. This strategy ensures that the retrieved documents match, to a degree, the reading ability of each individual user. YUM allows users to go through a one-time process where they can select their grade level, which is initially used as a target to eliminate resources that are not within half a grade level above or below the grade of the corresponding student. For estimating the readability of retrieved resources, YUM uses the Flesch-Kincaid readability formula (Flesch 1948), given that it is considered a standard by educators and institutions for measuring readability (Wall and Pentz 2016; West 2001; Wu et al. 2016).

3.4 Tracking children’s search to infer reading level

K-9 students have diverse reading abilities, which progressively improve over time but often differ within the same grade or class (Bowey et al. 1992; Shin et al. 2013). Consequently, a one-size-fits-all strategy is not applicable for conducting successful information-seeking tasks that lead to the retrieval of resources individual users can understand. YUM employs an adaptive strategy based on explicit feedback that users can provide by specifying whether the resources retrieved were “Too Easy”, “OK”, or “Too complex” for them. Children might not be experts in determining the readability of a document, however, YUM takes advantage of their perception over the multiple documents they have read, to obtain estimates about their reading levels. We are aware that children may decide against providing feedback about the degree of difficulty of the resources they access every time they perform a search. Consequently, unless indicated otherwise, YUM treats as “adequate” the resources that users access, even if they did not explicitly indicate the resource as readable.

We treat the problem of predicting the current reading level of users as a constraint satisfaction problem, where each feedback—implicit or explicit—provided by a student generates a constraint that needs to be satisfied according to the reading abilities of the student. For example, a student s giving a feedback of “Too complex” to a document of readability level 5 would generate the constraint \(r_s < 5\), stating that the reading level \(r_s\) of s is lower than 5. As shown in Eq. 1, the predicted reading level for s is the one that maximizes the amount of constraints that are satisfied.

$$r_s = {\mathop{\hbox{arg\,max}}\limits_{r}} \sum_{c_{i} \in C} {\left\{ \begin{array}{ll}f(c_{i}) &{\quad} if \; r \; satisfies \; c_{i}\\ 0 &{\quad} otherwise\end{array}\right.}$$
(1)

where \(r \in R=\{0,0.5,\ldots ,8.5,9\}\) represents every possible reading level value for the student and C is the set of constraints created based on the feedback provided on retrieved resources by s.

As reported in Collins-Thompson et al. (2011), a user’s reading proficiency needs to be estimated based on both current and past searches. Thus, Eq. 1 considers the time stamps of the created constraints, favoring those created more recently and discarding the ones created outside of the current academic year. In order to accomplish this, \(f(c_i)\) is a function that starts at value 9 for a new constraint \(c_i\) and decreases by 1 for each month since the corresponding feedback was provided until 0. We selected 9 as the number of months to consider as this represents the average length of an academic year. Initially, YUM defines two base constraints that represent one grade of deviation from the current readability of the student: \(r_s < p_s+0.5\) and \(r_s > p_s -0.5\) where \(p_s\) represents the prior readability of student s based either on the grade level selected the first time YUM is used or the \(r_s\) value for the previous academic year. These constraints give YUM a starting level, that will be adjusted as the student uses the environment.

3.5 YUM for teachers

Exploratory tasks can get hindered by issues that arise when children interact with popular search engines, both when formulating queries and when trying to understand retrieved results. YUM can help teachers overcome these issues so that teachers can maintain their focus on the higher-level discussion, rather than the manner in which students should interact with the search engine.

Furthermore, in order to better inform the teacher about the improvements achieved by his students, YUM can also serve as a monitoring tool that makes it possible for teachers to check students’ current reading level, based on the resources they have retrieved and their provided feedback in terms of complexity. This allows the teacher to better prepare for a class, as he is able to know the current state of the class in terms of reading level based on the readability of the documents the students have been reading when searching. In doing so, teachers can discover issues that affect the students individually or as a group.

We believe that YUM can facilitate learning when children use it for their information discovery assignments, as well as help teachers within the classroom environment by addressing the challenge of seamlessly integrating technology to perform everyday classroom activities (Danby 2013; Knight and Mercer 2015).

4 Studies of search as learning tools for children and teachers

In this section, we discuss the studies we conducted, namely: Study 1, the analysis of tools that can be used by children to conduct information discovery tasks (see Sect. 4.1), and Study 2 the analysis of tools employed by teachers in educational environments (see Sect. 4.2). Each study takes an important, yet different perspective, that of the child and that of the adult/teacher. The methodology for each study is described in their respective sections and are followed by a discussion of the qualitative and quantitative aspects that are relevant to each perspective. A list of the tools and corresponding URLs considered in our initial evaluation are shown in Table 1.

Table 1 Sampled tools used by children (for search and learning) and their teachers (for search and tracking student learning)
Full size table

4.1 Study 1: search tools for children

In this initial assessment we expand on the analysis framework presented in Gossen et al. (2013), adding several search engines as well as analyzed factors. For doing so, we examined a number of online search tools oriented to children listed in Table 2, including Kiddle.co, KidRex.org, SafeSearchKids.com, Gogooligans.comFootnote 3 and YUM. We also include Google in our study, since research has demonstrated that children tend to favor it over other search environments (Bilal and Boehm 2013).

Table 2 Overview of sampled search tools for children (Color table online)
Full size table

4.1.1 Method

There is a lack of benchmarks available for evaluating search-related tools focused on young users. Existing empirical studies on this topic are generally based on a limited number of participants (Bilal and Kirby 2002; Kammerer and Bohnacker 2012), due to the difficulty and time required to conduct experiments involving children. Alternatively, other studies (Duarte Torres et al. 2010; Duarte Torres and Weber 2011) are based on the existing AOL query log. Unfortunately, queries in this log are outdated and, as reported in Gossen (2016), there is a chance that queries identified as child queries, were mistakenly treated as such.

To address the aforementioned limitations, we collected our own sample of queries written by children. This sample includes 300 unique queries written by 50 children between the ages of 6 and 15. To elicit these queries, we asked various K-9 teachers in the Idaho (USA) area to present to their students an information discovery task for which the students had to create queries. The domain of the task was open, however, most of the children looked for information about films and animals, generating queries such as “When is finding Dory coming out?” and “How many cheetahs are in the world?”.

We submitted the newly-collected queries discussed in this section to each of the aforementioned search environments and examined their respective retrieved resources as well as the challenges children need to overcome when using each search tool. We discuss below the detailed findings pertaining to each of the dimensions considered for our assessment. These are: difficulty to retrieve adequate/appropriate resources, readability, query suggestions, and general user experience. An overview of these initial findings can be found in Table 2.

4.1.2 Results and analysis

Difficulty to retrieve adequate/appropriate resources One of the first issues children face when using a search engine is creating adequate queries that will eventually lead them to the information they are looking for Rieh et al. (2016). Children tend to write natural language queries, instead of the short, keyword-based queries search engines usually expect (Rieh et al. 2016). On average, the length of a query written by a child is 3.8 words (Duarte Torres et al. 2010), while 2.3 words (Lau and Horvitz 1999) is the average length for queries written by more mature audiences. In addition, children also tend to misspell words, but not necessarily in the same fashion that regular users do. For example, children commonly repeat letters in a word to emphasize it, such as in “faaaaast”, which can cause search engines to misunderstand the intended meaning of the word. Furthermore, terms pertaining to children vocabulary, such as the names of characters of children movies, can also be considered as misspellings by traditional spell-checkers. As an example, the query “Sven movie”, which refers to Frozen, a Disney movie, is automatically converted by popular search engines to “seven movie”, which leads to the retrieval of resources related to a movie not recommended for children (see Fig. 3).

Fig. 3
figure 3

Search results retrieved by Google for the query “Sven movie”

Full size image

Based on our assessment using children queries, we observed that for 21% of the queries, the child-oriented search tools considered in this analysis did not retrieve any result, or the results that were retrieved did not correspond with what the child would expect. We observed that Cyber Sleuth Kids was among the search tools that most struggle in terms of parsing and processing queries when they were written in a natural language or question form. In comparison, YUM failed to retrieve appropriate results for 12% of the queries. A more in-depth manual analysis lead us to discover that the reason behind difficulty on dealing with children queries mostly laid on the long length and the natural language structure of those queries.

The lack of retrieval of resources that children can comprehend is illustrated in Fig. 4. Each of the search tools considered—Google, which is favored by children, as well as Kidtopia and Kiddle, which are two popular children search tools—retrieve resources that are beyond the level of reading comprehension for young children. As a result, resources that are relevant with respect to the query “seven” are treated as non-relevant by the child who initiated the information discovery process.

Fig. 4
figure 4

Top-resources retrieved for the query “Seven” by: Google.com (top left), Kidtopia.com (top right), Kiddle.co (bottom left), and KidzSearch (bottom right)

Full size image

Another example that illustrates the difficulty experienced by existing search tools in retrieving adequate resources, consider the query “lollipop”. Submitting this query to Google generates a result set that includes resources that refer to the Android Operating System rather than resources about candies or songs, which is what a child would expect.

Readability Searching-to-learn cannot occur by simply excluding offensive content from the result set generated as a result of a child query: even when the search engine has understood the intent of a child query and retrieves results that match the information needs expressed by users, the suitability of retrieved resources is still not assured. K-9 students find difficult to understand documents containing complex or technical vocabulary. For example, in the case where a child is looking for information about chemistry, retrieving a scientific publication would not be adequate, while retrieving information from an elementary chemistry book would. If the retrieved documents are too complex, children may not succeed in completing their information discovery tasks. Therefore, the readability level of resources retrieved in response to a child query is also a relevant aspect to explore to quantify the success of a search from a reading for learning perspective. We computed the average readability level of the top-N results retrieved in response to children queries. Given that “children are known to systematically go through retrieved resources and rarely judge retrieved information sources” (Rieh et al. 2016), we computed the readability scores reported in Table 2 based on the top-3 documents retrieved in response to each query. For measuring the readability level of the retrieved resources, we selected the Flesch formula (1948), as it is considered a standard nationwide. Note that we used the grade level version of the Flesch formula (shown in Eq. 2), since it estimates text difficulty on ranges that match school grade levels, i.e., K-16, as opposed to the traditional reading ease version of the formula, which estimates text difficulty based on a 0-100 scale.

$$FK = 0.39 \times \left( \frac{Total\,Words}{Total\,Sentences}\right) + 11.8 \times \left( \frac{Total\,Syllables}{Total\,Words}\right) - 15.59$$
(2)

where Total Words refers to the total number of words in the text, Total Sentences denotes the number of sentences in the text, and Total Syllables captures the total number of syllables in the text.

For unbiased comparison purposes, we also used the popular SMOG (Mc 1969) formula, computed as shown below:

$$SMOG = 1.0430 \times \root \of {\frac{30 \times Total\,Complex\,Terms}{Total\,Sentences}} +3.1291$$
(3)

where Total Complex Terms refers to the number of terms that contain three or more syllables and Total Sentences denotes the number of sentences in the text.

Recall that YUM filters retrieved resources that have a complexity level within +/− 0.5 deviation from the reading level of each user, assuring that retrieved resources can be comprehended by its users. Therefore, we only computed the average readability levels of resources retrieved in response to queries posted on the other child-oriented search engines considered in this analysis. To provide context to the readability of resources filtered by YUM, however, we ran a simple experiment where we filtered resources with readability levels (estimated using the Flesch formula) above 9 retrieved in response to each the 300 test child queries. Thereafter, we computed the average readability level of top-3 resources using the SMOG formula (as it was done for the remaining search tools). As shown in Table 2, the average readability level of resources retrieved by YUM using SMOG is still close to level 9.

The majority of the environments included in our study target children, which is why ideally they should retrieve resources with a reading level within the grade levels of K-9. As shown in Table 2, however, the readability levels of retrieved resources were on average above a readability level of the 10th grade, and even one of the search engines (Safe Search Kids) retrieved resources that averaged a readability level of above the 15th grade (college level reading levels).

Query suggestions Even if a search intent module or “Did you mean”-style feature used on some search environments can help identify the most likely intent for each query, sometimes that is not enough. On the one hand, a query can be too short and therefore not contain enough information for a successful search. On the other hand, a query may have multiple meanings depending on the interests of the child. In either case, it is only the child himself who knows what he is really looking for. For this reason, it is important to provide the user with recommendations that can satisfy his needs.

As captured in Table 2, most of the search engines we analyzed showed no or poor support for helping children improve their queries. Google and Gogooligans suggested query reformulations while typing. However, these suggestions were not tailored to children or they did not go beyond dictionary based auto completion. For example, when “Sven” (the name of a character from the Disney movie “Frozen”) was typed, “Seven” (a movie not rated for children) was given as a suggestion in most of the search engines, which does not capture the intended meaning of the query considering that it was written by a child.

General user experience The quality of a search engine is not only determined by its retrieved results, the general search environment is also important (Gossen et al. 2013). We observed that the presence of ads was recurrent among the search engines considered in this study. These ads were usually indistinguishable from relevant retrieved resources, which can be confusing, and more importantly, sometimes these ads offered products that are unsuitable for children as they are not filtered with children in mind. For example, we found ads that referred to drug rehabilitation programs or anti-aging products among results retrieved by Safe Search Kids in response to queries such as “frozen characters.” Moreover, in some cases, the search results would blend in with the ads (as is the case of Kiddle). This makes the process of distinguishing ads from relevant resources even more difficult to children.

We also noticed that platform adaptability was an issue for some of the search engines, since they showed poor support for small screens, such as phones or tablets, making it hard for a child to use the same system in all platforms. This is a significant drawback, given that 71% of children frequently access the internet through a tablet (Ofcom 2014).

4.1.3 Discussion

Child-oriented search engines are abundant on the internet. However, most of the currently available tools only consist of child-friendly adaptations of Google safe search. Our analysis reveals that implementing new technologies can make a difference in the general usability of children’s search environments (Rutter et al. 2015). Examples of these new technologies include search intent and query suggestion algorithms tailored for children, and considering filtering retrieved resources based on readability levels. Perhaps even more importantly, these features can transform search environments into even more powerful tools that can foster search as learning, while learning to search. In addition to the aforementioned features, other important aspects impacting children’s usage include the presence or not of ads and that children may prefer to use a mobile-friendly adult-like tool such as Google (Rowlands et al. 2008), rather than a children-oriented tool that is only visible in desktop computers since they frequently use mobile devices (Kabali et al. 2015; Rideout and Saphir 2013).

4.2 Study 2: online tools used by teachers

In the previous section, we examined search tools children can use to perform information discovery tasks. We specifically focused on their ability or potential ability to foster learning as a result of a search, while learning how to search for suitable resources. In this section, we instead explore a number of tools teachers can use in the classroom setting, not only to complement their instruction, but also to oversee student learning from multiple perspectives. We include below a discussion of our findings, a summary of which can be found in Table 3.

Table 3 Overview of tools teachers can use to locate materials or track students’ progress (Color table online)
Full size table

Teachers can also benefit from using information technologies within the class environment. Web 2.0 technologies are increasingly prevalent in the classroom environment, both because of their possibilities to enhance teaching and learning and because of the need of current students to develop 21st century skills that will permit them to succeed in today’s society (Greenhill 2010; Sadaf et al. 2012; Shihab 2009). Many students already use Web 2.0 technologies on a daily basis, mostly for social networking purposes (Sadaf et al. 2012). Using Web 2.0 technologies to learn can leverage some of their current or existing expertise as well as allow children to use these technologies in different—perhaps more educational—ways (Sadaf et al. 2012). It is important to note that the adoption of these technologies is not limited to children. Recent surveys (Sadaf et al. 2012) suggest that teachers are willing to use Web 2.0 technologies in classroom environments, specially as a result of the engagement they produce among students and content, the enhancement in learning, and the access to learning even outside classroom.

Work setting standards have changed from a vertical structure, where only the top individuals of the pyramid had to think critically and the lower echelon simply followed directions, to a horizontal structure, where each individual is expected to collaborate with others and solve important problems using identification, searching, synthesizing, and communication skills (Leu et al. 2013). With this change, education plans oriented to meet these new requirements for the current industry demands have been developed. The Common Core State Standards Initiative (CCSS), is one attempt to meet this challenge. CCSS requires educators to emphasize higher-level thinking in the reading curriculum and focus on the acquisition of skills such as research and comprehension using digital tools, including search engines (Leu et al. 2013).

Furthermore, studies showcase the benefit of in-class exercises such as exploratory talks, where students are asked to solve a problem in groups discussing information found on resources obtained using a search engine (Knight and Mercer 2015). Educational researchers suggest that teachers must play the role of mediators in such tasks (Ullrich et al. 2008) by giving suggestions to improve exploration and discussing results obtained with the students. However, teachers might not be able to propose such tasks to their students and lead such discussions, if students have problems using search engines. This type of exercises are not feasible if children struggle to find the right queries or are not able to understand the retrieved documents, and thus constantly interrupt the discussion in order to address these problems. As mentioned by Kuiper et al. (2005) educators should be able to distinguish the learning benefits of teaching how to use information technologies and using information for learning.

4.2.1 Method

Again, here there is a lack of benchmarks available for evaluating search-related tools focused on young users from the teachers perspective. Our method for this study started by identifying tools designed for teachers based on internet searches and tools identified in related literature (Bennett and Lockyer 2004; Gossen 2016; Hammonds et al. 2013; Valenza et al. 2014). These tools were then reviewed by our research team and major themes were identified for initial analysis of each of these tools. This study provides an initial overview of these tools in the dimensions of: free to join, web and mobile accessibility, presence or non-presence of ads, the target age group of the children as identified by the tool authors/creators, and its general purpose.

4.2.2 Results and analysis

Purpose An initial look into tools teachers can use in the classroom revealed that two of the main purposes of these tools are to locate material for classes and keep track of students’ reading habits. While Instagrok and PBS Learning allow teachers to employ simple keywords to locate resources, filtering only occurs by grade. However, as previously stated, not all students in a class can comprehend materials at the same reading level. Scholastic, on the other hand, allows filtering by readability, but only books are available, which is a constraint, since books are not the only type of materials used in classroom activities. With regards to closing the loop between the teacher and the students a progress tracking component is necessary, indeed, sites like BiblioNasium and Reading Rewards make it possible for teachers (and parents) to keep track of the books their students are reading, and thus infer their reading abilities. However, these tools require manual entry and additional overhead of teachers, parents, and students to coordinate and enter this information. YUM, on the other hand, is an example of a tool that can allow teachers to locate online resources, beyond books, filtered by reading levels of individual students (or groups of students) and also to monitor student progress, in terms of their individual reading abilities, which is updated based on their daily interactions with the system.

Advertisements We have observed that tools like Kids Konnect and Scholastic include advertisements in their sites. The ads, however, do not pertain to educational content and make the navigation between pages difficult.

Target audience While a number of tools, including BookAdventure, Scholastic, and We video, support teachers from Kindergarten to the 12th grade, tools like BiblioNasium and Instagrok focus only on a limited range of grades, which is a constraint for teachers, as they may need to use different tools for different classes. YUM, in its current implementation, targets K-9 grades. However, due to its modularity and domain independent design, it can be extended to aid teachers in K-12 grades.

Cost A number of tools can be accessed free and teachers only need to pay if they want to acquire (i.e., purchase) a particular resource. Sites like Mindomo, KidsKonnect, and Reading Rewards, on the other hand, impose a subscription fee.

General observations We have observed that only about 50% of the tools can be accessed both through web and mobile apps. The remaining ones (including Scholastic and Reading Glue), are only web accessible.

4.2.3 Discussion

Most of the tools included in our analysis can be framed into two categories, whether they serve as a mean for retrieving educational resources or as a mean to track students progress or grades. However, none of the tools analyzed is capable of retrieving documents that are adequate to the current educational state of the student. In addition, for a teacher it is important to make a clear difference between teaching students to use information technologies and actually using the information to learn (Kuiper et al. 2005). This distinction can be achieved by considering information retrieval techniques and combining on the same environment learning to search, while searching for learning materials, which, as we previously illustrated, can be accomplished by tools such as YUM.

5 Lessons learned and future work

In a world flooded with information, combining searching-to-learn and learning-to-search is key in order to help children acquire the skills they need to become effective searchers and take advantage of such information. To address this task, we developed YouUnderstood.Me (YUM), the online environment we presented in this manuscript. YUM addresses issues children face when using popular search engines to conduct information seeking tasks. It can facilitate the learning that can occur while reading resources that are retrieved as a result of a child-initiated search. As part of our ongoing research efforts, we leverage the use of popular search engines, search intent and query suggestion modules we have developed, a readability-based filtering strategy and a novel tracking strategy, to enhance the searching-to-learn tasks conducted online and informing teachers of the progress of their students, in terms of reading and comprehension. We also leverage new functionalities, such as displaying the rephrased queries submitted to a search engine, emulating the popular “Did you mean” feature offered by popular search tools, to facilitate learning-to-search.

We conducted an initial assessment using queries written by K-9 children and demonstrated the need for environments such as YUM. Along the way, we also considered teachers as possible stakeholders of YUM, given their needs to (1) locate materials to enrich their classes that students can read and understand, and (2) keep track of the reading abilities of their students. We examined YUM from a teacher perspective, along with other tools teachers can turn to. We observed that offering a bridge to connect students and their teachers, so that teachers can focus on nurturing student learning through information discovery tasks, is a must.

We plan to extend YUM by implementing a number of enhancements. We are aware that the Flesch-Kincaid formula currently used in YUM may not be precise enough. Therefore, we plan to explore other models with the possibility of creating our own. Advantages of a new model would include allowing us to go beyond counting terms and syllables, and instead add information including web-page specific metadata as well as in-depth language information, such as syntax and semantics. An exploration of different filtering strategies will also be conducted based on web page authority and the level of maturity of the content retrieved, so that retrieved resources are more suitable to children. We are also aware that children may not provide explicit feedback for all the resources they read. Therefore, we also plan to explore ways of obtaining implicit feedback, such as analyzing the time spent reading the resources.

We believe an online assessment which complements the offline evaluation discussed on this manuscript is imperative. For this reason, future directions of our research work involve conducting a more in-depth study to better understand, quantify, and showcase the correlation between learning and information discovery tasks conducted using enhanced web search environments. Since the developmental stages and information needs of K-9 children are broad, we will conduct these studies based on more specific age ranges, such as 6–8 and 9–12.

In the future, we envision YUM will be an environment a child could use independently of the information discovery task pursued. For doing so, we plan to extend the range of documents YUM can retrieve, including searching local catalogs at school libraries. We also aim to add support for multiple languages so that children that know different languages can use their YUM account both at home and at school. Finally, we would also like conduct online experiments, which could lead to incorporate new ways of collaborative searching between students and teachers, in order to further enhance learning while fulfilling searching tasks.