Intelligent Food Information Provision to Consumers in an Internet of Food Era

Abstract

Food information is a crucial tool for facilitating consumers in decision-making activities related to their consumption process. Recent advances in “Internet of food” technologies (such as food sensors, cloud computing, food data analysis, and mobile app technologies) makes possible to conceive new consumer information platforms. The rationale is to empower consumers by letting them get more relevant food information than they usually obtain through on-product labeling, mass media or other traditional channels. In this paper, we envisage a new generation of food information provision services, called intelligent food services (IFSs), which would be responsive to consumer’s expectations and information needs. We outline IFS structure and main features as well as constitutive elements of user-IFS interaction context. Particularly, we focus on food-in-context awareness capability and we discuss its influence on consumer and IFS behaviors.

Keywords

1 Introduction

Food consumer behavior has changed substantially over the past decades. The contemporary food consumer, particularly the European consumer, has become more critical and outspoken, as he/she may be concerned with many heterogeneous attributes of food products. Along with, he/she demands and expects to get more relevant, accurate and reliable information necessary for his/her food decision making or learning.

These information needs may regard many heterogeneous elements pertaining to wellbeing, health, convenience, environmental, social, cultural, ethical and political concerns as consumers’ choices are driven by many different motivations related to enjoyment (e.g. sensory quality, conviviality), health (e.g. diabetes, allergies, obesity), religion (e.g. halal, kosher), environment safeguarding or other special lifestyle (e.g. organic, vegetarian, no genetically modified foods). Recent research has shown that: (a) consumers who are environmental conscious and ethically concerned, give high value to food information relating to environmental and social issues [1]; (b) consumers who want to make healthful food choices are more interested in information about food nutrition attributes, food origin and technological conditions under which the food has been produced and processed [2]; (c) as a reaction to frequent insertion of fraudulent, adulterated, misrepresented, and mislabeled food products into retail-to-consumer portion of food chains, many consumers strive for more and reliable information that guarantees the authenticity of their foods [3].

The challenge of food information management is to deliver the information that enables consumers to make appropriate choices according to their own individual objectives as they are involved in food consumption activities (planning, purchasing, preparation/cooking, eating and disposal/tidy-up). An effective and efficient food information service should deliver the right information to the right people, at the right time, and at the right place.

This challenge is particularly complex as it could require to meet a broad range of alternative information needs that are largely driven by the context in which food consumers make their decisions. This context may include many different entities (such as consumer, food, artifacts, and environment) and their attributes (such as identity, location, time, and activity currently involved in) depending on the situation they are relevant to.

Services that deliver massive and generic food information have a limited chance of success since they carry high search costs for the consumer and can hamper the ability to detect and correctly identify information that is needed in his/her decision or learning process [4, 5]. Employment of conventional channels (e.g., store shelf and package labels) allows neither to retrieve actionable information from sources that are appropriate for the current consumer’s situation, nor, more in general, to provide personalized food information.

Today’s consumers have access to a wealth of mobile app-based services that provide them with food information (food traceability, nutrition advices, recipes and purchasing support). However, the information provision of these services is neither contextualized nor consumer-centered. Other than the capacity to adapt to consumers’ mobility, it does not seem much different from usual food information provision through conventional media or Web sites [6, 7].

A promising channel could be represented by a new generation of food information services which would empower consumers to have control over their own food as well as would be responsive to their contextualized expectations and information needs. These services could be based on the exploitation of recent advances of “Internet of food” technologies (such as food sensors, cloud computing, food data analysis, and mobile app technologies). The entry in the consumer market of some smart food applications and devices are clear signals in this direction: automated “food scanner” or “food sniffer” for the biomolecular food analysis, mobile apps that use camera photos for the quantification of food volume/weight/portion size or for the assessment of food quality, smart digital kitchen scale apps, intelligent package for food traceability or for improvement of food safety, and many others. Although these technology tools focus on specific functionalities and their data/information exchange with other systems is limited, they are at the cutting edge of current food information technology and give some clues about future food information services to consumers.

In this paper we envisage these new food information services that we call intelligent food services (IFSs), and we argue that they are characterized by having the following capacities:

• to be context aware;

• to transform data to knowledge/intelligence;

• to enable social connection/interaction;

• to run on cloud/app/sensor-based infrastructure.

Main research contributions of this paper can be summarized as follows:

• a functional framework for IFS and some real case examples and usage scenarios.

• an interaction context model, with a particular focus on food in-context awareness characteristic that distinguishes an IFS from other information services.

2 Towards Intelligent Food Information Services

“Internet of Food” is an offshoot of the Internet of things. It can be viewed as a network of smart food things, i.e., food-related objects and devices which are augmented with sensing, computing and communication capabilities in order to provide advanced services. Smart food things include sensor-equipped information artifacts (e.g., food labels with RFID or NFC tags), time-temperature indicators and other sensors on packages to detect spoiled foods, sensor devices that spots bacterial infection in food and water, kitchen devices that generate a record of compliance with food safety protocols, wearables to count bites and estimate calories, and so on.

In past years, research scientists have devoted considerable resources to the development of analytical tools for a quantitative and/or qualitative analysis of various food aspects, such as the identity of product and ingredients, region of origin and/or species, quality attributes and variety of ingredients. Until recently, their approaches had to be implemented in a laboratory with large and expensive equipment to get reliable data. New technological developments, especially in food sensor miniaturization, have made available hand-held and low-cost devices to capture food data or food-related entities data (e.g., data from label, package, container, environment), and to communicate them with a specialized smartphone/tablet app. Despite their miniaturization, these devices incorporate an analytical precision and resolution almost equivalent to bench-top instruments [8]. These advances are forging the convergence of consumer and enterprises technologies, and they are contributing to development of the “Internet of Food”. Moreover, they make possible to conceive a new generation of food services that, throughout this paper, are referred to as intelligent food services (IFSs). These services are capable of carrying out intelligent functions (like sensing and monitoring, food data acquisition and analysis, food information searching and reasoning) to facilitate food decision-making.

The intelligence is essentially due to two main service characteristics:

• context aware. By leveraging on context-data acquisition and outcomes of a machine learning or a crowdsourcing process, IFSs are capable to identify and understand enough of a consumer’s current situation in order to address his/her specific food information needs;

• knowledge leverage. By reasoning with identified current context items and available food knowledge resources (from simple food facts to food knowledge bases or food ontologies) IFSs are able to provide the user/consumer with personalized and context-sensitive food information.

Due to these characteristics, IFSs are most likely to be more effective than conventional food information services (product labeling, mass media or other traditional channels). For instance, they can help consumers with meal planning that is personalized according to their health condition and other factors such as food cost and affordability, and they can warn consumers about allergens or intolerance forms of food they are going to ingest, based on their known medical history.

On one hand, they empower consumers with information that was not previously accessible. Providing information on the safety and quality of foods and processes, as well as on issues around environmental, social, and ethical aspects, they let consumers have transparency on which trust can be built. In this sense IFSs could represent a means to create social food awareness. On the other hand, they reduce the consumer’s cognitive load needed to interact within the context in which decisions are made.

Several real case examples (see next subsection) suggest a general framework describing IFS application systems. From the infrastructural point of view, IFSs could rely on cloud/app/sensor-based solutions. Service applications could run on a cloud computing infrastructure and be deployed via mobile device so they could interact with the user/consumer, food items and food-related entities through an app that has access to user’s mobile device resources (e.g. built-in device sensors and local databases) or is connected to an external sensor device. Besides, they could interact with a sensor networked environment (e.g. a place where sensors are attached to food-related objects and connected to a wireless network) in order to detect food-related events. Having this in mind, we introduce a conceptual architecture for an IFS system, which is visualized in Fig. 1.

Fig. 1.

Conceptual structure of an IFS application system.

This structure can be considered as a kind of backbone for a new class of applications supporting intelligent food information provision. The IFS system front-end is the interface between the user/consumer and the IFS system back-end, and it is responsible for interactions with the external environment (user’s request formulation, sensor data acquisition, and information presentation/visualization to the user); the IFS system back-end analyze context data versus domain-specific context knowledge (e.g., mining knowledge from lookup databases of previously identified food images, or of previously classified NIR spectra of food samples) to determine the current context configuration; successively, it exploits general food knowledge to adapt food information provision to the current user/consumer situation.

Domain-specific context knowledge can be acquired by using supervised learning from a training set of food samples data (e.g., previously classified NIR spectra of food samples) or by mining knowledge from a crowdsourced food properties database (e.g., a lookup database of previously identified food images).

2.1 Real Case Examples and Usage Scenarios

Over recent years, some pioneer companies have entered the food information market, exploiting benefits of IoF in order to provide consumers with food information. Such companies are bridging the gap between recent research advances in various scientific fields (such as spectroscopy [8, 9], machine vision [10, 11], hyperspectral imaging [12], odour analysis [13] and taste analysis [14]) and industry.

In what follows, we propose some usage scenarios, derived from real-world examples, to clarify how IFS are currently provided by these companies.

Usage scenario 1

A user is currently purchasing some meat in a store. He/she needs information about the freshness of that food. The user scans food with an odour-analysis based device (Smart Food Thing), able to acquire data about volatile organic compounds (VOCs) emitted by food [15]. The odour-analysis device is coupled to the user’s smartphone via mobile app. Data are sent to a cloud-based engine able to perform a pattern recognition analysis to classify some food characteristics of that food item. Results of classification are processed by IFS back-end. The mobile app tells the user about the level of food freshness of the meat he/she analyzed.

FOODsniffer (http://www.myfoodsniffer.com/) and CDx inc. (www.cdxlife.com) are two examples of companies providing this kind of IFS.

Usage scenario 2

A user is going to eat a cooked meal in a restaurant. He/she needs information dealing with his/her diet. The user takes multiple pictures of food by means of his/her smartphone. That device acquires data about surface condition and volume of that food item. A data analysis unit, by mining knowledge from a food properties database, is able to recognize multiple food in user’s plate and infer the mass of food products. IFS back-end processes food information exploiting a food knowledge database (e.g., nutritional facts tables). Moreover, data on user’s food activities (e.g. historical data on previous food assumed the user during the day) and user profile (e.g. health related issues) are processed in order to provide the user with customized information. A mobile app informs the user with about the amount of calories and nutrition in his/her plate and the number of calories he/she can still consume in the rest of the day.

Calorie Mama (http://www.caloriemama.ai) and Lose it! (https://www.loseit.com) are two examples of companies providing this kind of IFS.

Usage scenario 3

A user is going to cook some food in his/her kitchen. He/she needs information about calories and nutritional facts of that food. The user takes a picture of food by means of his/her smartphone, acquiring data about surface condition of that food. Data analysis is based on a real-time crowdsourced process, as the picture is analysed by expert volunteers that manually recognize food items. Data about nutritional facts are processed by IFS back-end and information on calories and nutritional facts is provided through a mobile app.

An example of solution based on crowdsourced analysis process is represented by Rise (https://www.rise.us/).

3 The Interaction Context

Internet of Food (IoF) is an offshoot of the Internet of Everything. It can be viewed as a network of smart food things, i.e., food-related objects and devices which are augmented with sensing, computing and communication capabilities in order to provide advanced services. Smart food things include sensor-equipped information artifacts (e.g., food labels with RFID or NFC tags), time-temperature indicators and other sensors on packages to detect spoiled foods, sensor devices that spots bacterial infection in food and water, kitchen devices that generate a record of compliance with food safety protocols, wearables to count bites and estimate calories, and so on.

An interaction is generally understood as the effect two or more entities will have upon their counterparts as they perform actions on each other. The user-service interaction context is a set of entities with their relevant properties (called items) characterizing the situation the consumer is currently involved in, while he/she is interacting with the service [17].

In user-IFS interaction context modeling, four broad classes of context entities should be taken into account:

• user: a consumer who requires the information necessary for his/her food decision making or learning process as he/she is involved in food consumption activities in a real spatial environment; he/she interacts with an IFS through interface devices located in the environment;

• food: foodstuff that the consumer is interested in. Food related stimuli are perceived by the user and food characteristic data are acquired by the IFS during the interaction;

• IFS interface devices: they comprise devices that are used by the IFS system: (a) to take input from and deliver output to the user in the foreground of user’s attention. These include food information artifacts (FIAs¹), such as labels, tables, RFID chips, and NFC tags, that allow the user to get their contained information in the place where the artifacts are located; (b) to take input from the user in the background of user’s attention, while he/she is involved in food consumption activities, such as many wearables for food intake monitoring [18]; (c) to take food and environment data with or without requiring user’s action or attention. These include sensor equipped food packaging and containers, food appliances, as well as small tools designed for food-related functions (e.g. kitchen or cooking utensils);

• environment: physical and organizational environment where interactions take place (e.g. a home kitchen, a restaurant, a food shop, etc.). Environmental conditions have direct or indirect influence on consumer and service behaviour during the interaction. Physical properties, like light, humidity, temperature, localization, spatial layout of the environment, may change quality attributes of the food or consumers’ perception of quality in food; organizational aspects, like rules, shop opening hours, working time, may drive the provision of information from the IFS.

According to many approaches for using and modelling context information, items of the above context entities fall into five categories [19]:

• Identity. A model of this item allow the service, or the user, to identify a context entity that is needed to know in the interaction with the counterpart. In particular, the user determines food identity by elaborating his/her food sensorial perceptions, like colour and appearance, taste, and odour, while the IFS system determines food identity by elaborating food data coming from instrumental measures of physical, chemical, biological, and microbiological food properties;

• Time. This item refers to temporal aspects of context entity properties. Item values may range from a current time representation to a complete time history of entity properties. When referred to food, values of this items allow to recognize or predict food quality over time. For instance, a time series of values of properties, like temperature, pH, or microbial growth, may help the user or the service to recognize when food is at its nutritional best and is safe to eat and when it should be disposed of to avoid ill health;

• Location. A model of this item may be quantitative or qualitative, and item values may represent current and previous positions of context entities in absolute or relative terms. When referred to food, location is a fundamental item of models for geographical traceability or geographic-based origin determination of food products.

• Activity. A model of this item describes fundamental changes of entity attributes that occur when a food activity is performed. In particular, changes of food characteristics, like temperature or size, may let the user or the service to determine the state progress in a current consumer food activity (e.g. cooking); as a further example, chewing sounds may let a service to classify the food a consumer is eating.

• Relations. This item refers to relations a context entity has established with other context entities. A relation between two context entities expresses a semantic dependency that emerges from user-service interaction situations that these context entities are involved in. Relations between food and other entities are essentially relations of spatial proximity (e.g., relative distance and orientation). For instance, food data capture is strongly dependent on the relative distance and orientation of the food product and the sensor device.

4 Food-in-Context Awareness

Interaction context-awareness refers to the ability of the service (respectively, user) to acquire context item information and to use it in the interaction with the counterpart; context item information is acquired by capturing context data (respectively, sensory/cognitive cues), analyzing them, and exploiting some context knowledge.

In an interaction session, context awareness has a significant influence on the behavior of the consumer, as service user, and on the behavior of the IFS, as food information provider. As matter of fact, context changes can trigger IFS/user actions on the counterpart, and context cues or data can support IFS/user to interpret actions performed by the counterpart. IFS and user mutually increase their context awareness, as they interact with each other. In particular, Fig. 2 shed light the role of context in an interaction between user and IFS, i.e. it further focuses on the Consumer-IFS interaction module reported in the functional architecture (see Fig. 1).

Fig. 2.

Contextualized user IFS interaction.

Food-in-context awareness is context awareness limited to food items, including relations with other context entities. In what follows we discuss more deeply user/consumer and service food-in-context awareness and their roles in the interaction. Successively, we summarize main technical approaches that can be used to make an information service food-in-context aware, as they are aimed to automatically determine information on food characteristics and performances.

User/consumer food-in-context awareness.

It is the user’s ability of knowing and experiencing contextual food items, i.e., sensory properties (such as appearance, taste, and odor) of food in the interaction context. Signals from contextual food can trigger a user’s information need that in its turn can trigger a request to a food information service. For instance, the presence of a new food product can trigger the user need for nutrition or health information about that product (e.g. what vitamins have).

More important, in a contextualized interaction the user attempts to make sense of data and information delivered by the service provider, within his/her current situation and his/her own knowledge acquired from past experiences. Contextualization is absolutely essential both to let the consumer make appropriate choices in his/her food consumption activities in which he/she is currently engaged and to enable consumer’s informal learning process that is crucial for his/her future related behavior.

User food-in-context awareness is at the core of user contextualization, since it provides the user with sensory and cognitive cues that allow him/her to refer delivered data and information to food items in the interaction context:

• value-laden information embedded within a food product can be effectively disclosed only by associating the perception of food-related stimuli (e.g. smell, flavor) with the logical information provided by the service;

• information content of labels/tags can be properly interpreted only by taking into account spatial relations between food products and these labels/tags. Think of a label on a food-shelf; we automatically refer the label information content to the closest food product on the food-shelf, and the label information content changes meaning as the label has moved to another place;

• food information provided by the service can be directly validated only by knowing specific physical conditions (e.g. light, humidity, temperature, localization, spatial layout, package integrity) or organizational features (e.g. operation rules, shop opening) of the environment surrounding food products the information content is referring to. For example, the “best before date” indicates the last date by which a product flavor and/or quality is best, but what is the value of this information if the product package has been opened?

Service food-in-context awareness.

It is the ability of the service to capture and analyze food data versus domain-specific food knowledge in order to acquire information on food in current context. Food data coming from instrumental measures of physical, physico-chemical, and kinetic food properties are correlated with sensor properties such as color, appearance, taste, odor, and sound.

Service food-in-context awareness is at the core of context-adapted information provided by an IFS that creates value in letting consumers make better informed food choices, while they are involved in related food activities. Context-adapted information provision is characterized by the following dimensions:

• context-triggered. Changes of some food items can be detected and used by an IFS as a means to trigger context data acquisition that, in turn, can lead to timely information provision, i.e., information delivered when and where desired or needed by the consumer. For instance, an IFS may alert the user of a sensed condition within a refrigerator that food spoilage may occur.

• context-situated. In a current interaction session, an IFS information provision makes exophoric reference to context entities (e,g, food products or food-related objects) surrounding the user/consumer who, in his/her contextualization process, is required to resolve this exophoric reference as well as to assign meaning to spatial constructs. Service context awareness lets an IFS make easy and smooth the consumer’s contextualization process by eliding spatially-retrievable information and inserting a minimal amount of unnatural clarification statements, in a food information provision. For instance, in a food supermarket, consumers could be guided towards what they’re looking for, by using simple spatial deictic expressions in utterances such as “take a left” or “head towards the stairwell”.

• context-focused. The IFS satisfies contextualized needs of information, i.e., consumer’s needs of information that can be derived from the knowledge of current food items. This kind of adaptation couples contextualization and personalization, as addressing not just decontextualized consumer’s needs, but what the consumer needs in the current context (e.g., in that place, at that time). It exploits an explicit context model that takes into account many context entities and it is broader than the only consumer model limited to personal characteristics such as preference, background, attitudes, and lifestyle. For instance, an IFS can suggest restaurants or food stores based on the user’s preferences and current location and time.

5 Conclusions and Future Works

Internet of food based solutions are promising a huge impact on both food business and food consumers. In this paper, the envisaged intelligent food services constitute a part of the Internet of food vision that is grounded on recent researches and technology advances.

In this work, we proposed a functional framework such services. We provided also some usage scenarios to clarify how emerging companies in food information market exploiting benefits of IoF to provide context based information services. Moreover, we have introduced and discussed the concept of food-in-context awareness as a crucial ability for these services in order to provide consumer’s situation adapted information.

In our opinion, this constitutes a contribution to realistically conceive innovative food information services that could let consumers be more food aware and could help them to make more informed decision when involved in food related activities.

The emergence of these services poses new challenges and problems but it also gives new business opportunities for tech startups in the Food Information Ecosystem. A critical point concerns with the consumer acceptance of these new technologies. Future research will be addressed to apply the Technology Acceptance Model (or its extensions) to evaluate the individuals’ acceptance of this new generation of food information services.

In the paper we have presented some real-case applications, however a comprehensive study of companies operating in the food information market is missing. Further research will be devoted to conduct a survey aimed to understand technology foundation and business models of startup companies leveraging on IoF technologies, and to identify potential trails leading to the emergence of IoF based ecosystems.