Introduction

From the day mobile devices have had cameras, they have been used to decode markers. The most popular markers today are QR-Code and Datamatrix. These markers can be found in many items starting from fast-moving consumer goods (FMCG) and going all the way to industrial and medical usage. According to Tiwari (2016), the first one-dimensional (1D) marker was invented in the 1960s. Since markers have evolved from 1D barcodes to multi-color two-dimensional (2D) markers (Fig. 1) [56]. Marker development has followed other technologies that relate, like cameras and image vision. Currently, there are multiple marker types, some more general, some more use-case specific.

Fig. 1
figure 1

Evolution of barcodes, adapted from [56]

Full size image

This research extends standard QR-Code and Datamatrix markers to smart tags. Smart tags primary function is to provide information to its user regarding the status of item [38]. Printed non-electronic smart tags function by communicating through the physical senses (human vision) or machine vision.

Functional inks are used to extend standard QR-Code and Datamatrix markers. These inks form the part of the marker which acts as an active sensor by reacting to environmental variables like humidity, temperature, or light. Functional inks are suitable to the normal label print process of markers. The chemical structure of the functional ink determines how and on which environmental variables ink reacts.

As summary, this research looks into solving following research questions:

  • How to embed functional ink inside the QR-Code or Datamatrix?

  • How does the embedding impact the decoding performance of QR-Code or Datamatrix?

  • Can CIEDE2000 color difference algorithms accurately recognize state of the functional ink?

The rest of the paper is organized as follows. Section “Related Work” goes through related work. Section “Materials and Methods defines what smart tags are, how label printing works, introduces use-cases that relate to smart tags, and looks into the proposed technical approach. Section “Experiments” discusses details of experiments, and in Section “Results and Discussion”, results of the experiments are discussed. Finally, Section “Conclusions” concludes the work, followed by future research topics in Section Future Work.

Related Work

Relevant past research work has focused on exploring possibilities of intelligent packaging in the FMCG industry. As part of intelligent packaging smart tags have been seen as a possibility to track items and their status. For example, this research has been done in the context of food items by Yam [60], Mohebi and Marquez [34], Realini and Marcos. [43], Kalpana et al. [21], and Fuertes et al. [8].

Related research has developed new advanced markers. These markers have one difference from traditional markers. Where traditional markers use black and white values to present binary values, advanced markers use a limited spectrum of colors. Parikh and Jancke proposed an approach to recognize multiple colors in 2D color barcodes [36]. Bulan and Sharma proposed using dot orientation, with colors, to encode data into high-capacity barcode [4]. Grillo et al. proposed a new barcode that uses colors to include more data [13]. Bulan and Sharma proposed a barcode that uses the cyan, magenta, and yellow colorant separations to enable high-capacity barcode [5]. Subpratatsavee and Kuacharoen proposed a new barcode that reduces the physical space needed for the barcode. In their research, Subpratatsavee and Kuacharoen present the design and the implementation of a high-capacity two-dimensional barcode [50]. Taveerad and Vongpradhip proposed color QR-Code to hold more data through a novel encoding concept [54]. Common to all these approaches is that in theory, black & white marker data can only be translated into 1s and 0s when decoded. However, colors, or shades, can be translated to as many values as possible to identify [41]. Most of these past studies relate to using colors to encode more data into barcodes. Or how to change data that barcode contains through colors. John and Raahemifar provided an overview of color barcodes. They also proposed a binarization and grouping algorithm to encode data to form a color barcode [20]. Wasule and Metkar took into account the intensity variation which occurs while decoding colored barcodes and proposed an approach that will increase the capacity of barcodes beyond threefold; in their work, they used quantization of grey levels [59]. Ramalho et al. used the concept of super-modules to encode more data into QR-Code but also to make QR-Code more secure [42].

Functional inks and mainly their properties have been studied during recent years. Gao et al., in their research, focused on time-temperature indicators (TTIs) data modelling [9]. Li and Chen used two different functional inks in their proposal as a new Dynamic and Sensitive Barcode (D&S), D&S can react to environmental state change [27]. Chen et al. used a vice-versa approach and made the paper reactive to environmental changes, and they also used smart devices to decode color information [6]. Kulčar et al. focused their research on the properties of functional inks regarding state changes [24].

One of the most relevant research around functional inks and smart tags is a study by Gligoric et al. In this research, authors defined possible approaches for smart tags [12]. Quite similar work was done by Hakola and Vehmas [14]. However, these studies mainly focus on uses-cases and properties of functional inks, and only briefly define technical frameworks for ink-based smart tags

This research uses CIEDE2000 (KL, KC, KH) algorithm for color difference calculations. CIEDE2000 has been used in many studies where color difference is matched against human perception or in general color comparison. This work has been conducted for example by Taoa et al. [52], where authors used the algorithm to recognize leaf color differences. By Nguyen et al. [35] in the context of rice color recognition and by López et al. [30] for mortar color differences. Usage of KL = 1, KC = 1, and KH = 1 parametric values is the most common approach when using CIEDE2000 [11, 31, 37]. Past research has also achieved good results with KL = 2, KC = 1, and KH = 1 parametric values [10, 32]. The third set of parametric values used in this research is KL = 2.76, KC = 1.58, and KH = 1, which has been used in the past, especially in digital and printed images [29].

The past research approaches smart tags from different point-of-view; they rarely consider the actual FMCG label printing process. And previous color difference research has focused on color difference calculations; especially, in use-cases where color comes from a physical object, or colors have a different hue.

Contributions of this research work are:

  1. (a)

    Technical approach to extend standard QR-Code and Datamatrix markers to smart tags, without significant impact on marker decoding performance.

  2. (b)

    Verify the suitability of embedding functional inks into QR-Code and Datamatrix markers.

  3. (c)

    CIEDE2000 algorithms’ parameter comparison with printed colors.

Materials and Methods

Labels

Smart tags have been defined in the past research in various ways. They might refer to radio-enabled electronic devices, e.g., [62], radio frequency identification (RFID), or near field communication (NFC) tags, e.g., [1]. Smart tags have also been considered as printed electronics, e.g., [45].

Fig. 2
figure 2

Categories of labels [23]

Full size image

In this research, smart tags are standard two-dimensional markers with added intelligence from functional inks. Smart tags are printed on self-adhesive labels, and finally glued to FMCG products. This paper does not take into account labelling/printing that is done directly on items, like laser marking. According to Kirwan (2012), the invention of self-adhesive labels was in the mid-1930s. Labels were back then used to apply price and decoration on store items. Currently, the FMCG industry uses labels to add value to a product item during its life-cycle. The latest development of labels has made them smart, smart-active, or smart-intelligent (Fig. 2). Smart labels have various usages like tracking products, monitoring their temperature, and indicating food freshness [23].

Smart tags have the functionality to react to environmental changes and identify individual items, rather than presenting only static content. This categorizes smart tags as customized labels [14]. Smart tags defined in such a way are part of intelligent packaging, an emerging technology in the FMCG. Intelligent packaging uses smart tags as the communication function between the package and the user. Smart tags and their information help in the decision-making related to the item, to achieve the added value, like enhanced food quality, user experience, and safety [60].

Label Printing Process

Labels with smart tags are either (a) pre-printed in print houses or (b) printing occurs just before they are applied (in-house) [49].

Depending on the chosen label printing facility, pre-print or in-house, multiple technologies can be used to print labels.

  • Rotary and semi-rotary letterpress.

  • Flexographic printing.

  • UV-flexographic printing.

  • Screen printing.

  • Offset printing.

  • Rotogravure printing.

  • Thermal printing.

  • Laser and inkjet printing [23].

All listed technologies are suitable for the pre-printing of labels. For labels printed in the production line, thermal, laser, and inkjet printing are plausible. These non-conventional technologies can also print customized information that varies per label. It is also possible to mix conventional and non-conventional ways of printing; this is called hybrid printing [23].

Electronic/non-electronic labels can occupy as much as 50% of the package costs. However, in most low-priced products, like food, the price of the package should not exceed 10% of the total costs of the product [58].

Functional Inks

Functional inks can report exposure to environmental influences by switching between two states of optical properties. The state of the ink and its absolute value depends on the current properties of the used ink. Functional inks can ultimately appear either in an active (1) or non-active (0) binary state. Switch between states is relative to physical influences. For example, water/humidity (hydrochromism), temperature (thermochromism), and the intensity of light (photochromism) can change the state of the functional ink [15].

Depending on the environmental influence, and properties of the ink, change between two binary states occurs in a specified time. During this period, functional ink can also have values between 0 and 1. Optically visible change between states can either be a change between colors or color-changing its intensity. Also, depending on the chemical compound of the ink, change can be reversible or irreversible [3].

According to Zabala et al., the change in functional ink can represent accumulated exposure (total), or if the ink has exceeded its activation point [61]. Therefore, functional inks can track continuous exposure during the whole life-cycle, or exposure to an environment lower/higher than the threshold set.

Functional inks in this research are compatible with the high-speed printing process (conventional, non-conventional, and hybrid). And to meet labelling cost requirements, the price of the functional ink is close to the price range of standard color inks. The advantage of the smart tag with functional ink is that there are no electronics in it. This makes the manufacturing technically less complex than RFID/NFC-based smart tags. The advantage is also in recycling, as some regulations define RFID/NFC-based tags as electronics. Using functional ink raises the possibility to design innovative products for markets not yet addressed by the electronic tags [12]. Functional inks are suitable for most printing methods presented in the previous chapter, including flexographic, gravure, screen, and inkjet [28].

When smart tags are developed for the needs of the food industry, they should not impose thread to items inside packages or when in direct contact with food items. There are three main ways of food safety can be compromised by the ink used in labels/packages [46]:

  • Migration, components of ink pass through the substrate

  • Invisible set-off—transfer of components of ink from the printed side to the food-facing side, for example when packages are stacked.

  • Gas-phase transfer of ink components via the air in the packaging to food.

Some functional inks are suitable for short- or long-term direct food contact (DFC). These inks meet regulations set by authorities like EuPIA (European Printing Ink Association) and U.S. Food and Drug Administration (FDA). However, some functional inks can only be used in non-food contact. Past research has also developed options for this like impermeable barriers/papers, which can be used when printing smart tags [61]. As this paper only focuses on proposing a general approach, research of these options is left for future research.

Use-Cases for Smart Tags in FMCG Industry

Fast-Moving-Consumer-Goods are perishable packaged products purchased and consumed by all members of society. FMCG cover items like food and beverages; often, these goods are also considered as Consumer Packaged Goods (CPG). One of the main purposes of the product packaging is to communicate important information about item(s) in the package to the user. Information user gets through labels, which might relate to the ingredients/material of the product. However, the label can also contain other information, like nutritional facts. Packages, and their labels, look to meet customer experience expectations. However, some of their functionality is also responsible for fostering a circular economy. Part of this is cradle-to-grave tracking and actions that relate. Typical life-cycle (cradle-to-grave) of an product is illustrated in Fig. 3.

Fig. 3
figure 3

Product life-cycle and its relation to intelligent packaging [60]

Full size image

As shown in Fig. 3, intelligent packaging links to all phases of the product life-cycle. In this context, smart tags should deliver various information to the user. This information contains either data about the current status of the product or information that relates to the product’s historical status. Depending on the life-cycle phase, user might be interested in a lifetime, temperature history, freshness status, package condition, authenticity, or something else [23].

Smart tag-related research has recognized suitable use-cases which directly or indirectly contribute to meeting the needs of consumers, producers, and manufacturers [7]. These uses-cases are classified as follows:

  • Manufacturing (Raw material, Product manufacturing, Packaging).

  • Distribution & Consume.

  • Recycle [38].

Listed use-cases are related to item identification and providing information about the item to the current stakeholder or user. When the packaged item has a smart tag on its label, the item has intelligent packaging. The intelligent package is “a packaging that contains an external or internal indicator to provide information about aspects of the history of the package and/or the quality” [44]. Some use-cases are more suitable for intelligent packaging than others.

As printed smart tags belong to the same category as RFID tags’ most use-cases can be derived from RFID/NFC use-cases [12]. Printed smart tags lack a radio interface, so the decoder must have a line of sight and relatively close distance. However, printed smart tags support a visual interface that low-cost RFID/NFC do not currently have. Generally, the costs of the smart tags are part of the packaging costs.

Technical Approach for the Smart Tag

Smart tags developed for the FMCG, must be able to: (a) avoid any false negatives (samples that seem safe but are dangerous) and (b) they should have as few false positives (samples that seem unsafe but are healthy) as possible.

If a functional marker detects false negatives, it provides information that the item is safe, although the reality is vice versa. False positives are not that crucial, but providing false positives weakens the trust towards the system. False positives provide information that the item is unsafe even if they are not. The price of the marker is also a meaningful factor and how the user can decode the marker [39].

Intelligent packaging use-cases are dependent on two features. First is a unique identifier, which provides a way to trace items’ history and link life-cycle events of the item into databases or other services [38]. The second feature is a use-case-specific sensor printed with functional ink. Therefore, markers must be printed with two different inks, one that is static (unique identifier) and one that can react to environmental changes (sensor) [51].

The following three methods can be when extending markers with functional ink [51]. These are:

  • (a) Functional ink placed outside of the marker, a solution used by many of the existing smart tags. This increases the required physical print area and might make decoding of the functional ink state complex.

  • (b) Functional ink changes markers’ data, and this solution is more advanced and requires specific and matching functional inks and compatible marker contents.

  • (c) Functional ink placed inside of the marker, a solution presented in this paper.

In the proposed approach, functional ink is placed inside the marker without disturbing the reserved cells, like a timing pattern. This does not exceed markers’ error correction capability. However, due to ink misplacement or dye growth, the approach uses functional ink intensity satisfying requirements for symbols set by GS1 standard (bar code symbol print quality test specification—two-dimensional symbols) [16]. In these requirements, the contrast between the symbols black and white cells should not be lower than 40%. Placing functional ink inside marker also affects modulation criteria, which compares local contrast to global contrast. As functional ink area disturbs the global threshold, it might affect the probability of incorrect cell color identification. In such a case, base markers’ capability to recover from error/destroyed data is used while decoding. When using QR-Code or Datamatrix as the base marker, around 30% of destroyed data can be recovered [17, 18].

The proposed approach has the following advantages: (a) it does not occupy more physical space than adding standard QR-Code, (b) QR-Code can work as call-to-action, and (c) it is technically suitable for the current label printing processes. Technically approach might be suitable for use-cases with more than one functional ink (sensor), but more research is needed.

Figure  4 shows proposed approach in QR-Code. In this approach, the sensor area would be defined by rectangle

$$\begin{aligned}&{x = 8}\\&{y = n-3}\\&{w = n-9} \\&{h = 2}, \end{aligned}$$

where n is the count of cells marker has in vertical/horizontal direction and indexes of the cells start from top-left corner where index is (0,0) and bottom-right corner has index (\(n-1,n-1\)). When sensor data are occupying such a rectangle, it does not occupy more space than 24 cells (\(\tilde{1}3\%\)) of 189 data cells in the smallest possible Qr-Code (21\(\times\)21) [17]. Some research needs to be done in the future to explore if this amount of sensor cells is enough for accurate color recognition. Theoretically, the sensor could occupy more area within QR-Code. However, this might affect the decode performance of the QR-Code. Especially performance could be lower in challenging environments if more errors occur in other parts of the QR-Code [26]. Using only one row for the sensor might affect sensor data decoding, especially in small QR-Codes. This is because the QR-Code generator algorithm optimizes the ratio between white and black cells, but might sometimes generate rows where black/white cells have the majority [17].

Fig. 4
figure 4

Approach to embed functional ink into QR-Code

Full size image

Functional ink inside the marker is straightforward to locate. When decoding a standard QR-Code, an algorithm finds coordinates of each cell. As the algorithm knows the location of each cell, it can also define the coordinates of the sensor area. In the proposed approach, standard QR-Code scanner applications can decode contents of the QR-Code, but cannot decode sensor area information.

Decoding of the sensor value happens with a special algorithm. This algorithm calculates the difference between white and black cells’ color values within the sensor area. Then, the difference is compared to the maximum difference, calculated in the same way but from the reference area. The color difference is calculated by CIEDE2000(KL, KC, KH) formula [31]. The performance of different parameters is compared later in the experiments section. Figure  5 shows the process chart of the algorithm. The sensor is considered active if its value is over 10% of the reference maximum difference.

Fig. 5
figure 5

Sensor data decoding process of the proposed approach

Full size image

Figure 6 shows the reference area (REF) and sensor area (S1). These areas work as sources for the calculation of the white-black or sensor-black difference. The location of the reference area is two rows above the sensor area because of possible functional ink spreading or misplacement. Placing the reference area further from the sensor are might alter it to ambient lighting changes like shadows, different to ones that the sensor has.

Fig. 6
figure 6

Reference area (REF) and sensor area (S1) extraction

Full size image

Even though in Fig. 4, it shows sensor in the QR-Code marker, the approach should apply to other markers with the error correction capability, like Datamatrix. As such, the addition of the sensor affects the general performance of the marker, discussed in the next chapter. Depending on how much data are recoverable from damaged markers (Datamatrix or QR-Code), height of the sensor area can vary in different sized markers. In this research, sensor area is always two rows in height.

Experiments

This section focuses on the practical experiments with the proposed approach. The focus of the first experiment is to validate how sensor embedding impacts the decoding performance of the marker. The focus of the second experiment is to test the calculation of the sensor value in simulated use-cases. Finally, the third test experiments with three real-life markers using actual functional ink.

Past research has shown that the size of the marker should be larger than 10\(\times\)10mm and have at least 4 pixels (px) per cell [40, 53]. Based on this information, for the first two experiments, three different markers in size 20 mm \(\times\) 20 mm were printed with an office laser-jet (Canon ImageRunner C5535i). Printing was done in 300dpi to standard office A4-paper (Canon Black Label Plus 80 g/m\({^2}\)).

Markers had different data to represent different item-level tracing options:

  1. (a)

    EAN/UPC-13 data [47], 13 digits, illustrated as 25\(\times\)25 sized QR-Code

  2. (b)

    EPC data, 26 digits [55], illustrated as 29\(\times\)29 sized QR-Code

  3. (c)

    UUID data [57], 36 digits, illustrated as 37\(\times\)37 sized QR-Code.

Using different sizes is to experiment with how the proposed approach performs when there is a different amount of pixels per cell.

The third experiment uses real-life markers generated with the proposed approach. These markers were printed with the following functional inks:

  1. (1)

    LCR Hallcrest cold activated thermochromic ink, clear to green CMYK (22,7,17,0) at 7 \(^\circ\)C, 38 cm\({^3}\)/m\({^2}\), 1 printed layer [25]. Printed using flexographic printing.

  2. 2)

    Datamatrix with SICPA heat activated thermochromic ink, clear to magenta CMYK(0,100,0,0) at 26 \(^\circ\)C, 38 cm\({^3}\)/m\({^2}\), 1 printed layer [48]. Printed using screen printing and flexographic printing.

  3. 3)

    Datamatrix with Sunlase TDS sulfuric acid reactive ink, clear to black CMYK(0,0,0,100), 38 cm\({^3}\)/m\({^2}\), 1 printed layer Printed using screen printing.

The third experiment has also the difference that it uses Datamatrix codes where functional ink is embedded in the same way as in QR-Code used previously.

QR-Code Integrity Experiment

The first test experimented with two versions of each marker (a) version where sensor area was transparent (sensor off), in Fig. 7 top row. And (b) version where sensor was totally black (sensor on), in Fig. 7 bottom row.

Latter markers simulate a situation where the sensor destroys all white cells within the specified data region, and error correction recovers these data. This experiment looks to validate that adding a sensor does not prevent the decoding of the marker.

Fig. 7
figure 7

Markers for the first experiment

Full size image

In this experiment, iPhone 7 and iPhone 11 Pro were used, with a standard camera application, to decode QR-Code markers from different distances. Controlled environment, 25 cm \(\times\) 20 cm \(\times\) 40 cm sized white box, was used for the test. The test environment had a moving sledge for distance control and adjustable LED lights for ambient lighting control. Decoding distance was measured by moving the sledge to the furthest distance where decoding of the marker happens continuously. In this experiment, decoding distance was tested in two ambient light levels, 400lx (office lighting) (Table 1) & 15lx (dark environment) (Table 2).

Table 1 Experiment results of the 400lx experiment
Full size table
Table 2 Experiment results of the 15lx experiment
Full size table

Sensor Value Calculation Experiment

The second experiment focuses on testing if the algorithm defined in the previous section can identify sensor states when different sensor colors are used. Second experiment uses same marker sizes and contents as the first experiment. However, this time with different sensor colors and their intensities. Four pure colors magenta (\(C = 0.0, M = 1.0, Y = 0.0, K = 0.0\)), cyan (\(C = 1.0, M = 0.0, Y = 0.0, K = 0.0\)), yellow (\(C = 0.0, M = 0.0, Y = 1.0, K = 0.0\)) and black (\(C = 0.0, M = 0.0, Y = 0.0, K = 1.0\)). And three mixed colors red (\(C = 1.0, M = 1.0, Y = 0.0, K = 0.0\)), green (\(C = 1.0, M = 0.0, Y = 1.0, K = 0.0\)) and purple (\(C = 1.0, M = 1.0, Y = 0.0, K = 0.0\)) were used.

For each color, three marker sizes were printed with different intensity (100%, 80%, 60%, 40%, and 20%) levels. Markers were captured, with iPhone 7, Nokia TA-1032, and iPhone 11 Pro to images from two distances 10 cm and 20 cm, in the same ambient light levels as in experiment one. Aside from capturing markers in a controlled environment, images were captured in random contexts, including daily environments like fridges, office tables, and store shelves. This totalled 1260 images for the second experiment, and examples of photos used in this experiment are shown in Fig. 8.

Fig. 8
figure 8

Examples of photos used for the second experiment

Full size image

In the second experiment, QR-Codes were decoded with a custom Python application [19]. This application used algorithm shown in Fig. 5 to recognize sensor state. In the experiment, three CIEDE formulas with different parametric values for KL, KC, and KH were used to calculate color difference. These formulas were CIEDE2000(1,1,1), CIEDE2000(2.76,1.58,1), and CIEDE2000(2,1,1).

Tables 3, 4, and 5 show the results of the second experiment. Results are separated into three tables depending on parametric values. The first column shows sensor color, and the following columns show the lowest and highest value of the sensor. These results can be used to estimate the usability and accuracy of the algorithm and its parameters.

Table 3 Results from the second experiment, CIEDE2000 (1,1,1) algorithm
Full size table
Table 4 Results from the second experiment, CIEDE2000 (2.76,1.58,1) algorithm
Full size table
Table 5 Results from the second experiment, CIEDE2000 (2,1,1) algorithm
Full size table

The third and final test experimented with smart tags printed with real functional inks. These smart tags were based on Datamatrix and photographed in a real-life environment with iPhone 7, Nokia TA-1032, and iPhone 11 Pro. First before exposing them to a reactive environment. After exposure, markers were photographed again. Fifty images were captured for each marker and sensor state, totalling 300 images. Examples of photos are seen in Fig. 9.

Fig. 9
figure 9

Examples of photos used for the third experiment

Full size image

The same Python application as in experiment 2 was used to decode markers. Table 6 shows the results of the third experiment. Columns show different CIEDE200 parameters, and values range from decoding. On the rows are different marker states

Table 6 Results from the third experiment
Full size table

Results and Discussion

Based on the results from the first experiment, adding a sensor inside the marker reduces the decoding distance only slightly. Distance is decreased more in a dark environment, but the absolute change in the decoding distance is small. In everyday use, this does not have a significant impact. Markers with more data are affected less as the sensor occupies a relatively smaller area from the data area.

In the second experiment, simulated sensors with different intensities [0%, 20%, 40%, 60%, 80%, and 100%] were decoded with the proposed approach. The second experiment included sensor value calculations with the CIEDE2000 algorithm. Three different parametric (KL, KC, and KH) values were tested, (1,1,1),(2.76,1.58,1) and (2,1,1). Results show that when sensor intensity is 40% or more, all parametric combinations can recognize the state of the sensor. When intensity is lower (20%), calculations are not working accurately, and incorrect states are sometimes recognized. Especially, algorithm has challenges with the yellow ink. With higher intensities, the algorithm gives values over 0.1. This can be considered as a threshold value. When the sensor has a value over 0.1, color exists, and the sensor is ON. Values vary highly, especially with higher intensities. Therefore, the algorithm cannot be used for accurate color value identification.

The third experiment showed that the proposed approach works well with actual functional inks. However, the algorithm provides sometimes lower values for the sensor as in experiment two. This might be from the label background color differences or calibration of the ink intensity. In the third experiment, all CIEDE2000 parameter combinations provided values that ranged quite widely. In practice, this means that the approach is suitable for sensor state recognition. However, it fails when used to recognize the absolute value of the sensor.

Past research has shown that it is possible to detect colors with smartphone cameras [6]. However, due to restrictions like camera accuracy, ambient light, print, and the paper quality, limited amount of color shades can be recognized [2, 22]. In the past, multiple approaches for color identification in barcodes have been proposed, for example, by John and Raahemifar [20], Bagherinia and Manduchi [2]. In these approaches, different colors represent different data. The same restrictions apply when functional ink operates as a sensor, and a limited amount of shades can be recognized. The most reliable way to use functional ink is to use high enough color intensity. It is challenging to recognize the actual shade of the functional ink. However, recognition of functional inks states, ON or OFF, is quite straightforward. The proposed approach cannot recognize shades accurately, as there is variation in results. One possible approach to overcome this problem might be calibration. In traditional calibration, color charts are used for camera calibration [2]. However, in the case of smart tags, calibration has a small amount of data in its use. These data include only labels background color (paper color), black cells of the marker, and possible color reference points. Together with calibration, reference points can be used. With reference points, it is possible to compare the state of functional ink into specified reference values [6]. However, if reference points are used, they must be printed with color as close as possible to functional ink’s value in a specific state.

Actual color shade recognition with functional ink is a challenging topic as the shade is depending on multiple factors. Ink intensity is the most important factor. However, the direction of state change that occurred last has its impact. In other words, color is slightly different if it has been reached during a negative or positive change. Also, change between states has different speeds in different environments and colors. In some cases, when changing from color to transparent, some color can remain [24].

Conclusions

To support the data-driven applications in the FMCG sector, more data need to be collected. Without this data collection, it is not possible to implement optimal decision-making processes. Currently, the obstacle is not on cloud services or data processing. But on how we collect data from low-cost items and track & trace them during their life-cycle. One approach is to integrate low-cost smart tags with functional ink into items. Functional ink would be able to react to environmental changes and work as a sensor.

Implementation of such markers can be done in various ways, and this paper presented a low-cost way to embed functional ink inside a standard marker. With this approach, the marker has two features: a unique identifier and sensor. Unique identifiers are used to track the item, and the sensor can react to environmental changes. Different color and ink combinations can react to different environmental variables like temperature, time, and humidity. Information from the sensor can be decoded by observing the color changes. One approach to recognize the color and state of the sensor is to use a simple color comparison. CIEDE2000 algorithm with (2.76,1.58,1) parametric values fits for this purpose, and it works reliably when the sensor’s color intensity is 40% or higher. On lower intensities, proposed approach sometimes provides incorrect results.

Yet, more work is needed in the field of color recognition. If color can be recognized accurately, it would allow more use-cases around functional inks and make functional inks usable for use in challenging environments.

Future Work

More research is needed on the computer vision side, especially on how to decode sensor color and its value better accuracy. Also wrapping and challenging conditions might impact proposed approach. Some options, which might achieve better results, could be advanced mathematical calculations, usage of reference colors, or using calibration methods. Usage of machine learning (ML) or artificial (AI) intelligence could also be one option. However, this might make the solution more complex, unless ML/AI solution can be run in a decoding device, rather than in the cloud. When ML/AI is considered, it could be possible to build a rich database containing information about data to color conversion or artificial intelligence models can be trained and used [33].