Computer Vision and the Internet of Things Ecosystem in the Connected Home
An automatic food replenishment system for fridges may help people with cognitive and motor impairments to have a constant food supply at home. More even, sane people may benefit from this system because it is difficult to know accurately and precisely which goods are present in the fridge every day. This system has been a wish and a major challenge for both white good companies and food distributors for decades. It is known that this system requires two things: a sensing module for food stock tracking and another actuating module for food replenishment. The last module can be easily addressed since nowadays there exist many smartphone applications for food delivering, in fact, many food distributors allow their end-users to schedule food replenishment. On the contrary, food stock tracking is not that easy since this requires artificial intelligence to determine not only the different type of goods present in the fridge but also their quantity and quality. In this work, we address the problem of food detection in the fridge by a supervised computer vision algorithm based on Fast Region-based Convolutional Network and an internet of things ecosystem architecture in the connected home for getting high performance on training and deployment of the proposed method. We have tested our method on a data set of images containing sixteen types of goods in the fridge, built with the aid of a fridge-cam. Preliminary results suggest that it is possible to detect different goods in the fridge with good accuracy and that our method may rapidly scale.
KeywordsComputer vision Connected home Ecosystem Internet of things Machine learning
This is a working progress being part of a research project (Computational Prototype for IoT Environments) among Colombia Institutions: Universidad Nacional, Universidad de Caldas and MABE, with code 36715.
- 1.S. A. LTD: FridgeCAM (2016). https://smarter.am/fridgecam/. Accessed 30 Dec 2018
- 2.Färnström, F., Johansson, B., Åström, K.: Computer vision for determination of fridge contents. In: Proceedings SSAB 2002, Symposium on Image Analysis, pp. 45–48 (2002)Google Scholar
- 3.Floarea, A.D., Sgârciu, V.: Smart refrigerator : a next generation refrigerator connected to the IoT. In: Proceedings of the 2016 8th International Conference Electronics Computers and Artificial Intelligence (ECAI) (2017)Google Scholar
- 4.Hachani, A., Barouni, I., Ben Said, Z., Amamou, L.: RFID based smart fridge. In: 2016 8th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–4 (2016)Google Scholar
- 5.Hazen, T.J.: Microsoft and Liebherr Collaborating on New Generation of Smart Refrigerators (2016). https://blogs.technet.microsoft.com/machinelearning/2016/09/02/microsoft-and-liebherr-collaborating-on-new-generation-of-smart-refrigerators/ Accessed 15 Dec 2017
- 8.Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision (ICCV) 2015, pp. 1440–1448 (2015)Google Scholar
- 9.Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition 2014, pp. 580–587 (2014)Google Scholar
- 11.Hosang, J., Benenson, R., Schiele, B.: How good are detection proposals, really? In: Proceedings of the British Machine Vision Conference 2014, pp. 24.1–24.12 (2014)Google Scholar
- 12.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks.In: Advances in Neural Information Processing System, pp. 1–9 (2012)Google Scholar
- 13.He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, Sun, Jian: Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. In: Fleet, David, Pajdla, Tomas, Schiele, Bernt, Tuytelaars, Tinne (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_23CrossRefGoogle Scholar
- 14.Hosang, J., Benenson, R., Schiele, B.: Learning non-maximum suppression. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pp. 6469–6477 (2017)Google Scholar
- 15.Microsoft: Microsoft Cognitive Toolkit. Microsoft (2016)Google Scholar