Skip to main content

Advertisement

SpringerLink
Log in

Chronos: a NoSQL system on flash memory for industrial process data

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

Within Électricité de France (EDF) hydroelectric power stations, IGCBoxes are industrial mini PCs dedicated to industrial process data archiving. These equipments expose distinctive features, mainly on their storage system based exclusively on flash memory due to environmental constraints. This type of memory had notable consequences on data acquisition performance, with a substantial drop compared with hard disk drives. In this setting, we have designed Chronos, an open-source NoSQL system for sensor data management on flash memories. Chronos includes an efficient quasi-sequential write pattern along with an index management technique adapted for process data management. As a result, Chronos supports a higher velocity for inserted data, with acquisition rates improved by a factor of 20–54 over different solutions, therefore solving a practical bottleneck for EDF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://lias-lab.fr/~bchardin/chronos.

  2. SSD Mtron MSD SATA3035-032.

  3. Flash chip HYNIX HY27UG088G5B with an ALCOR AU6983HL controller.

  4. Kingston DataTraveler R500 64 Go.

  5. Kingston SD card SD/2GB B000EOMXM0.

  6. See Footnote 2.

  7. See Footnotes 3 and 4.

  8. For instance, for our benchmark (cf. Sect. 4.2), with 17 bytes tuples (among which 12 are the key), and 4096 bytes sectors, \(L = 240\) and \(F = 256\), then 99.6 % of the tree is made of leaves.

  9. Kingston DataTraveler R500 64 Go.

  10. MySQL is CPU bound for insertions on hard disk drives, all other queries and systems combinations are I/O bound.

  11. AMD Geode 500 MHz, 2 GB CompactFlash storage device.

References

  1. Pasteur, O., Léger, S.: Results of the use of MySQL free DBMS as a data historian. EDF internal technical repport, H-P1D-2007-02670-FR (2007)

  2. Bouganim, L., Jónsson, B.T., Bonnet, P.: uFLIP: Understanding flash IO patterns. In: CIDR’09: 4th Biennial Conference on Innovative Data Systems Research, Asilomar, USA (2009)

  3. Wang, Y., Goda, K., Kitsuregawa, M.: Evaluating non-in-place update techniques for flash-based transaction processing systems. In: DEXA’09: 20th International Conference on Database and Expert Systems Applications, Linz, Austria, pp. 777–791 (2009)

  4. Birrell, A., Isard, M., Thacker, C., Wobber, T.: A design for high-performance flash disks. Oper. Syst. Rev. 41(2), 88–93 (2007)

    Article  Google Scholar 

  5. Chardin, B., Pasteur, O., Petit, J.-M.: An FTL-agnostic layer to improve random write on flash memory. In: FlashDB’11: 1st International Workshop on Flash-Based Database Systems, Hong Kong, China, pp. 214–225 (2011)

  6. Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H., Prabhakaran, V.: Removing the costs of indirection in flash-based SSDs with nameless writes. In: HotStorage’10: 2nd Workshop on Hot Topics in Storage and File Systems, Boston, USA, pp. 1–5 (2010)

  7. Graefe, G.: Modern B-tree techniques. Found. Trends Databases 3(4), 203–402 (2011)

    Article  Google Scholar 

  8. Olson, M.A., Bostic, K., Seltzer, M.I., Berkeley, D.B.: In: FREENIX’99: 1999 USENIX Annual Technical Conference. FREENIX Track, Monterey, USA, pp. 183–191 (1999)

  9. Transaction Processing Performance Council: TPC Benchmark C Standard Specification (2007)

  10. Transaction Processing Performance Council: TPC Benchmark H Standard Specification (2008)

  11. Arasu, A., Cherniack, M., Galvez, E., Maier, D., Maskey, A.S., Ryvkina, E., Stonebraker, M., Tibbetts, R.: Linear road: a stream data management benchmark. In: VLDB’04: 30th International Conference on Very Large Data Bases, Toronto, Canada, pp. 480–491 (2004)

  12. Pasteur, O.: Overview of the long-term stored data in power generation units. EDF internal technical repport, H-P1D-2007-01076-FR (2007)

  13. Chardin, B., Lacombe, J.-M., Petit, J.-M.: Data historians in the data management landscape. In: TPCTC’12: 4th TPC Technology Conference on Performance Evaluation & Benchmarking, Istanbul, Turkey (2012)

  14. Shafer, I., Sambasivan, R.R., Rowe, A., Ganger, G.R.: Specialized storage for big numeric time series. In: HotStorage’13: 5th Workshop on Hot Topics in Storage and File Systems, pp. 1–5. (2013)

  15. Sumbaly, R., Kreps, J., Gao, L., Feinberg, A., Soman, C., Shah, S.: Serving large-scale batch computed data with project voldemort. In: Proceedings of the 10th USENIX Conference on File and Storage Technologies, FAST’12, pp. 18–18 (2012)

  16. Woodhouse, D.: Jffs : The Journalling Flash File System. http://sources.redhat.com/jffs2/jffs2 (2001)

  17. Konishi, R., Amagai, Y., Sato, K., Hifumi, H., Kihara, S., Moriai, S.: The linux implementation of a log-structured file system. SIGOPS Oper. Syst. Rev. 40(3), 102–107 (2006)

    Article  Google Scholar 

  18. Stoica, R., Athanassoulis, M., Johnson, R., Ailamaki, A.: Evaluating and repairing write performance on flash devices. In: DaMoN’09: 5th International Workshop on Data Management on New Hardware, Providence, USA, pp. 9–14 (2009)

  19. Lee, S.-W., Moon, B.: Design of flash-based DBMS: an in-page logging approach. In: SIGMOD’07: 33rd International Conference on Management of Data, Beijing, China, pp. 55–66 (2007)

  20. Kim, Y.-R., Whang, K.-Y., Song, I.-Y.: Page-differential logging: an efficient and DBMS-independent approach for storing data into flash memory. In: SIGMOD’10: 36th International Conference on Management of Data, Indianapolis, USA, pp. 363–374 (2010)

  21. Li, Y., Hey, B., Luo, Q., Yi, K.: Tree indexing on flash disks. In: ICDE’09: 25th International Conference on Data Engineering, Shanghai, China, pp. 1303–1306 (2009)

  22. Agrawal, D., Ganesan, D., Sitaraman, R., Diao, Y., Singh, S.: Lazy-adaptive tree: an optimized index structure for flash devices. Proc. VLDB Endow. 2(1), 361–372 (2009)

    Article  Google Scholar 

  23. Dobreva, V., Albutiu, M.-C., Brunel, R., Neumann, T., Kemper, A.: Get tracked: a triple store for rfid traceability data. In: Advances in Databases and Information Systems, vol. 7503 of Lecture Notes in Computer Science, Springer, Berlin, pp. 167–180 (2012)

  24. Agrawal, N., Prabhakaran, V., Wobber, T., Davis, J.D., Manasse, M., Panigrahy, R.: Design tradeoffs for SSD performance. In: USENIX’08: 2008 USENIX Annual Technical Conference, Boston, USA, pp. 57–70 (2008)

  25. Lee, S.-W., Moon, B., Park, C.: Advances in flash memory SSD technology for enterprise database applications. In: SIGMOD’09: 35th International Conference on Management of Data, Providence, USA, pp. 863–870 (2009)

  26. Jung, M., Kandemir, M.: Revisiting widely held SSD expectations and rethinking system-level implications. SIGMETRICS Perform. Eval. Rev. 41(1), 203–216 (2013)

    Article  Google Scholar 

  27. Bjorling, M., Bonnet, P., Bouganim, L., Dayan, N.: The necessary death of the block device interface. In: CIDR’13: 6th Biennial Conference on Innovative Data Systems Research, Asilomar, USA (2013)

Download references

Author information

Authors and Affiliations

  1. LIAS/ISAE-ENSMA, Université de Poitiers, 1 avenue Clément Ader, 86961, Chasseneuil, France

    Brice Chardin

  2. EDF R&D, 6 Quai Watier, 78401, Chatou, France

    Jean-Marc Lacombe

  3. Université de Lyon, CNRS, LIRIS, INSA-Lyon, 7 av Jean Capelle, 69621, Villeurbanne, France

    Jean-Marc Petit

Authors
  1. Brice Chardin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Marc Lacombe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Marc Petit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brice Chardin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chardin, B., Lacombe, JM. & Petit, JM. Chronos: a NoSQL system on flash memory for industrial process data. Distrib Parallel Databases 34, 293–319 (2016). https://doi.org/10.1007/s10619-015-7175-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10619-015-7175-0

Keywords

Search

Navigation