Facilitate External Sorting for Large-Scale Storage on Shingled Magnetic Recording Drives
In the era of big data and cloud computing, both external data process techniques and new storage mediums are proposed to process and accommodate the sheer amount of information with data-intensive applications. For instance, external sorting algorithms perform sorting operations directly on the storage devices to lower the data transfer latency and increase system performance. On the other hand, Shingled Magnetic Recording (SMR) is proposed to increase the areal density by overlapping tracks. However, the overlapping technique also introduces the random-write restriction because writing a track also destroys the valid data on overlapped tracks. This constraint could induce signification write amplification issue when performing external sorting on SMR drives. To mitigate the write amplification issue, this paper proposes a sort-friendly SMR drive design to lower the write amount of external sorting algorithms on SMR drives. The experimental results show that the proposed design could lower the external sorting latency by 61.99% when compared with the external merge sort algorithm.
KeywordsShingle magnetic recording External sorting Cloud computing
- 1.Knuth, D.: The Art of Computer Programming, vol. 3, 2nd edn. Addison-Wesley (1998)Google Scholar
- 2.Quero, L.C., Lee, Y.S., Kim, J.S.: Self-sorting SSD: producing sorted data inside active SSDs. In: Mass Storage Systems and Technologies (MSST), 2015 31st Symposium, pp. 1–7 (2015)Google Scholar
- 3.Lee, Y.-S., Quero, L.C., Kim, S.-H., Kim, J.-S., Maeng, S.: ActiveSort: efficient external sorting using active SSDs in the MapReduce framework. In: Future Generation Computer Systems (2016)Google Scholar
- 5.Yahoo: Yahoo! cloud serving benchmark @ONLINE. https://github.com/brianfrankcooper/YCSB/wiki (2015)
- 6.Seagate: Seagate archive hdd @ONLINE. http://www.seagate.com/www-content/product-content/hdd-fam/seagate-archive-hdd/en-us/docs/archive-hdd-ds1834-5c-1508us.pdf (2015)