Empirical Software Engineering

, Volume 22, Issue 3, pp 1405–1437

The Debsources Dataset: two decades of free and open source software

  • Matthieu Caneill
  • Daniel M. Germán
  • Stefano Zacchiroli

DOI: 10.1007/s10664-016-9461-5

Cite this article as:
Caneill, M., Germán, D.M. & Zacchiroli, S. Empir Software Eng (2017) 22: 1405. doi:10.1007/s10664-016-9461-5


We present the Debsources Dataset: source code and related metadata spanning two decades of Free and Open Source Software (FOSS) history, seen through the lens of the Debian distribution. The dataset spans more than 3 billion lines of source code as well as metadata about them such as: size metrics (lines of code, disk usage), developer-defined symbols (ctags), file-level checksums (SHA1, SHA256, TLSH), file media types (MIME), release information (which version of which package containing which source code files has been released when), and license information (GPL, BSD, etc). The Debsources Dataset comes as a set of tarballs containing deduplicated unique source code files organized by their SHA1 checksums (the source code), plus a portable PostgreSQL database dump (the metadata). A case study is run to show how the Debsources Dataset can be used to easily and efficiently instrument very long-term analyses of the evolution of Debian from various angles (size, granularity, licensing, etc.), getting a grasp of major FOSS trends of the past two decades. The Debsources Dataset is Open Data, released under the terms of the CC BY-SA 4.0 license, and available for download from Zenodo with DOI reference 10.5281/zenodo.61089.


Software evolution Source code Free software Open source Debian Dataset 

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Matthieu Caneill
    • 1
  • Daniel M. Germán
    • 2
  • Stefano Zacchiroli
    • 3
    • 4
  1. 1.Université Grenoble AlpesGrenobleFrance
  2. 2.University of VictoriaVictoriaCanada
  3. 3.Sorbonne Paris Cité, IRIF, UMR 8243, CNRSUniversité Paris DiderotParisFrance
  4. 4.InriaParisFrance

Personalised recommendations