On the Sustainability of Academic Software: The Case of Static Analysis Tools
|
Joenio Costa Federal University of Bahia |
Christina Chavez Federal University of Bahia |
Paulo Meirelles Federal University of São Paulo |
Software Sustainability
human, social, economic, environmental, technical
Robert Goodland. 2002. Sustainability: Human, Social, Economic and Environmental.
Software Technical Sustainability
Concerned with the long-term usage of software and its capacity to evolve with changing conditions and requirements.
B. Penzenstadler and H. Femmer. 2013. A Generic Model for Sustainability with Process- and Product-specific Instances.
Academic Software
Software developed to collect, process, or analyze research results published in the academic literature.
ALLEN, A. et al. 2017. Engineering academic software (dagstuhl perspectives workshop 16252).
James Howison and James D. Herbsleb. 2011. Scientific software production: incentives and collaboration.
How measure Academic Software Sustainability?
Three exploratory studies:
- Publicization
- Life cycle
- Recognition
Study 1
Publicization of Academic Software
url, license, source code, and download availability
found 60 academic software projects of static analysis
Inclusion criterion and keywords
| Criteria | Keywords |
|---|---|
| Mentions the project, software or tool | tool or framework |
| Makes software available for download | download or available |
| Identifies project URL | http, https or ftp |
| Static analysis domain | static analysis or parser |
Inclusion criterion for extraction
| Criteria | Explanation |
|---|---|
| Identifiable | Is it possible to identify a software project among the outputs of the article? |
| Available | Can we find mention to the URL for download the software project? |
Official online presence (RQ1.1)
from 60 academic software, 15 projects (25%) do not have an official online presence
Sofware available for download (RQ1.2)
from 60 academic software, 24 projects (40%) are not available for download
Source code available (RQ1.3)
from 36 academic software available for download, 34 projects has their source code available
Software license available (RQ1.4)
from 34 projects with the source code available, 13 did not provide any software license, and 21 used free software license
Study 2
Life cycle of Academic Software
evolution stage
CAPILUPPI, A. et al. 2007. Adapting the staged model for software evolution to free/libre/open source software.
Evolution stage of academic software (RQ2.1)
| Evolution stage | Academic Software | % |
|---|---|---|
| Initial development | 20 | 33% |
| Evolution | 2 | 3% |
| Servicing | 6 | 10% |
| Phaseout | 3 | 5% |
| Closedown | 24 | 40% |
| Unknown | 5 | 8% |
Study 3
Recognition of Academic Software
number and types of mentions
A mention means any occurrence of the name of the academic software in a scientific publication
Mentions to academic software (RQ3.1)
only 2 projects – GUIZMO and protopurity – were not found in searches carried out in ACM and IEEE
Uses of academic software (RQ3.1)
there are 124 “usage” mentions to a set of 26 projects
Contributions to academic software (RQ3.1)
there are 43 “contribute” mentions to a set of 17 projects, that is, only 28% of projects received source code contributions from studies after its initial publication
Discussion and Conclusions
Evolution of the recognition of academic software for static analysis published in ASE and SCAM
Growth of 38% per year in the number of mentions
from 60 academic software projects of static analysis studied, 13 projects (22%) did not have academic recognition
Influence of the life cycle stage on the recognition of academic software
large number of mentions for close-down projects – ESBMC, PARSEWeb, and TestEra – including recent publications between 2016 and 2017
Academic software projects are not yet recognized by Science as first-class citizens
Carole Goble. 2014. Better software, better research.
Future Work
- Why some tools in closedown stage continue to be cited?
- The results found regarding academic software recognition are similar to general software engineering publications?
Obrigado!
joenio@joenio.me
Esta apresentação está disponível em:
http://joenio.me/sustainability-academic-software
(código-fonte: http://github.com/joenio/joenio.me)
Histórico de apresentações
Onde e quando esta apresentação foi realizada
- 21 Setembro 2018, USP-ICMS, São Carlos, SP, CBSOFT SBES 2018