Indexado en

Licencia y uso

Citaciones

Altmetrics

Grant support

This research was funded by grant PID2022-137846NB-I00 funded by MCIN/AEI/10.13039/501100011033, by "ERDF A way of making Europe" and the FINESSE project, Spain (PID2021-122270OB-I00) . This research was also supported by the Madrid Region R & D program, Spain (project FORTE, P2018/TCS-4314) and the SATORI-UAM project (TED2021-129381B-C21) .

Análisis de autorías institucional

Compartir

Reliability of systematic literature reviews on test-driven development

Publicado en:INFORMATION AND SOFTWARE TECHNOLOGY 184 (): 107762- - 2025-08-01 184(), DOI: 10.1016/j.infsof.2025.107762

Autores: Uyaguari, Fernando; Acuna, Silvia T; Castro, John W; Dieste, Oscar; Juristo, Natalia

Afiliaciones

Resumen

Context: Test-driven development (TDD) is a software development technique studied empirically over the last few decades. There are several systematic literature reviews (SLRs) on TDD. The reliability of these studies should not be taken for granted because SLRs are highly dependent on the context and researcher decisionmaking. Objective: This study determines, analyses and synthesizes the limited overlap between SLRs on TDD and its influence on the conclusions and results with respect to the code quality and developer productivity response variables. Method: A tertiary study was conducted to source SLRs on TDD from the scientific literature, and the primary studies referenced in each SLR were analysed. We compared SLRs with similar objectives, SLRs with similar response variables, and all SLRs. We analysed the differences between the selected primary studies and their impact on the conclusions and results. Results: The overlap between SLRs with similar response variables (54 %) is greater than between SLRs with similar objectives (36 %). Only three per cent of the primary studies are included in all eight analysed SLRs. Conclusions regarding external quality and productivity may vary across the SLRs on TDD. While we found that SLR results are similar, these results may differ when authors classify primary studies by experiments and case studies. Conclusion: SLRs with similar response variables tend to be more repeatable than SLRs with similar objectives and SLRs addressing the same topic. The SLR authors' criteria with respect to the consistency of evidence may influence the conclusions of SLRs on TDD. The results of SLRs where all primary studies count equally appear to be consistent. The SLR authors' criteria for selecting primary studies may influence the results classified by case studies and experiments.

Palabras clave