The browser you are using is not supported by this website. All versions of Internet Explorer are no longer supported, either by us or Microsoft (read more here: https://www.microsoft.com/en-us/microsoft-365/windows/end-of-ie-support).

Please use a modern browser to fully experience our website, such as the newest versions of Edge, Chrome, Firefox or Safari etc.

Evaluation of Traceability Recovery in Context: A Taxonomy for Information Retrieval Tools

På svenska

Author

Editor

  • Teresa Baldassarre
  • Marcela Genero
  • Emilia Mendes
  • Mario Piattini

Summary, in English

Background: Development of complex, software intensive systems generates large amounts of information. Several researchers have developed tools implementing information retrieval (IR) approaches to suggest traceability links among artifacts. Aim: We explore the consequences of the fact that a majority of the evaluations of such tools have been focused on benchmarking of mere tool output. Method: To illustrate this issue, we have adapted a framework of general IR evaluations to a context taxonomy specifically for IR-based traceability recovery. Furthermore, we evaluate a previously proposed experimental framework by conducting a study using two publicly available tools on two datasets originating from development of embedded software systems. Results: Our study shows that even though both datasets contain software artifacts from embedded development, the characteristics of the two datasets differ considerably, and consequently the traceability outcomes. Conclusions: To enable replications and secondary studies, we suggest that datasets should be thoroughly characterized in future studies on traceability recovery, especially when they can not be disclosed. Also, while we conclude that the experimental framework provides useful support, we argue that our proposed context taxonomy is a useful complement. Finally, we discuss how empirical evidence of the feasibility of IR-based traceability recovery can be strengthened in future research.

Publishing year

2012

Language

English

Pages

111-120

Publication/Series

16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012)

Document type

Conference paper

Publisher

IEEE - Institute of Electrical and Electronics Engineers Inc.

Topic

  • Computer Science

Keywords

  • empirical software engineering
  • traceability recovery
  • information retrieval
  • experiment

Conference name

16th International Conference on Evaluation & Assessment in Software Engineering

Conference date

2012-05-14

Conference place

Ciudad Real, Spain

Status

Published

Project

  • Embedded Applications Software Engineering

ISBN/ISSN/Other

  • ISBN: 978-1-84919-541-6