Detecting Inconsistencies in Software Architecture Documentation Using Traceability Link Recovery

by Jan Keim , Sophie Corallo , Dominik Fuchß , and Anne Koziolek

Published at the 20th IEEE International Conference on Software Architecture (ICSA 2023), March 13-17 2023.

Additional presentation at the Software Engineering 2024 (SE24), the symposium of the German Computer Science Societiy (Gesellschaft für Informatik (GI)) together with the Austrian Computer Society.

Approach Overview

Abstract

Documenting software architecture is important for a system’s success. Software architecture documentation (SAD) makes information about the system available and eases comprehensibility. There are different forms of SADs like natural language texts and formal models with different benefits and different purposes. However, there can be inconsistent information in different SADs for the same system. Inconsistent documentation then can cause flaws in development and maintenance. To tackle this, we present an approach for inconsistency detection in natural language SAD and formal architecture models. We make use of traceability link recovery (TLR) and extend an existing approach. We utilize the results from TLR to detect unmentioned (i.e., model elements without natural language documentation) and missing model elements (i.e., described but not modeled elements). In our evaluation, we measure how the adaptations on TLR affected its performance. Moreover, we evaluate the inconsistency detection. We use a benchmark with multiple open source projects and compare the results with existing and baseline approaches. For TLR, we achieve an excellent F1-score of 0.81, significantly outperforming the other approaches by at least 0.24. Our approach also achieves excellent results (accuracy: 0.93) for detecting unmentioned model elements and good results for detecting missing model elements (accuracy: 0.75). These results also significantly outperform competing baselines. Although we see room for improvements, the results show that detecting inconsistencies using TLR is promising.

Cite this paper

  1. Detecting Inconsistencies in Software Architecture Documentation Using Traceability Link Recovery
    Jan Keim, Sophie Corallo, Dominik Fuchß, and Anne Koziolek
    In 2023 IEEE 20th International Conference on Software Architecture (ICSA), 2023
    DOI 
    @inproceedings{keim_detecting_2023,
  author = {Keim, Jan and Corallo, Sophie and Fuchß, Dominik and Koziolek, Anne},
  booktitle = {2023 IEEE 20th International Conference on Software Architecture (ICSA)},
  title = {Detecting Inconsistencies in Software Architecture Documentation Using Traceability Link Recovery},
  year = {2023},
  volume = {},
  number = {},
  pages = {141-152},
  doi = {10.1109/ICSA56044.2023.00021},
}