An Exploratory Study on the Usage of Gherkin Features in Open-Source Projects

Chandorkar, Adwait; Patkar, Nitish; Sorbo, Andrea Di; Nierstrasz, Oscar (March 2022). An Exploratory Study on the Usage of Gherkin Features in Open-Source Projects. In: Workshop on Validation, Analysis and Evolution of Software Tests (VST 2022, co-located with SANER 2022). 10.1109/SANER53432.2022.00134

[img]
Preview
Text
Patk22b.pdf - Accepted Version
Available under License Publisher holds Copyright.

Download (403kB) | Preview

With behavior-driven development (BDD), domain experts describe system behavior and desired outcomes through natural language-like sentences, e.g., using the Gherkin language. BDD frameworks partially convert the content of Gherkin specifications into executable test code. Previous studies have reported several issues with the current BDD practice, for example long repetitive Gherkin specifications and slow-running test suites. Data tables and additional features were added to the Gherkin syntax to express compactly test inputs (e.g., provide different combinations of input values and desired outputs to run tests multiple times) and also to improve the readability of Gherkin files (henceforth called spec files). However, there is no empirical evidence about the actual usage of these Gherkin features. To fill this gap, we analyzed the content of 1,572 spec files extracted from 23 open-source projects. For each spec file, we collected a set of metrics modeling the structure and the usage of the different Gherkin features. We found that only a minority of the considered spec files (i.e., 590) used data tables that contain two rows, on average. We also used statistical tests to compare the contents of spec files with and without data tables and found significant differences between the two populations, especially for what concerns the number of lines of code (LoC). On the one hand, our results shed some light on the discrepancies between the recommendations for defining Gherkin specifications and their actual adoption in practice. On the other hand, our findings demonstrate that the adoption of additional features, such as data tables, might only partially help to reduce the length of Gherkin specifications.

Item Type:

Conference or Workshop Item (Paper)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF)
08 Faculty of Science > Institute of Computer Science (INF) > Software Composition Group (SCG) [discontinued]

UniBE Contributor:

Patkar, Nitish Shriniwas, Nierstrasz, Oscar

Subjects:

000 Computer science, knowledge & systems

Language:

English

Submitter:

Oscar Nierstrasz

Date Deposited:

15 Aug 2022 13:58

Last Modified:

26 Jul 2023 10:48

Publisher DOI:

10.1109/SANER53432.2022.00134

Uncontrolled Keywords:

scg-pub snf-asa3 scg22 jb22 snf-imad

BORIS DOI:

10.48350/171927

URI:

https://boris.unibe.ch/id/eprint/171927

Actions (login required)

Edit item Edit item
Provide Feedback