Analyzing the Effectiveness of a System Testing Tool for Software Product Line Engineering

Supplemental Material


Participants Characterization

Division of Participants into Groups:
Group 1 | Group 2

Subjects' Profile - Experiment

ID English Reading Participation in industrial projects Experience (in months) Testing Tools *
Development Testing Programming Java Testing SPL
1 advanced No No 48 12 6 18 A
4 intermediate No No 204 100 40 24 A
5 advanced Yes Yes 78 24 42 24 A, F
6 advanced No No 30 12 6 12 B
10 advanced Yes No 12 6 4 24 A
12 advanced No Yes 30 12 12 12 C
2 advanced Yes No 48 28 6 36 A
3 intermediate Yes No 31 15 4 12 -
7 intermediate Yes No 180 60 24 9 A
8 advanced Yes Yes 24 12 24 36 C
9 intermediate Yes Yes 36 10 36 12 -
11 intermediate No No 24 6 6 18 -

(*) Testing Tools: [A] JUnit | [B] NUnit | [C] RSpec | [D] Test::Unit | [E] Selenium | [F] Other

Subjects' Profile - Replications

ID English Reading Participation in industrial projects Experience (in months) Testing Tools *
Development Testing Programming Java Testing SPL
1 advanced No No 24 10 0 0 -
2 intermediate Yes No 72 24 3 12 -
5 intermediate No No 24 9 3 0 -
8 intermediate No No 24 9 12 6 -
10 advanced Yes Yes 24 11 2 0 A
13 intermediate No No 24 10 0 0 -
14 intermediate No No 18 9 2 0 -
15 advanced No Yes 48 2 2 0 A
17 intermediate No No 24 0 5 1 -
18 basic Yes No 24 6 0 2 -
3 intermediate No No 18 9 2 0 -
4 intermediate No No 24 10 2 0 -
6 advanced Yes Yes 72 20 6 0 B
7 intermediate No No 24 9 2 0 -
9 advanced No No 24 9 2 0 -
11 intermediate No No 24 9 3 0 -
12 intermediate No No 20 10 2 0 -
16 basic Yes No 32 24 6 0 -
19 intermediate No Yes 36 12 4 0 -

(*) Testing Tools: [A] JUnit | [B] NUnit | [C] RSpec | [D] Test::Unit | [E] Selenium | [F] Other