Defects4J Workability for APR Results

Results for workability setup-test and trivial patch experiments on Defects4J version 2.0.

setup-test-results.csv contains the result of the setup-test algorithm, executed 20 times on the full Defects4J dataset. The first rows contain meta-information on the runs: which of two different hosts for the VM was used, the parallelism setting, the aggregated (counted) results, and the total runtime (wall time) for the complete run.

setup-test-run-*.json contain the raw output of the runs. The format is a list of objects for each bug in the dataset. Each such execution has the following fields:

• id
  
• executionIndex
  
• commandLine (list of the command-line to launch the tool)
  
• exitStatus (of the tool process)
  
• result (the result object, as documented in the README of the geneseer tool)
  
• logOutput
  
• started (timestamp)
  
• ended (timestamp)

The result field can be mapped to the values seen in the csv file via the following jq command-line:

jq -r '.[]|[if((.result.exception|type)=="string") then if(.result.exception|contains(" differs when run with coverage"))or(.result.exception|contains("Did not get single result when running test "))or(.result.exception|contains("Got duplicate test names in test suite result"))or(.result.exception|contains(" timed out when run with coverage")) then "INCONSISTENT_TEST_SUITE" elif(.result.exception|contains("net.ssehub.program_repair.geneseer.evaluation.CompilationException")) then "COMPILATION_ERROR" else "UNCLASSIFIED: "+.result.exception end else .result.result//.exitStatus end]|@csv'

setup-test-without-classpath-fixes.csv contains the same as setup-test-results.csv information, but for a single run when not applying the custom fixes for the classpaths as exported by Defects4J. setup-test-without-classpath-fixes.json contains the raw output for this execution.

geneseer-ease-rene-2026.zip contains the source code of the genseer tool. geneseer.jar contains the compiled tool with all dependencies included (geneseer-no-d4j-fixes.jar contains the compiled program with all dependencies included and the classpath fixes disabled).

The file jgenprog-results.csv contains which defects in Defects4J 1.0 were reported to be fixed by jGenProg in M. Martinez and M. Monperrus, “Astor: Exploring the design space of generate-and-validate program repair beyond GenProg,” Journal of Systems and Software, vol. 151, pp. 65–80, May 2019, as well as whether they are also included in Defects4J 2.0.

The file single-statement-deletion-results.csv contains the results of the experiment that deletes sing