This repository contains the evaluation results of PCART, assessing its API compatibility detection and repair capabilities across different datasets and scenarios.
-
RQ1_and_RQ2/
Evaluates PCART's performance on API parameter compatibility detection and repair.- Uses the PCBench and MLCatchUp datasets.
- Compares PCART against MLCatchUp and Relancer.
- Evaluates PCART's ability to extract complex API invocation patterns
-
RQ3/
Assesses PCART's effectiveness in real-world Python projects.- Evaluated on 30 real projects collected from GitHub.
-
RQ4/
Compares ChatGPT (GPT-4o) vs. PCART on API compatibility detection and repair.- Uses 58 test cases from PCBench (29 compatible, 29 incompatible).
-
RQ5/
Measures PCART's runtime efficiency.- Computes the average runtime per API processed across test cases.
- Each directory contains corresponding experiment results and analysis.
- For detailed evaluation metrics and results, refer to the PCART research paper:
π PCART: Automated Repair of Python API Parameter Compatibility Issues
π Happy Researching with PCART! π
PCART-evaluation is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). See LICENSE for details.