|
1 |
| -# kaprese |
| 1 | +# kaprese |
| 2 | + |
| 3 | +Collection of APR tools by SAL (Software Analysis Lab) at Korea University. |
| 4 | + |
| 5 | +## Quick Start |
| 6 | + |
| 7 | +You can run `kaprese` by following the steps below. |
| 8 | + |
| 9 | +### Prerequisites |
| 10 | + |
| 11 | +`kaprese` requires the following software to be installed: |
| 12 | + |
| 13 | +- Python 3.12 or higher (required) |
| 14 | +- Docker 24 or higher (recommended, we did not test it on other versions) |
| 15 | + |
| 16 | +### Installation |
| 17 | + |
| 18 | +`kaprese` can be installed via `pip`: |
| 19 | + |
| 20 | +```bash |
| 21 | +pip install kaprese |
| 22 | +``` |
| 23 | + |
| 24 | +### Usage |
| 25 | + |
| 26 | +You can run `saver` engine on `flint-1` benchmark as follows: |
| 27 | + |
| 28 | +```bash |
| 29 | +kaprese benchmark preset c |
| 30 | +kaprese engine preset saver |
| 31 | +kaprese run -b flint-1 spearmint-1 -e saver |
| 32 | +# See the logs. If you want to see more logs set LOV_LEVEL environment variable to INFO, e.g., |
| 33 | +# LOG_LEVEL=info kaprese run -b flint-1 spearmint-1 -e saver |
| 34 | +``` |
| 35 | + |
| 36 | +## Benchmarks and Engines |
| 37 | +`kaprese` provides a set of benchmarks and engines. |
| 38 | +You can register them by calling `kaprese benchmark preset <language>` and `kaprese engine preset <engine>`. |
| 39 | +The following table shows the list of supported benchmarks and engines. |
| 40 | + |
| 41 | +### Benchmarks |
| 42 | +| Benchmarks | Repository | |
| 43 | +| :--------- | :---------------------------------------------------------------------------------------- | |
| 44 | +| `c` | C benchmarks in [kupl/starlab-benchmarks](https://github.com/kupl/starlab-benchmarks) | |
| 45 | +| `ocaml` | OCaml benchmarks in [kupl/starlab-benchmarks](https://github.com/kupl/starlab-benchmarks) | |
| 46 | + |
| 47 | +### Engines |
| 48 | +| Engines | Repository | |
| 49 | +| :------ | :-------------------------------------------------------- | |
| 50 | +| `saver` | [kupl/SAVER_public](https://github.com/kupl/Saver_public) | |
| 51 | +| `cafe` | [kupl/LearnML](https://github.com/kupl/LearnML) | |
| 52 | + |
| 53 | +## How to Add Your Own |
| 54 | + |
| 55 | +All benchmarks and engines are registered to the global registry (by default, registry is located in `~/.kaprese`). |
| 56 | +You can add your own benchmarks and engines by registering them with a small python code. |
| 57 | + |
| 58 | +### Add a Benchmark |
| 59 | +A benchmark is an instance of `kaprese.core.benchmark.Benchmark`. |
| 60 | +You can initialize a benchmark with the following 4 parameters: |
| 61 | +- `name`: name of the benchmark (must be unique). |
| 62 | +- `image`: docker image name, e.g., `ghcr.io/kupl/starlab-benchmarks/c:flint-1`. |
| 63 | +- `language_command` or `_language`: command to find the language of the benchmark, if you set `_language`, `language_command` is ignored. |
| 64 | + the command will be run inside the docker container from `image`. |
| 65 | +- `workdir_command` or `_workdir`: command to find the working directory of the benchmark, if you set `_workdir`, `workdir_command` is ignored. |
| 66 | + the command will be run inside the docker container from `image`. |
| 67 | +Then by calling `register` method of `Benchmark`, you can register the benchmark to the global registry. |
| 68 | + |
| 69 | +For example, the following code defines a benchmark named `flint-1`: |
| 70 | +```python |
| 71 | +from kaprese.core.benchmark import Benchmark |
| 72 | + |
| 73 | +flint_1 = Benchmark( |
| 74 | + name="flint-1", |
| 75 | + image="ghcr.io/kupl/starlab-benchmarks/c:flint-1", |
| 76 | + language_command="cat metadata.json | jq -r .language", # c |
| 77 | + workdir_command="cd $(cat metadata.json | jq -r .buggyPath) && pwd", # /workspace/buggy |
| 78 | +) |
| 79 | +flint_1.register() |
| 80 | + |
| 81 | +``` |
| 82 | + |
| 83 | +### Add an Engine |
| 84 | +An engine is an instance of `kaprese.core.engine.Engine`. |
| 85 | +You can initialize an engine with the following 6 parameters: |
| 86 | +- `name`: name of the engine (must be unique). |
| 87 | +- `supported_languages`: list of supported languages, e.g., `["c", "java"]`. |
| 88 | +- `supported_os`: list of supported operating systems, e.g., `["ubuntu:20.04", "debian:12"]`. |
| 89 | +- `location`: location to the docker context directory of the engine, e.g., `~/saver`. |
| 90 | +- `build_args`: arguments to pass to `docker build` command to fill `ARG` variables in the Dockerfile, e.g., `{"SOME_ARG": "some_value"}`. |
| 91 | +- `exec_commands`: list of commands to execute the engine, e.g., `["saver ...", "cp ..."]`, |
| 92 | + all commands are joined with `;`, in other words, all commands are executed and the return code of the last command is returned. |
| 93 | +Note that, your dockerfile **MUST** gets the benchmark image as an argument and starts build from it, e.g., |
| 94 | +```dockerfile |
| 95 | +ARG BENCHMARK_IMAGE |
| 96 | +FROM ${BENCHMARK_IMAGE} |
| 97 | +... |
| 98 | +``` |
| 99 | +In the same way, you **MUST** specify the benchmark image as an argument when you build the engine image, e.g., |
| 100 | +```python |
| 101 | +from kaprese.core.engine import Engine |
| 102 | + |
| 103 | +saver = Engine( |
| 104 | + "saver", |
| 105 | + supported_languages=["c"], |
| 106 | + supported_os=["ubuntu:20.04"], |
| 107 | + location="https://github.com/kupl/kaprese-engines#main:context/saver/starlab-benchmarks", # location of saver context for preset benchmarks |
| 108 | + build_args={"BENCHMARK_IMAGE": "ghcr.io/kupl/starlab-benchmarks/c:flint-1"}, |
| 109 | + exec_commands= ["saver ...", "cp ..."], # see kaprese/engines/saver.py |
| 110 | +) |
| 111 | +``` |
| 112 | +Then by calling `register` method of `Engine`, you can register the engine to the global registry. |
| 113 | +``` |
| 114 | +saver.register() |
| 115 | +``` |
| 116 | + |
| 117 | +## How to Contribute |
| 118 | +Any contributions are welcome! |
| 119 | +You can leave an [issue](https://github.com/kupl/kaprese/issues) or make a [pull request](https://github.com/kupl/kaprese/pulls). |
0 commit comments