Add scripts to run the benchmark for the web This set of scripts run the benchmark and generates graphs with the results suitable for publishing on the web. The main script is run.sh.
Add a helper script to improve accuracy when benchmarking This script put the CPU in 'performance' mode to avoid distortion in the results because of CPU frequency changes. It also gives the process maximum CPU time and I/O priority using nice and ionice respectively. Finally, it only uses the number of CPUs specifiend in the CPUS environment variable to run the program. The scripts is used like this: CPUS=2 ./bench.sh some_prog --some-args This runs "some_prog" passing "--some-args" as arguments, using 2 CPUs.
stats.py: Allow specifying input expression and output format Now the script can take an arbitrary (optional) expression as the first argument to parse the input file, using a separator (taken as the third argument) and field specification for the expression, using $1 for the first field, $2 for the second, etc. (similar to AWK). As second argument the script takes the output format, in Python format specification, with keys: min, max, mean, and std. The defaults are '$1' for the first argument (expression to parse as input), '%(min)s,%(mean)s,%(max)s,%(std)s' as output format and ',' as input separator. This defaults match the old behaviour.
Update micro benchmarks Remove some micro benchmarks that provided no added value, rename others to have shorter names and add a few; some Olden benchmarks and a couple of benchmarks to exercise concurrency.
Collect micro benchmarks run time statistics Run the micro benchmarks several times to collect timing statistics. Only the total run time is collected (in CSV format) and then the minimum, mean, maximum and standard deviation are calculated based on the collected values. Add arguments to some micro benchmarks that needed them too.
Update to work with current state of the project The naive collector is now gone, there are no statistics collection, etc. The object files and binaries are generated in separate directories now. The statistics graph generation is still there but not working, only building the benchmarks works, including dil, which is added as a submodule for simplicity (and keeping track of working version).