Description
PR #5905 (and previously #5903, #5892, and #5886) modified the benchmarking code to use a production-like evaluator for the bls12-381 and Marlowe benchmarks. Unexpectedly this appeared to increase the time of the validation benchmarks by about 2%. Experimentation suggests that the validation times are sensitive to the number and order of the exports in PlutusBenchmark.Common. The functions evaluateCekForBench
and evaluateCekLikeInProd
were exported from this file but are unused elsewhere; when the first of these was removed the validation benchmarks pretty much went back to where they were (and this is the version that was merged), but if the second is also removed the second then the 2% slowdown reappears. There were other examples where permuting the exports more or less randomly gave an overall slowdown of 4.6%.
It's not clear what's causing this: one possibility is changes in inlining.