Skip to content

Validation benchmark times vary with order/number of exports in `PlutusBenchmark.Common #5906

Open
@kwxm

Description

@kwxm

PR #5905 (and previously #5903, #5892, and #5886) modified the benchmarking code to use a production-like evaluator for the bls12-381 and Marlowe benchmarks. Unexpectedly this appeared to increase the time of the validation benchmarks by about 2%. Experimentation suggests that the validation times are sensitive to the number and order of the exports in PlutusBenchmark.Common. The functions evaluateCekForBench and evaluateCekLikeInProd were exported from this file but are unused elsewhere; when the first of these was removed the validation benchmarks pretty much went back to where they were (and this is the version that was merged), but if the second is also removed the second then the 2% slowdown reappears. There were other examples where permuting the exports more or less randomly gave an overall slowdown of 4.6%.

It's not clear what's causing this: one possibility is changes in inlining.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions