cloud benchmarks

[dberenbaum]

We need a way to benchmark in at least AWS, and ideally also in Azure and GCS.

Too often, we are missing realistic benchmarks for cloud-centric operations (for example, #9098).

Related:

• push/pull: compare with s5cmd/aws s3 cp dvc-bench#420
• cloud versioning dvc-bench#408
• Benchmark dvc gc -c dvc-bench#407

[efiop]

Discussed before that they should be part of dvc.testing

[omesser]

@dberenbaum @efiop - Can we please add this for Q2 as roadmap item?
It's also possible to get some help (from ex-CML?) with setting up automation for multi-cloud infra set-up

[dberenbaum]

Still need to prioritize, but we have had cloud performance as a priority that we were hoping to get to even in Q1, so I think it makes sense to keep it for Q2 and include benchmarking.

[efiop]

Though dvc-bench has all the clouds hot-swappable by default as well (by design), so might shove that there. I guess this is going into the same discussion we had before about whether or not dvc-bench should be part of dvc or a standalone project.

[dberenbaum]

To clarify the requirements, the most important is to benchmark a realistic cloud remote (s3) more than to individually test each remote (which is needed but not as high priority IMO).