GH-46771: [Python][C++] Implement pa.arange function to generate array sequences

[raulcd]

Rationale for this change

When slicing arrays with non-trivial steps we were using numpy.arange to generate the indices for take. As numpy is an optional dependency, implementing it via Python caused a performance penalty. Creating a pyarrow function to build our own ranges that mimics Python range or numpy arange is useful for that uses case and might also be useful for other use cases. Currently we only generate Array[Int64] we could potentially generate more types.

What changes are included in this PR?

provide a pa.arange function that allows us to generate indices when slicing arrays.

Are these changes tested?

Yes new tests added.

Are there any user-facing changes?

No but a new pyarrow.arange function has been added.

• GitHub Issue: [Python] If numpy is available use it for normalizing slice #46771

[raulcd]

I am unsure whether this is what you had in mind @pitrou but would appreciate some tips on possible improvements. I've been testing and it stills is slightly slower than the np.arange original implementation. With the original np.arange implementation:

In [2]: a = pa.array(np.arange(0, 2_000_000))

In [3]: %timeit a[slice(1, 1_000_000, 2)]
763 μs ± 4.28 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each

with the current PR one:

In [2]: a = pa.array(np.arange(0, 2_000_000))

In [3]: %timeit a[slice(1, 1_000_000, 2)]
1.08 ms ± 43.7 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

[raulcd]

I don't think this should go on src/arrow/python/helpers.cc but I am unsure on where should it belong, maybe on src/arrow/python/pyarrow.cc @pitrou what are your thoughts?

[pitrou]

Nothing obvious comes to mind. Perhaps a new file if it doesn't fit elsewhere?

[raulcd]

I've moved it to src/arrow/python/util.{h,cc}

[github-actions]

⚠️ GitHub issue #46771 has been automatically assigned in GitHub to PR creator.

[raulcd]

It seems to be on-pair now:

In [2]: %timeit np.arange(1, 1_000_000, 2)
219 μs ± 10.6 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [3]: %timeit pa.arange(1, 1_000_000, 2)
212 μs ± 1.97 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

[pitrou]

These are all cases where step is conveniently a divisor of the length, and where either start or stop is zero. Can you make these a bit more interesting and diverse?

[raulcd]

I tried to improve the tests scenarios with more interesting cases