Skip to content

Distributed Ray wr.s3.write_parquet implementation #1519

Closed
@kukushking

Description

@kukushking

Is your feature request related to a problem? Please describe.
Ensure wr.s3.write_parquet runs at scale leveraging Ray

Additional context
This is part of the Wrangler @ Scale release

Links
https://docs.ray.io/en/latest/data/key-concepts.html
https://docs.ray.io/en/latest/data/package-ref.html#creating-datasets
https://docs.ray.io/en/latest/data/package-ref.html#custom-datasource-api
https://docs.ray.io/en/latest/data/package-ref.html#built-in-datasources

P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions