Doc: Hint implications on use write.data.path and orphan removal together

[dramaticlly]

Today orphan removal does not respect write.data.path and I believe we shall let customer be ware of potential implications when used together. Karu did attempt to fix in #12278 but we shall at least call out the risk, this is particular important for GDPR compliance and privacy related data needs

See local build

[Fokko]

I would never recommend setting the write.data.path outside of the location path. I'm not sure if we would go down a path where we give hints on how to fix things in a way that aren't recommended. I think this is a slippery slope that we don't want to go down, for example, the hint does not include write.metadata.path, which could also lead to orphaned files.

[dramaticlly]

I would never recommend setting the write.data.path outside of the location path. I'm not sure if we would go down a path where we give hints on how to fix things in a way that aren't recommended. I think this is a slippery slope that we don't want to go down, for example, the hint does not include write.metadata.path, which could also lead to orphaned files.

Thanks @Fokko, I think that's the better way to structure this hint. We kind of get the same message from AWS S3 folks, where write.data.path and write.metadata.path allows for customization, but generally recommended to stay within table location. Do you think it would be helpful to highlight such recommendation somewhere in the documentation? I am thinking about on the configuration page https://iceberg.apache.org/docs/nightly/configuration/

[Fokko]

Thanks for the context @dramaticlly. After thinking about a bit more, I think it makes more sense to put this in the orphan files cleanup routine? Including a warning that it will delete files that are not associated with the table. My main concern was that people would use orphan file cleanup at shared locations, where it might delete data that should not be deleted.