Skip to content

[SPARK-52423][CONNECT] Add missing golang package name to pipelines.proto #51126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

grundprinzip
Copy link
Contributor

@grundprinzip grundprinzip commented Jun 9, 2025

What changes were proposed in this pull request?

The newly added pipelines.proto file is missing the golang package identifier that is needed for the Spark Connect Go client.

Why are the changes needed?

Spark 4.0 compatibility

Does this PR introduce any user-facing change?

No

How was this patch tested?

Manually tested as part of apache/spark-connect-go#142.

Was this patch authored or co-authored using generative AI tooling?

No

@LuciferYang
Copy link
Contributor

Could you create a Jira ticket for this?

@grundprinzip
Copy link
Contributor Author

Could you create a Jira ticket for this?

Sure can do this. I was just trying to avoid the overhead of creating a JIRA ticket that needs 5 lines of descriptions for a one line change.

@grundprinzip grundprinzip changed the title [ADHOC]Add missing golang package name to pipelines.proto [SPARK-52423]Add missing golang package name to pipelines.proto Jun 9, 2025
@LuciferYang

This comment was marked as outdated.

@HyukjinKwon HyukjinKwon changed the title [SPARK-52423]Add missing golang package name to pipelines.proto [SPARK-52423][CONNECT]\ Add missing golang package name to pipelines.proto Jun 9, 2025
@HyukjinKwon HyukjinKwon changed the title [SPARK-52423][CONNECT]\ Add missing golang package name to pipelines.proto [SPARK-52423][CONNECT] Add missing golang package name to pipelines.proto Jun 9, 2025
@grundprinzip
Copy link
Contributor Author

Additionally, what does [ADHOC] refer to? It seems that this module tag hasn't appeared in Spark before.

Adhoc refers to an ad-hoc change, similar to [MINOR] or [FIX] etc.

@grundprinzip
Copy link
Contributor Author

It would be great to merge this into the 4.0 branch for the next maintenance release. This way dependencies using the release branches will receive the update as well. WDYT?

@LuciferYang
Copy link
Contributor

LuciferYang commented Jun 9, 2025

Protobuf breaking change detection and Python CodeGen check failed

@github-actions github-actions bot added the PYTHON label Jun 9, 2025
@grundprinzip
Copy link
Contributor Author

Thanks! I regenerated the Python files. I didn't expect the Python files referencing the Go package.

@grundprinzip
Copy link
Contributor Author

merging to master and 4.0

@LuciferYang
Copy link
Contributor

merging to master and 4.0

Has it really been merged? I couldn't find SPARK-52423 in the commit log of the master

@grundprinzip
Copy link
Contributor Author

No, I wasn't able to merge it because of conflicts in 4.0. Will merge to master only and then cherry-pick to 4.0.

grundprinzip added a commit to grundprinzip/spark that referenced this pull request Jun 11, 2025
…roto

The newly added `pipelines.proto` file is missing the golang package identifier that is needed for the Spark Connect Go client.

Spark 4.0 compatibility

No

Manually tested as part of apache/spark-connect-go#142.

No

Closes apache#51126 from grundprinzip/missing-golang-package.

Authored-by: Martin Grund <[email protected]>
Signed-off-by: Martin Grund <[email protected]>
@grundprinzip
Copy link
Contributor Author

Actually, the pipelines proto is not part of 4.0, so this is not a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants