Skip to content

KAFKA-19018,KAFKA-19063: Implement maxRecords and acquisition lock timeout in share fetch request and response resp. #19334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 1, 2025

Conversation

apoorvmittal10
Copy link
Contributor

PR add MaxRecords to share fetch request and also adds AcquisitionLockTimeout to share fetch response. PR also removes internal broker config of max.fetch.records.

@github-actions github-actions bot added triage PRs from the community core Kafka Broker consumer performance KIP-932 Queues for Kafka clients labels Apr 1, 2025
@apoorvmittal10 apoorvmittal10 added ci-approved and removed performance triage PRs from the community labels Apr 1, 2025
…meout in share fetch request and response resp.
Copy link
Member

@AndrewJSchofield AndrewJSchofield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Some initial comments. I'll take another pass soon.

@@ -39,6 +39,8 @@
"about": "The top-level response error code." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
"about": "The top-level error message, or null if there was no error." },
{ "name": "AcquisitionLockTimeout", "type": "int64", "versions": "0+",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please can you add Ms to the field name? In RPCs, the convention for timeouts is always to have Ms on the end, such as RebalanceTimeoutMs. Also, they are always int32, which is probably appropriate here too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"about": "The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored." },
"about": "The maximum bytes to fetch. See KIP-74 for cases where this limit may not be honored." },
{ "name": "MaxRecords", "type": "int32", "versions": "0+",
"about": "The maximum number of records to fetch. This is a soft limit and records can exceed upto the end of fetch log batch." },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest for the second sentence "This limit can be exceeded for alignment of batch boundaries".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@AndrewJSchofield AndrewJSchofield merged commit 4aa8120 into apache:trunk Apr 1, 2025
21 of 22 checks passed
janchilling pushed a commit to janchilling/kafka that referenced this pull request Apr 4, 2025
…meout in share fetch request and response resp. (apache#19334)

PR add `MaxRecords` to share fetch request and also adds
`AcquisitionLockTimeout` to share fetch response. PR also removes
internal broker config of `max.fetch.records`.

Reviewers: Andrew Schofield <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants