Skip to content

Sqlstorage retry #812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 5, 2025
Merged

Sqlstorage retry #812

merged 3 commits into from
May 5, 2025

Conversation

bpfz
Copy link
Contributor

@bpfz bpfz commented May 2, 2025

Adds resilience pipeline to SqlServerPersistentStorageSettings. Default value is the same incremental retry policy used for the sql server source.

Adds two generic helper extensions to execute in the pipeline for any Task/Task<T>. Did not add a sync version as outcome only supports at Task and ValueTask.

Generally retries are added to connection.OpenAsync and transaction.CommitAsync()/transaction.RollbackAsync().

What do we think about it conceptually? A bit uncertain regarding the bulkcopy, if it's a good idea to run that in this way.

bpfz added 2 commits May 2, 2025 18:09
Introduce Polly-based resilience pipeline for retrying
database operations in `BaseSqlRepository` and
`SqlServerPersistentStorage`. Add `ResiliencePipeline`
property to `SqlServerPersistentStorageSettings` and
implement extension methods in `SqlServerStorageExtensionsHelpers`.
Update existing async operations to utilize the new
pipeline for improved error handling and retries.
@bpfz bpfz requested a review from Ulimo May 2, 2025 17:06
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark

Benchmark suite Current: 33b9686 Previous: 22d4209 Ratio
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.InnerJoin 408538740 ns (± 7964456.241947022) 458826533.3333333 ns (± 10640413.19005047) 0.89
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.LeftJoin 495851570 ns (± 20669608.827175334) 564994640 ns (± 28550338.329818636) 0.88
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ProjectionAndNormalization 148185650 ns (± 4844704.478316735) 169842820 ns (± 7144067.768715523) 0.87
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.SumAggregation 171645990 ns (± 12798089.158863608) 171581250 ns (± 12415362.812660772) 1.00
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithMapAggregation 1853463620 ns (± 131747641.62756851) 1982296640 ns (± 123349811.75190067) 0.94
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.WindowSum 377445090 ns (± 16453520.824731978)
FlowtideDotNet.Benchmarks.Stream.StreamBenchmark.ListAggWithStructAggregation 1478522330 ns (± 83089154.77579288)

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@Ulimo Ulimo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@bpfz bpfz force-pushed the sqlstorage_retry branch from c2a2c84 to 33b9686 Compare May 5, 2025 12:22
@bpfz bpfz merged commit 583f6f0 into main May 5, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants