Skip to content

Commit 2fd22b5

Browse files
UlimoCopilot
andauthored
Preperation of release 0.13.0 (#709)
* Preperation of release 0.13.0 Added security note update and release notes in the blog. * Update docs/blog/2025-02-10-release-0-13-0/index.md Co-authored-by: Copilot <[email protected]> * removed note about stream and IBufferWriter, that is covered by minimizng allocations --------- Co-authored-by: Copilot <[email protected]>
1 parent 053ecc0 commit 2fd22b5

File tree

2 files changed

+62
-1
lines changed

2 files changed

+62
-1
lines changed

SECURITY.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ These are the versions that will currently be updated with security updates:
66

77
| Version | Supported |
88
| ------- | ------------------ |
9-
| 0.12.x | :white_check_mark: |
9+
| 0.13.x | :white_check_mark: |
10+
| 0.12.x | :x: |
1011
| 0.11.x | :x: |
1112
| 0.10.x | :x: |
1213
| 0.9.x | :x: |
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
slug: release-0-13-0
3+
title: Release 0.13.0
4+
tags: [release]
5+
---
6+
7+
## Major changes
8+
9+
### New serializer to improve serialization speed
10+
A new custom serializer has been implemented that follows the Apache Arrow serialization while minimizing extra allocations and memory copies.
11+
12+
Additionally, the default compression method was also changed from using ZLib to Zstd.
13+
This change was also made to improve serialization performance.
14+
15+
### Support for pause & resume
16+
17+
A new feature has been added to allow pausing and resuming data streams, making it easier to conduct maintenance or temporarily halt processing without losing state.
18+
19+
For more information, visit https://koralium.github.io/flowtide/docs/deployment/pauseresume.
20+
21+
### Integer column changed from 64 bits to dynamic size
22+
23+
The integer column was changed to now instead select the bit size based on the data inside of the column.
24+
This change reduces memory usage for columns with smaller integer values. Bit size is determined on a per-page basis, so pages with larger values will only use higher bit sizes when necessary.
25+
26+
### Delta Lake Support
27+
28+
This version adds support to both read and write to the Delta Lake format. This allows easy integration
29+
to data lake storage. To learn more about delta lake support, please visit: https://koralium.github.io/flowtide/docs/connectors/deltalake
30+
31+
### Custom data source & sink changed to use column based events
32+
33+
Both the custom data source and sink have now been changed to use column based events.
34+
This improves connector performance by eliminating the need to convert data between column-based and row-based formats during streaming.
35+
36+
## Minor changes
37+
38+
### Elasticsearch connector change from Nest to Elastic.Clients.Elasticsearch
39+
40+
The Elasticsearch connector has been updated from the deprecated `Nest` package to `Elastic.Clients.Elasticsearch`. This change requires stream configurations to be adjusted for the new connection settings.
41+
42+
Additionally, connection settings are now provided via a function, enabling dynamic credential management, such as rolling passwords.
43+
44+
### Add support for custom stream listeners
45+
46+
Applications can now listen to stream events like checkpoints, state changes, and failures, allowing for custom exit strategies or monitoring logic.
47+
48+
49+
Example:
50+
51+
```csharp
52+
.AddCustomOptions(s =>
53+
{
54+
s.WithExitProcessOnFailure();
55+
});
56+
```
57+
58+
### Cache lookup table for state clients
59+
60+
An internal optimization adds a small lookup table for state client page access, reducing contention on the global LRU cache. This change has shown a 10–12% performance improvement in benchmarks.

0 commit comments

Comments
 (0)