Python: Add Batch support #3555

BoazBD · 2025-04-07T07:26:20Z

Issue link

This Pull Request is linked to issue (URL): 3546

Checklist

Before submitting the PR make sure the following are checked:

This Pull Request is related to one issue.
Commit message has a detailed description of what changed and why.
Tests are added or updated.
CHANGELOG.md and documentation files are updated.
Destination branch is correct - main or release
Create merge commit if merging release branch into main, squash otherwise.

Yury-Fridlyand · 2025-04-07T15:27:19Z

Please don't forget changelog

shohamazon

Great job 🚀

shohamazon · 2025-04-20T08:24:37Z

CHANGELOG.md

@@ -1,5 +1,5 @@
 #### Changes
-
+* Python: Add python pipeline support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))


Suggested change

* Python: Add python pipeline support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))

* Python: Add Batches support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))

shohamazon · 2025-04-20T08:26:14Z

python/dev_requirements.txt

+sphinx-rtd-theme
+deprecated >= 1.2.18


Suggested change

deprecated >= 1.2.18

deprecated >= 1.2.0

Suggested change

deprecated >= 1.2.18

deprecated

maybe, u should check whats best

python/python/glide/__init__.py

python/python/glide/async_commands/batch.py

shohamazon · 2025-04-20T08:34:44Z

python/python/glide/async_commands/batch.py

-        >>> transaction = BaseTransaction()
+    Transaction vs Pipeline:
+        - Transactions (is_atomic=True) ensure that all commands are executed atomically.
+          In a transaction, all keys must belong to the same slot. This means if any key is in a different slot,


In a transaction, all keys must belong to the same slot. - this is not the place for this doc, it should be in ClusterBatch

all this section can be removed

shohamazon · 2025-04-22T09:02:14Z

python/python/tests/test_batch.py

+
+        for _, info in result[0].items():
+            assert isinstance(info, bytes)
+            assert config_key in info.decode()


confiig_key: config_value in info.decode()

shohamazon · 2025-04-22T09:04:16Z

python/python/tests/test_batch.py

+
+        pipeline.get(key)
+
+        with pytest.raises(RequestError):


check the error message

shohamazon · 2025-04-22T09:04:42Z

python/python/tests/test_batch.py

+        assert result[4] == b"value1"
+        assert result[5] == b"value2"
+
+        assert isinstance(result[2], Exception)


Suggested change

assert isinstance(result[2], Exception)

assert isinstance(result[2], RequestError)

shohamazon · 2025-04-22T09:04:57Z

python/python/tests/test_batch.py

+        assert isinstance(result[2], Exception)
+        assert "wrong number of arguments" in str(result[2])
+
+        assert isinstance(result[3], Exception)


Suggested change

assert isinstance(result[3], Exception)

assert isinstance(result[3], RequestError)

shohamazon · 2025-04-22T09:05:22Z

python/python/tests/test_batch.py

+
+    @pytest.mark.parametrize("cluster_mode", [True, False])
+    @pytest.mark.parametrize("protocol", [ProtocolVersion.RESP2, ProtocolVersion.RESP3])
+    async def test_pipeline_timeout(self, glide_client: TGlideClient):


merge this with the transaction test and call it test batch timeout

Yury-Fridlyand · 2025-04-23T03:17:18Z

.github/workflows/python.yml

@@ -253,6 +253,7 @@ jobs:
              run: |
                  python -m pip install --upgrade pip
                  pip install sphinx sphinx-rtd-theme
+                  pip install -r dev_requirements.txt


Please note changes coming from #3541

Yury-Fridlyand · 2025-04-23T03:19:27Z

python/python/tests/test_batch.py

-            ]
-        )
-        transaction.fcall(func_name, [], arguments=["one", "two"])
+        if batch.is_atomic:


Why? Are there any restrictions for function commands in pipelines?

its because function commands are routed to all nodes, so the response is different

Signed-off-by: BoazBD <[email protected]>

BoazBD requested a review from a team as a code owner April 7, 2025 07:26

BoazBD requested a review from Copilot April 7, 2025 07:26

This comment was marked as off-topic.

Sign in to view

BoazBD force-pushed the boaz/pipelines-pythin-python-wrapper branch from d4fb032 to 41dc46b Compare April 7, 2025 07:35

Yury-Fridlyand added python Python wrapper breaking breaking changes labels Apr 7, 2025

BoazBD force-pushed the boaz/pipelines-pythin-python-wrapper branch 6 times, most recently from bae1fb6 to 76dda70 Compare April 10, 2025 14:27

BoazBD requested a review from shohamazon April 15, 2025 13:52

BoazBD changed the title ~~Add python pipeline support~~ Python: Add Batch support Apr 20, 2025

shohamazon mentioned this pull request Apr 20, 2025

[Task] Python: add pipelines / batches wrapper support #3544

Open

shohamazon reviewed Apr 22, 2025

View reviewed changes

Yury-Fridlyand reviewed Apr 23, 2025

View reviewed changes

Yury-Fridlyand mentioned this pull request Apr 23, 2025

C#: batch #3611

Open

20 tasks

BoazBD and others added 9 commits April 28, 2025 14:09

add python pipeline support

a452889

Signed-off-by: BoazBD <[email protected]>

shorten rows for flake8

8e29ac6

Signed-off-by: BoazBD <[email protected]>

add transaction timeout test and allign docs with java

196ded5

Signed-off-by: BoazBD <[email protected]>

fix linters

a482853

Signed-off-by: BoazBD <[email protected]>

fix more linters and failed tests

54db06e

Signed-off-by: BoazBD <[email protected]>

add requirements download to sphinx

14736aa

Signed-off-by: BoazBD <[email protected]>

allign with lior's url changes

6c306a0

Signed-off-by: BoazBD <[email protected]>

fix imports sort

a02e743

Signed-off-by: BoazBD <[email protected]>

fix shoham comments 1

d898b8e

Signed-off-by: BoazBD <[email protected]>

BoazBD force-pushed the boaz/pipelines-pythin-python-wrapper branch from 0fe1f67 to d898b8e Compare April 28, 2025 14:11

BoazBD added 2 commits April 28, 2025 14:17

fix docs indentation

4efde29

Signed-off-by: BoazBD <[email protected]>

fix imports sort

d86c438

Signed-off-by: BoazBD <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Add Batch support #3555

Python: Add Batch support #3555

BoazBD commented Apr 7, 2025 •

edited

Loading

This comment was marked as off-topic.

Yury-Fridlyand commented Apr 7, 2025

shohamazon left a comment

shohamazon Apr 20, 2025

shohamazon Apr 20, 2025

shohamazon Apr 20, 2025

shohamazon Apr 20, 2025

shohamazon Apr 20, 2025

shohamazon Apr 22, 2025

shohamazon Apr 22, 2025

shohamazon Apr 22, 2025

shohamazon Apr 22, 2025

shohamazon Apr 22, 2025

Yury-Fridlyand Apr 23, 2025

Yury-Fridlyand Apr 23, 2025

shohamazon Apr 27, 2025

		@@ -1,5 +1,5 @@
		#### Changes

		* Python: Add python pipeline support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))

	* Python: Add python pipeline support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))
	* Python: Add Batches support ([#3555](https://github.com/valkey-io/valkey-glide/pull/3555))

	assert isinstance(result[2], Exception)
	assert isinstance(result[2], RequestError)

	assert isinstance(result[3], Exception)
	assert isinstance(result[3], RequestError)

Python: Add Batch support #3555

Are you sure you want to change the base?

Python: Add Batch support #3555

Conversation

BoazBD commented Apr 7, 2025 • edited Loading

Issue link

Checklist

This comment was marked as off-topic.

Yury-Fridlyand commented Apr 7, 2025

shohamazon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BoazBD commented Apr 7, 2025 •

edited

Loading