Skip to content

Update run_rc_validation_python_yaml.yml #34675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

liferoad
Copy link
Contributor

@liferoad liferoad commented Apr 18, 2025

TESTED=https://github.com/apache/beam/actions/runs/14542339938/job/40802547165

This error is expected with Beam 2.64.0:

Error message from worker: generic::unknown: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 313, in _execute
    response = task()
               ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 388, in <lambda>
    lambda: self.create_worker().do_instruction(request), request)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 658, in do_instruction
    return getattr(self, request_type)(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 689, in process_bundle
    bundle_processor = self.bundle_processor_cache.get(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/sdk_worker.py", line 512, in get
    processor = bundle_processor.BundleProcessor(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1133, in __init__
    self.ops = self.create_execution_tree(self.process_bundle_descriptor)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1193, in create_execution_tree
    get_operation(transform_id))) for transform_id in sorted(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1038, in wrapper
    result = cache[args] = func(*args)
                           ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1168, in get_operation
    tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
          ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1038, in wrapper
    result = cache[args] = func(*args)
                           ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1168, in get_operation
    tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
          ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1038, in wrapper
    result = cache[args] = func(*args)
                           ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1178, in get_operation
    return transform_factory.create_operation(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1498, in create_operation
    return creator(self, transform_id, transform_proto, payload, consumers)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1831, in create_par_do
    return _create_pardo_operation(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1876, in _create_pardo_operation
    dofn_data = pickler.loads(serialized_fn)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/internal/pickler.py", line 50, in loads
    return desired_pickle_lib.loads(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/internal/cloudpickle_pickler.py", line 93, in loads
    unpickled = cloudpickle.loads(s)
                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/apache_beam/yaml/__init__.py", line 18, in <module>
    from apache_beam.yaml.yaml_transform import *
  File "/usr/local/lib/python3.12/site-packages/apache_beam/yaml/yaml_transform.py", line 37, in <module>
    from apache_beam.yaml import yaml_provider
  File "/usr/local/lib/python3.12/site-packages/apache_beam/yaml/yaml_provider.py", line 44, in <module>
    import clonevirtualenv
ModuleNotFoundError: No module named 'clonevirtualenv'

passed through:
==>
    dist_proc/dax/workflow/worker/fnapi_service_impl.cc:1336

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@github-actions github-actions bot added the build label Apr 18, 2025
@liferoad liferoad requested a review from Abacn April 18, 2025 22:35
Copy link
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant