usb hotplug device number changes on host not propagated to instance

[gotnone]

Is there an existing issue for this?

• There is no existing issue for this bug

Is this happening on an up to date version of Incus?

• This is happening on a supported version of Incus

Incus system details

config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- network_sriov
- console
- restrict_dev_incus
- migration_pre_copy
- infiniband
- dev_incus_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- dev_incus_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- backup_compression
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- images_all_projects
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- instances_nic_host_name
- image_copy_profile
- container_syscall_intercept_sysinfo
- clustering_evacuation_mode
- resources_pci_vpd
- qemu_raw_conf
- storage_cephfs_fscache
- network_load_balancer
- vsock_api
- instance_ready_state
- network_bgp_holdtime
- storage_volumes_all_projects
- metrics_memory_oom_total
- storage_buckets
- storage_buckets_create_credentials
- metrics_cpu_effective_total
- projects_networks_restricted_access
- storage_buckets_local
- loki
- acme
- internal_metrics
- cluster_join_token_expiry
- remote_token_expiry
- init_preseed
- storage_volumes_created_at
- cpu_hotplug
- projects_networks_zones
- network_txqueuelen
- cluster_member_state
- instances_placement_scriptlet
- storage_pool_source_wipe
- zfs_block_mode
- instance_generation_id
- disk_io_cache
- amd_sev
- storage_pool_loop_resize
- migration_vm_live
- ovn_nic_nesting
- oidc
- network_ovn_l3only
- ovn_nic_acceleration_vdpa
- cluster_healing
- instances_state_total
- auth_user
- security_csm
- instances_rebuild
- numa_cpu_placement
- custom_volume_iso
- network_allocations
- zfs_delegate
- storage_api_remote_volume_snapshot_copy
- operations_get_query_all_projects
- metadata_configuration
- syslog_socket
- event_lifecycle_name_and_project
- instances_nic_limits_priority
- disk_initial_volume_configuration
- operation_wait
- image_restriction_privileged
- cluster_internal_custom_volume_copy
- disk_io_bus
- storage_cephfs_create_missing
- instance_move_config
- ovn_ssl_config
- certificate_description
- disk_io_bus_virtio_blk
- loki_config_instance
- instance_create_start
- clustering_evacuation_stop_options
- boot_host_shutdown_action
- agent_config_drive
- network_state_ovn_lr
- image_template_permissions
- storage_bucket_backup
- storage_lvm_cluster
- shared_custom_block_volumes
- auth_tls_jwt
- oidc_claim
- device_usb_serial
- numa_cpu_balanced
- image_restriction_nesting
- network_integrations
- instance_memory_swap_bytes
- network_bridge_external_create
- network_zones_all_projects
- storage_zfs_vdev
- container_migration_stateful
- profiles_all_projects
- instances_scriptlet_get_instances
- instances_scriptlet_get_cluster_members
- instances_scriptlet_get_project
- network_acl_stateless
- instance_state_started_at
- networks_all_projects
- network_acls_all_projects
- storage_buckets_all_projects
- resources_load
- instance_access
- project_access
- projects_force_delete
- resources_cpu_flags
- disk_io_bus_cache_filesystem
- instance_oci
- clustering_groups_config
- instances_lxcfs_per_instance
- clustering_groups_vm_cpu_definition
- disk_volume_subpath
- projects_limits_disk_pool
- network_ovn_isolated
- qemu_raw_qmp
- network_load_balancer_health_check
- oidc_scopes
- network_integrations_peer_name
- qemu_scriptlet
- instance_auto_restart
- storage_lvm_metadatasize
- ovn_nic_promiscuous
- ovn_nic_ip_address_none
- instances_state_os_info
- network_load_balancer_state
- instance_nic_macvlan_mode
- storage_lvm_cluster_create
- network_ovn_external_interfaces
- instances_scriptlet_get_instances_count
- cluster_rebalance
- custom_volume_refresh_exclude_older_snapshots
- storage_initial_owner
- storage_live_migration
- instance_console_screenshot
- image_import_alias
- authorization_scriptlet
- console_force
- network_ovn_state_addresses
- network_bridge_acl_devices
- instance_debug_memory
- init_preseed_storage_volumes
- init_preseed_profile_project
- instance_nic_routed_host_address
- instance_smbios11
- api_filtering_extended
- acme_dns01
- security_iommu
- network_ipv4_dhcp_routes
- network_state_ovn_ls
- network_dns_nameservers
- acme_http01_port
- network_ovn_ipv4_dhcp_expiry
- instance_state_cpu_time
- network_io_bus
- disk_io_bus_usb
- storage_driver_linstor
- instance_oci_entrypoint
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
auth_user_name: gotnone
auth_user_method: unix
environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: REDACTED
  driver: lxc
  driver_version: 6.0.4
  firewall: nftables
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
    idmapped_mounts: "true"
    netnsid_getifaddrs: "true"
    seccomp_listener: "true"
    seccomp_listener_continue: "true"
    uevent_injection: "true"
    unpriv_binfmt: "false"
    unpriv_fscaps: "true"
  kernel_version: 6.6.84.1-microsoft-standard-WSL2+
  lxc_features:
    cgroup2: "true"
    core_scheduling: "true"
    devpts_fd: "true"
    idmapped_mounts_v2: "true"
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    network_veth_router: "true"
    pidfd: "true"
    seccomp_allow_deny_syntax: "true"
    seccomp_notify: "true"
    seccomp_proxy_send_notify_fd: "true"
  os_name: Debian GNU/Linux
  os_version: "12"
  project: default
  server: incus
  server_clustered: false
  server_event_mode: full-mesh
  server_name: T480-THINK
  server_pid: 207
  server_version: "6.11"
  storage: dir
  storage_version: "1"
  storage_supported_drivers:
  - name: dir
    version: "1"
    remote: false

Instance details

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Debian trixie amd64 (20231215_05:24)
  image.os: Debian
  image.release: trixie
  image.serial: "20231215_05:24"
  image.type: squashfs
  image.variant: default
  volatile.base_image: 49c21af4943e3d2a9032f4d76a432c4fdfcca3115200a566ee4fbd8d5243acf1
  volatile.cloud-init.instance-id: 2d023906-023b-4d37-a0e0-9a6ed9708ec4
  volatile.eth0.host_name: veth30d875ea
  volatile.eth0.hwaddr: 00:16:3e:ac:0d:ed
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: fc4552fe-b59b-4c30-90b9-2ea03c02c6e5
  volatile.uuid.generation: fc4552fe-b59b-4c30-90b9-2ea03c02c6e5
devices:
  c:
    path: /mnt/c
    source: /mnt/c
    type: disk
  daplink:
    gid: "1000"
    productid: f001
    type: usb
    uid: "1000"
    vendorid: c251
  jlink:
    gid: "1000"
    productid: "1015"
    type: usb
    uid: "1000"
    vendorid: "1366"
  jlink_vcp:
    gid: "1000"
    mode: "0666"
    path: /dev/ttyACM0
    required: "false"
    type: unix-char
    uid: "1000"
  logic:
    gid: "1000"
    mode: "0660"
    productid: 608c
    type: usb
    uid: "1000"
    vendorid: 1d50
  rs485:
    gid: "1000"
    mode: "0666"
    path: /dev/ttyUSB0
    required: "false"
    type: unix-char
  stlinkv2:
    gid: "1000"
    productid: "3748"
    type: usb
    uid: "1000"
    vendorid: "0483"
ephemeral: false
profiles:
- default
- x11
stateful: false
description: ""

Instance log

No response

Current behavior

USB devices are not properly passed to the instance for hardware that applies a firmware load or other change that causes the host to give the USB device a new device number

We will specifically look at the Fx2lafw device, but others have noted similar problems with android adb and fastboot which causes similar usb bus disconnect and reconnect events

Initial Plug In

Host:

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/usb/bus/001
001
007

Instance:

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
007

After Bus Disconnect and Reconnect

Running pulseview loads firmware on the USB device and then causes a USB bus disconnect and reconnect cycle.

Host:

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 008: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
001
008

Instance:

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 008: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
007

Problem and Workaround

Note how the /dev/bus/usb/001/007 char file remains and the char file for the new 008 device enumeration does not get propagated to the instance

One unpleasant workaround is to edit the device within the config for the instance incus config edit chibios
or to re-apply the device config for this particular device after each bus re-enumeration.

Expected behavior

USB bus re-enumerations should be passed seamlessly to instances according to the device rules in their configs

Steps to reproduce

1. Configure an instance to use a usb device with type: usb
2. Start the instance and connect an appropriate USB device to the host
3. See the device properly passed into the instance in /dev/bus/usb/*
4. Disconnect the usb device and reconnect it
5. The host gives the usb device a new device number
6. See the instance keeps the old device char file
7. Edit the instance config file or re-apply the device config to the instance
8. See the instance properly enumerate the correct device number in /dev/bus/usb/*

[stgraber]

Can you run udevadm monitor on the host while one of those operations is happening?

[gotnone]

Logfile

monitor.log.gz

Operations Logged

1. start udevadm monitor
2. start instance
3. finish start instance at time 25857.659283
4. connect usb device
5. finish first usb device connection at time 25900.902649
6. something strange happens device seems to be removed by something
7. strange thing finishes at time 25908.513097
8. look at lsusb and /dev/bus/usb/* from instance no usb bus in the instance

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 019: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 013: ID 1a86:7523 QinHeng Electronics CH340 serial converter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb
ls: cannot access '/dev/bus/usb/': No such file or directory

9. attempt to edit the instance config modifying the device entry slightly mode: 0660 -> mode: 0666
10. finish editing it appears that the log is still saying time 25908.513097
11. look at lsusb and /dev/bus/usb/* from the instance things look OK in the instance

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 019: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 013: ID 1a86:7523 QinHeng Electronics CH340 serial converter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
019

12. launch pulseview which will cause the firmware upload and subsequent usb disconnect / reconnect
13. disconnect complete at time 26539.416888
14. reconnect complete at time 26593.724486
15. something strange happens again
16. something strange finishes at time 26601.506013
17. check lsusb and /dev/bus/usb/001 from instance note the incorrect char device

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 020: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 013: ID 1a86:7523 QinHeng Electronics CH340 serial converter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
019

19. attempt to edit the instance config modifying the device entry slightly mode: 0666 -> mode: 0660
20. finish editing it appears that the log is still saying time 26601.506013
21. check lsusb and /dev/bus/usb/001 everything looks OK in the instance

$ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 020: ID 1d50:608c OpenMoko, Inc. Fx2lafw
Bus 001 Device 013: ID 1a86:7523 QinHeng Electronics CH340 serial converter
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
$ ls /dev/bus/usb/001
020

[stgraber]

I need the udevadm monitor output for ONLY the moment where the device changes bus address.

Having anything extra in the log makes it unfortunately unusable as I can't tell what event we may be able to react to to handle the situation.

[stgraber]

Ah, I see, we have some timestamps, so should be able to figure it out with that :)

[stgraber]

Can you run incus monitor --pretty while having the USB device change bus address?
I wonder if we have anything interesting in our debug log.

If that doesn't show us anything useful, we're going to have to resort to a debugging build with a bunch of debugging in the uevent handler and in the USB device hook that's receiving those events.

[gotnone]

Unfortunately nothing shows up in the incus monitor --pretty when the usb device number changes.

I did record what happens when I edit the config file to make the usb device changes propagate to the instance, but it seems like a debugging build may be in order.

monitor_pretty.log.gz

[stgraber]

It's Incus 6.11 from the Zabbly repository running on Debian 12 on amd64?

[gotnone]

Yes, but all that on Windows WSL2. It is certainly possible that there is some shenanigans caused by WSL.

This was same setup was working reasonably well with LXD 5.0.2

[stgraber]

https://dl.stgraber.org/incus-1944
7eb8b80bc2d50ad3be3f99967cb9ae5b213b01d6f269fa3a03b8953eff135f78

You'll want to:

• mv /opt/incus/bin/incusd /opt/incus/bin/incusd.orig
• curl https://dl.stgraber.org/incus-1944 -o /opt/incus/bin/incusd
• chmod +x /opt/incus/bin/incusd
• systemctl restart incus

Then trigger the issue again with incus monitor --pretty running, hopefully it will be quite noisy now. All the extra debug has the stgraber: prefix

[gotnone]

Thank you for putting this build together I logged two events:

Disconnect:
monitor_disconnect_pretty.log.gz

Reconnect:
monitor_connect_pretty.log.gz

On the reconnect there were two stages. One normal part and then a pause of a 5-10 seconds and then a second stage. I didn't catch the timestamp in the actual act, but looking at the file I see a 6 second gap with the strange second stage starting at time="2025-04-16T01:06:12-05:00" or line 584

[stgraber]

The event has all the expected fields so it should have gone through.

The only reason I can find in the code for it not being processed is if it didn't come through libudev. Can you check if your system is running udevd? Normally you should see systemd-udevd running.

root@d12:~/incus# ps aux | grep udev
root         235  0.0  0.1  26560  1492 ?        Ss   03:56   0:00 /lib/systemd/systemd-udevd

[gotnone]

Yes udevd appears to be running.

$ ps aux | grep udevd
root          60  1.3  0.0  25152  6144 ?        Ss   23:25   0:00 /lib/systemd/systemd-udevd
1000000      539  0.5  0.0  31812  9216 ?        Ss   23:25   0:00 /usr/lib/systemd/systemd-udevd
1000000      903  0.7  0.0  30572  8576 ?        Ss   23:25   0:00 /usr/lib/systemd/systemd-udevd
gotnone      986  0.0  0.0   6336  2048 pts/2    S+   23:25   0:00 grep udevd

I also tried a fresh wsl distribution (Debian 12.9) and I am still seeing the same issue there. The usb device is usable in the incus host, but trying to push it to the instance requires editing the config after the host sees the reconnect event.

$ ps aux | grep udevd
root          54  0.0  0.0  25140  6144 ?        Ss   22:54   0:00 /lib/systemd/systemd-udevd
1000000      552  0.0  0.0  31420  9216 ?        Ss   22:56   0:00 /usr/lib/systemd/systemd-udevd
gotnone     1664  0.0  0.0   6332  2176 pts/2    S+   23:24   0:00 grep udevd

It is possible that something else is going on on the wsl side or with the usbipd communication. When I updated to incus from LXD I had also updated wsl and usbipd since I had tested using this hardware in the instance.

[stgraber]

New debug binary up for testing.
Hash is now 0627893dc5cc2d88707e6f9a2dc5501e33cf249b86efbede6c5d92ff76752fa0