Compatibility problem with increased BLE Tx buffers

[dhananjaygj]

Issue:

There is a compatibility problem with windows machines when Tx Buffers and pending Tx Buffers are increased to 3.

Background:

This issue was initially discussed here under zephyr developers list.
https://lists.zephyrproject.org/g/devel/topic/25802151?p=Created,,,20,1,0,0

Last update to this discussion is to provide the sniffer log for my problem. I tried to sniff the BLE packets using nRF52840 development kit as sniffer but unfortunately the packets were not logged reliably. Sometimes i was only able to see the advertisement packets but no connection packets afterwards. So instead i created a minimal HID keyboard application to reproduce my issue in a dev kit.

This application is implemented using zephyr's HID sample as reference but with necessary changes for keyboard. The HID report map used is same as the one used in nRF SDK examples. Once extracted kindly place the project inside zephyr/samples/bluetooth/ (the same location where current peripheral_hids sample is present). This is because of locations of Battery and Device information services. I disabled the BT_SETTINGS to connect to different host devices without much hassle of forgetting device keys but this doesn't matter.

I tested my sample application using nRF52840 pca10056 Dev Kit. Once connected the application sends characters 'a-z, 0-9' every 3 seconds. The factor to be noted is the Tx Buffer changes in prj.conf. With default buffer settings (i.e. 2) and there is no problem and am able to receive HID data properly in both Android and Windows 10. But when the Tx Buffer is increased to 3, i am able to receive data in Android properly but not on Windows. In Windows machine, characters are not received properly sometimes same character is printed in windows for long time as explained in my above post.

Sample Application:

peripheral_hids_keyboard.tar.gz

[dhananjaygj]

I did some more debugging and playing with different systems. I can find that this compatibility issue occurs mostly in systems with Intel Wireless Bluetooth drivers. My Windows Machine as well as my Ubuntu machine (separate) has Intel Bluetooth Chipsets. At the same time this Tx buffer changes did not affect Android as well as CSL chipset (Dongle) used on Ubuntu.

[Vudentz]

Can you use btmon to collect the HCI trace on Ubuntu? There could be a problem in the HID layer as well but lets first check the data is not corrupted for some reason.

[dhananjaygj]

@Vudentz Thanks for the reply. I have attached btmon trace i collected for this issue using nRF52832 dev kit and Ubuntu laptop.

In my sample application,

1. I used Tx buffer configuration as
   CONFIG_BT_CONN_TX_MAX=5
   CONFIG_BT_CTLR_TX_BUFFERS=5

2. The connection interval is not updated from application.

3. I send ASCII characters (0x04 - 0x27) on button press through HID profile.

In the trace we can observe that on second time i try to send data, characters are abruptly stopped receiving on the host side and disconnects after sometime.

on the Zephyr side i notice these warning for the remaining characters not sent:

[bt] [WRN] att_chan_get: Not connected
[bt] [WRN] gatt_notify: No buffer available to send notification
[bt] [WRN] att_chan_get: Not connectedss. Error -12
[bt] [WRN] gatt_notify: No buffer available to send notification
[bt] [WRN] att_chan_get: Not connectedease. Error -12
[bt] [WRN] gatt_notify: No buffer available to send notification
[bt] [WRN] att_chan_get: Not connectedss. Error -12
[bt] [WRN] gatt_notify: No buffer available to send notification
[bt] [WRN] att_chan_get: Not connectedease. Error -12
[bt] [WRN] gatt_notify: No buffer available to send notification
[bt] [WRN] att_chan_get: Not connectedss. Error -12
[bt] [WRN] gatt_notify: No buffer available to send notification

btmon_ble_host_crash.txt

Note: I updated my btmon trace with option -w to see in wireshark

[jhedberg]

From the last email on the mailing list thread:

I did some more debugging on my problem and i noticed that whenever the characters are stopped receiving by windows host i get a disconnection in my device with either
LMP RESPONSE TIMEOUT / LL RESPONSE TIMEOUT (0x22) or CONNECTION TIMEOUT (0x08) always.

Those disconnection reasons are usually not something that the host can do anything about (except perhaps changing the connection parameters).

@dhananjaygj are you still seeing this with the latest master branch?

Adding @cvinayak since this could be something controller related as well (though everything seems to indicate a fairly buggy windows side)

[dhananjaygj]

@jhedberg last I checked with zephyr 1.13 and I still sae those errors. While debugging more we saw that this issue seems to be more specific to the BLE chipset used in that laptop. I saw this error in both windows 10 and Ubuntu os installed in that laptop. When too much data is sent within a connection interval, the Intel 3165 controller (used in Lenovo e480 laptop) seems to be NACKing forever after some point. This happens mostly when data is queued at the edge of connection interval. Laptops with more recent Intel chipsets don't seem to have this problem. As a workaround for compatibility we control the rate of data generation.

[Vudentz]

@dhananjaygj Shall we close this one, it doesn't seem this is a problem at our end.

[dhananjaygj]

@Vudentz Yes we can close this until i find some new information. 👍