Black Lives Matter - Action and Equality. ... Adafruit joins the Stop Hate for Profit campaign.
0

"RuntimeError: buffer size must match format" in BLE scannin
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

"RuntimeError: buffer size must match format" in BLE scannin

by andreas_johnsen on Sun Apr 19, 2020 2:37 pm

Hei,

I get "RuntimeError: buffer size must match format" when running this example from the BLE documentation on a nrf52 dongle (Adafruit CircuitPython 5.2.0 on 2020-04-09; PCA10059 nRF52840 Dongle with nRF52840):

Code: Select all | TOGGLE FULL SIZE
from adafruit_ble import BLERadio

radio = BLERadio()
print("scanning")
found = set()
for entry in radio.start_scan(timeout=60, minimum_rssi=-80):
    addr = entry.address
    if addr not in found:
        print(entry)
    found.add(addr)

print("scan done")

The output is:
Code: Select all | TOGGLE FULL SIZE
code.py output:
scanning
<Advertisement  >
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
<Advertisement flags=<AdvertisingFlags general_discovery > >
<Advertisement  >
<Advertisement  >
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
Traceback (most recent call last):
  File "code.py", line 9, in <module>
  File "adafruit_ble/advertising/__init__.py", line 299, in __str__
  File "adafruit_ble/advertising/__init__.py", line 180, in __get__
RuntimeError: buffer size must match format


Any help to solve this problem would be greatly appreciated.

andreas_johnsen
 
Posts: 6
Joined: Sun Apr 19, 2020 1:42 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by tannewt on Mon Apr 20, 2020 1:02 pm

Hi! Would you mind modifying `adafruit_ble/advertising/__init__.py` so that it prints the buffer it's trying to parse? I think it'll be hard to replicate here because it may be a particular advertisement in your environment causing the problem.

tannewt
 
Posts: 1669
Joined: Thu Oct 06, 2016 8:48 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by andreas_johnsen on Tue Apr 21, 2020 7:38 am

Yes, of course, my tests are dependent on my Bluetooth environment. Easy to forget. :)

I added the following line before the return statement at line 180 in the __init__.py that is rasing the runtime errors:
print ("obj.data_dict[self._adt]: ", obj.data_dict[self._adt])

A typical output looks like this:
Code: Select all | TOGGLE FULL SIZE
code.py output:
scanning
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
<Advertisement  >
obj.data_dict[self._adt]:  b'\x0c'
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
<Advertisement flags=<AdvertisingFlags general_discovery > >
obj.data_dict[self._adt]:  b'\x0c'
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
<Advertisement flags=<AdvertisingFlags limited_discovery general_discovery > >
<Advertisement flags=<AdvertisingFlags general_discovery > >
<Advertisement  >
obj.data_dict[self._adt]:  b'\x08'
obj.data_dict[self._adt]:  b'\x00'
Traceback (most recent call last):
  File "code.py", line 9, in <module>
  File "adafruit_ble/advertising/__init__.py", line 300, in __str__
  File "adafruit_ble/advertising/__init__.py", line 181, in __get__
RuntimeError: buffer size must match format

I have noted that the two last buffers that are printed before the runtime error are always these:

obj.data_dict[self._adt]: b'\x08'
obj.data_dict[self._adt]: b'\x00'

andreas_johnsen
 
Posts: 6
Joined: Sun Apr 19, 2020 1:42 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by tannewt on Tue Apr 21, 2020 1:37 pm

Mind also printing the adt and the format? It's a bit tricky to know what field it is otherwise.

Dan just found that I was using ringbuf a bit incorrectly so this may be a result of that. Thanks!

tannewt
 
Posts: 1669
Joined: Thu Oct 06, 2016 8:48 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by andreas_johnsen on Thu Apr 23, 2020 2:13 am

Thanks for following up.

I added adt and format to the print statement.

A typical output looks likt this:
Code: Select all | TOGGLE FULL SIZE
code.py output:
scanning
obj.data_dict[self._adt]:  b'\x0c' self._adt:  10 self._format:  <b
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
<Advertisement  >
<Advertisement flags=<AdvertisingFlags general_discovery > >
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
<Advertisement  >
obj.data_dict[self._adt]:  b'\x0c' self._adt:  10 self._format:  <b
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
<Advertisement flags=<AdvertisingFlags limited_discovery general_discovery > >
<Advertisement  >
<Advertisement  >
<Advertisement  >
<Advertisement  >
<Advertisement  >
obj.data_dict[self._adt]:  b'\x08' self._adt:  10 self._format:  <b
obj.data_dict[self._adt]:  b'\x00' self._adt:  25 self._format:  <H
Traceback (most recent call last):
  File "code.py", line 9, in <module>
  File "adafruit_ble/advertising/__init__.py", line 300, in __str__
  File "adafruit_ble/advertising/__init__.py", line 181, in __get__
RuntimeError: buffer size must match format

The last two print statements before the RuntimeError are always:

obj.data_dict[self._adt]: b'\x08' self._adt: 10 self._format: <b
obj.data_dict[self._adt]: b'\x00' self._adt: 25 self._format: <H

andreas_johnsen
 
Posts: 6
Joined: Sun Apr 19, 2020 1:42 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by danhalbert on Thu Apr 23, 2020 9:13 am

Scott is asleep, and you may be too, but here's another thing to print:
In advertising/__init__.py, just after these lines, could you add a print statement that prints the raw bytes of the incoming advertisement? Thanks.

Code: Select all | TOGGLE FULL SIZE
    @classmethod
    def from_entry(cls, entry):
        """Create an Advertisement based on the given ScanEntry. This is done automatically by
           `BLERadio` for all scan results."""
           print("advertisement_bytes", entry.advertisement_bytes)  # <----------- NEW
           self = cls()
           ...


Do you know where these advertisements are coming from? Are you generating them? If so, could you post the code that creates them?

danhalbert
 
Posts: 2031
Joined: Tue Aug 08, 2017 12:37 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by andreas_johnsen on Fri Apr 24, 2020 4:22 pm

This is a typical output with the new print statment added:
Code: Select all | TOGGLE FULL SIZE
code.py output:
scanning
advertisement_bytes b'\x1c\x16\xf1\xfeN\xd5\x00\xbb\x95_%\xe7\xea\x87\x98\x1aC\t\x9c\x0c\xd5\xc6\x91\xf1\xab\xb6\x9e\x131'
<Advertisement  >
advertisement_bytes b'\x03\x03\x9f\xfe\x17\x16\x9f\xfe\x02F9nXn6XUJJY\x00\x00\x01q\xac\x187V'
<Advertisement  >
advertisement_bytes b'\t\xff\xe0\x00\x01Y\xca\x8d\x1fA'
advertisement_bytes b'\x02\x01\x06\n\xffL\x00\x10\x05K\x1c\x99\x9a8'
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
advertisement_bytes b'\x1e\xff\x06\x00\x01\t \x02\x199^\xbe\xc2\xb3\x0b\xf7f\xaf\xf6M\xaf<\x9dn\xe2\xf9\x9d9\x0b\xa1\x0c'
<Advertisement  >
advertisement_bytes b'\x02\x01\x06\n\xffL\x00\x10\x05\x01\x1c\x8b\xa7\x08'
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
advertisement_bytes b'\x1c\x16\xf1\xfe>\xbe\x10\x19\x85!p\xba}\xea\xad3\xf7k\xe7\x94lrHw\\\xdf\xbe~2'
<Advertisement  >
advertisement_bytes b'\x02\x01\x1e\x1b\xffu\x00B\x04\x01@^\\I}\x95\xee6^I}\x17\xee5\x01\x00\x00\x00\x00\x00\x00'
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
advertisement_bytes b'\x1c\x16\xf1\xfe=\xbe\x10\x19\x85\xb0\xf4}\x82\x9f>J\xf80as\x9d\xa4\xe2o\xad\x10\xa2\xb42'
<Advertisement  >
advertisement_bytes b'\x02\x01\x06\n\xffL\x00\x10\x05\x01\x1c\xef\xcfJ'
<Advertisement flags=<AdvertisingFlags general_discovery le_only > >
advertisement_bytes b'\x02\x01\x1a\x02\n\x0c\x0b\xffL\x00\x10\x06\x03\x1ee\x06\x97;'
obj.data_dict[self._adt]:  b'\x0c' self._adt:  10 self._format:  <b
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
advertisement_bytes b'\x14\xffL\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00'
advertisement_bytes b'\x02\x01\x1a\x02\n\x0c\n\xffL\x00\x10\x05\x03\x1cn\xee,'
obj.data_dict[self._adt]:  b'\x0c' self._adt:  10 self._format:  <b
<Advertisement tx_power=12 flags=<AdvertisingFlags general_discovery > >
advertisement_bytes b'\x1b\xffu\x00B\x04\x01\x80`d\x1c\xb0\xf2+sf\x1c\xb0\xf2+r\x01\x00\x00\x00\x00\x00\x00'
<Advertisement  >
advertisement_bytes b'\x02\x01\x1a\x0b\xffL\x00\t\x06\x03\x03\n\x00\x01\x08'
<Advertisement flags=<AdvertisingFlags general_discovery > >
advertisement_bytes b'\x02\x01\x06\n\xffL\x00\x10\x05K\x1c\x99\x9a8'
advertisement_bytes b'\x1c\x16\xf1\xfe\x93\x11\x1e\x87\xa8v\xbdO\xa2;\xe2si3N\x0b\xb1..I#z\x8b\x182'
<Advertisement  >
advertisement_bytes b'\x1c\x16\xf1\xfe@\xbe\x10\x19\x85O\xf9\r~\xd2-S\xcc\xac>\xe8\xf5)k_7k\xb4\xee2'
<Advertisement  >
advertisement_bytes b'\x02\x01\x06\n\xffL\x00\x10\x05\x01\x1c\x8b\xa7\x08'
advertisement_bytes b'\x02\x01\x06\x03\x03\xf1\xfe\x02\x19\x00\x02\n\x08\x08\t@MD07A6'
obj.data_dict[self._adt]:  b'\x08' self._adt:  10 self._format:  <b
obj.data_dict[self._adt]:  b'\x00' self._adt:  25 self._format:  <H
Traceback (most recent call last):
  File "code.py", line 13, in <module>
  File "code.py", line 10, in <module>
  File "adafruit_ble/advertising/__init__.py", line 301, in __str__
  File "adafruit_ble/advertising/__init__.py", line 181, in __get__
RuntimeError: buffer size must match format

There is a pattern of the last output of advertisement_bytes before the RuntimeError. Here are three examples:
Code: Select all | TOGGLE FULL SIZE
advertisement_bytes b'\x02\x01\x06\x03\x03\xf1\xfe\x02\x19\x00\x02\n\x08\x08\t@MD0C3C’
advertisement_bytes b'\x02\x01\x06\x03\x03\xf1\xfe\x02\x19\x00\x02\n\x08\x08\t@MD0E6A’
advertisement_bytes b'\x02\x01\x06\x03\x03\xf1\xfe\x02\x19\x00\x02\n\x08\x08\t@MD07A6’

I'm not generating the Bluetooth advertisements and not sure where they come from. But I have noticed that Bluetooth scanners like the LightBlue iOS app list tens, maybe hundres nearby peripherals. This is very strange and I will try to find what generates these advertisements.

andreas_johnsen
 
Posts: 6
Joined: Sun Apr 19, 2020 1:42 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by tannewt on Fri Apr 24, 2020 8:19 pm

Thank you for the full byte string. I reproduced it here. The issue is that the "appearance" ADT value is too short. It's supposed to be two bytes and the advertisement has one. Its the b"\x02\x19\x00" portion. The "@MD07A6" is the complete name of the device and there is a 0xfef1 service which is registered to the company CSR.

A simple hack would be to try/except around the "getattr". In the long term we may want to add placeholder "InvalidValue" sort of object. I'm not exactly sure the best way to approach it. We could also simply try/except and omit it from the string. I'm sure this isn't the last bad advertising packet we see. :-)

tannewt
 
Posts: 1669
Joined: Thu Oct 06, 2016 8:48 pm

Re: "RuntimeError: buffer size must match format" in BLE sca

by andreas_johnsen on Sun Apr 26, 2020 8:50 am

Thank you very much for your feedback and suggestion (I added try/except as suggested).

I agree that all Bluetooth code processing received messages should handle ill-formed messages. Since it can be considered as external input, it can potentially be used as an attack vector.

Thanks again.

andreas_johnsen
 
Posts: 6
Joined: Sun Apr 19, 2020 1:42 pm

Please be positive and constructive with your questions and comments.