0

AIO error: data missing required value
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

AIO error: data missing required value

by Rcayot on Thu Mar 04, 2021 12:15 pm

okay, long post with a lot of code.

I have a ble280 sensor that I connected to my Raspberry Pi Zero W which read the data, and sent it to AIO, ran for about a month with few problems. This was indoors, while on my desk.

I decided that I wanted the sensor outside, and that I wanted it to be battery powered and the Pi and wifi consumes more power than I wanted to deal with, so I rewrote the program to run on my itsybitsy nRF52840. Well, after a few mis-steps, I got the dode to run and send the temperature, humidity, and pressure data to my raspberry Pi zero W over BLE. The issue then was that sending that data forward to AIO was giving me fits. Thsi is the error I get:

"Adafruit_IO.errors.RequestError: Adafruit IO request failed: 422 Unprocessable Entity - request failed - Record invalid. Failed to save data to MYUSERNAME/feeds/temperature, data missing required value.

Now here is the send to AIO portion of the code that worked:
Code: Select all | TOGGLE FULL SIZE
while True:
    humidity = bme280.relative_humidity
    temperature = bme280.temperature
    pressure = bme280.pressure
    if humidity is not None and temperature is not None and pressure is not None:
        print('Temp={0:0.1f}*C Humidity={1:0.1f}% Pressure={2:0.1f}hPa'.format(temperature, humidity, pressure))
        # Send humidity and temperature feeds to Adafruit IO
        temperature = '%.2f'%(temperature)
        humidity = '%.2f'%(humidity)
        pressure = '%.2f'%(pressure)
        aio.send(temperature_feed.key, str(temperature))
        aio.send(humidity_feed.key, str(humidity))
        aio.send(pressure_feed.key, str(pressure))
        time.sleep(300)


for whatever reason, the formating lines " temperature = '%.2f'%(temperature)" are required, and obtained from example code and I have an idea it is a formatting statement.

Now, when I send the data over BLE to the PI Zero W, I use:

Code: Select all | TOGGLE FULL SIZE
while True:
    ble.start_advertising(advertisement)
    print("Waiting to connect")
    while not ble.connected:
        pass
    print("Connected")
    while ble.connected:
        time.sleep(5)
        temperature = bme280.temperature
        uart.write("%0.2f " % temperature)
        time.sleep(5)
        humidity = bme280.relative_humidity
        uart.write("%0.2f " % humidity)
        time.sleep(5)
        pressure = bme280.pressure
        uart.write("%0.1f " % pressure)
        time.sleep(300)


This is recieved by the Pi zero W using the following:

Code: Select all | TOGGLE FULL SIZE
while True:
    if not uart_connection:
        print("Trying to connect...")
        for adv in ble.start_scan(ProvideServicesAdvertisement):
            if UARTService in adv.services:
                uart_connection = ble.connect(adv)
                print("Connected")
                break
        ble.stop_scan()
    if uart_connection and uart_connection.connected:
        uart_service = uart_connection[UARTService]
        while uart_connection.connected:
            temperature = uart_service.readline().decode("utf-8")
            #temperature = '%.2f'%(temperature)
            print(temperature)
            #aio.send(temperature_feed.key, str(temperature))
            humidity = uart_service.readline().decode("utf-8")
            #humidity = '%.2f'%(humidity)
            print(humidity)
            aio.send(humidity_feed.key, str(humidity))
            pressure = uart_service.readline().decode("utf-8")
            #pressure = '%.2f'%(pressure)
            print(pressure)
            aio.send(pressure_feed.key, str(pressure))


Now, If I comment out the aio.send commands, the data is recieved by the Pi and prints. However, if I uncomment the aio.send commands, I get the errors noted above. Now, the AIO portions of the two programs are identical, as I copied from one to the other. I have tried to include or exclude the formatting code: #temperature = '%.2f'%(temperature) without success. I have tried a number of other things without avail, one error is that the value has to be a real number not a string, things like that.

Any ideas that can help would be appreciated. I would include the entire code, buyt we would have three fairly long pieces of code here and might be too cumbersome, but anything anyone needs to help, I will include.

Thanks,
Roger Ayotte

Rcayot
 
Posts: 138
Joined: Sat Feb 08, 2020 6:48 pm

Re: AIO error: data missing required value

by brubell on Fri Mar 05, 2021 11:12 am

for whatever reason, the formating lines " temperature = '%.2f'%(temperature)" are required, and obtained from example code and I have an idea it is a formatting statement.

Adafruit IO Python's `send()` call expects the payload formatted as a string.

Could you modify these lines:
Code: Select all | TOGGLE FULL SIZE
            #temperature = '%.2f'%(temperature)
            print(temperature)
            #aio.send(temperature_feed.key, str(temperature))



to:

#temperature = '%.2f'%(temperature)
print(temperature)
print(type(temperature)) # lets us see the type of the temperature variable you constructed above
#aio.send(temperature_feed.key, temperature

brubell
 
Posts: 1325
Joined: Fri Jul 17, 2015 10:33 pm

Re: AIO error: data missing required value

by Rcayot on Fri Mar 05, 2021 1:40 pm

thank you for your help.

Here is what I changed my code to per your suggestion:
Code: Select all | TOGGLE FULL SIZE
while True:
    if not uart_connection:
        print("Trying to connect...")
        for adv in ble.start_scan(ProvideServicesAdvertisement):
            if UARTService in adv.services:
                uart_connection = ble.connect(adv)
                print("Connected")
                break
        ble.stop_scan()
    if uart_connection and uart_connection.connected:
        uart_service = uart_connection[UARTService]
        while uart_connection.connected:
            temperature = uart_service.readline().decode("utf-8")
            temperature = '%.2f'%(temperature)
            print(temperature)
            print(type(temperature)) # lets us see the type of the temperature variable you constructed above
            aio.send(temperature_feed.key, str(temperature))


What happens is:
Code: Select all | TOGGLE FULL SIZE
pi@raspberrypi:~/mu_code $ python3 temp_humidity_ble.py
Trying to connect...
Connected
Traceback (most recent call last):
  File "temp_humidity_ble.py", line 56, in <module>
    temperature = '%.2f'%(temperature)
TypeError: must be real number, not str


So evidently the information (payload?) coming from the BLE-UART is not amenable to that kind of formatting.

If I just remove the offending line from the code, the following error occurs:
Code: Select all | TOGGLE FULL SIZE
pi@raspberrypi:~/mu_code $ python3 temp_humidity_ble.py
Trying to connect...
Connected

<class 'str'>
Traceback (most recent call last):
  File "temp_humidity_ble.py", line 59, in <module>
    aio.send(temperature_feed.key, str(temperature))
  File "/home/pi/.local/lib/python3.7/site-packages/Adafruit_IO/client.py", line 154, in send_data
    return self.create_data(feed, payload)
  File "/home/pi/.local/lib/python3.7/site-packages/Adafruit_IO/client.py", line 254, in create_data
    return Data.from_dict(self._post(path, data._asdict()))
  File "/home/pi/.local/lib/python3.7/site-packages/Adafruit_IO/client.py", line 127, in _post
    self._handle_error(response)
  File "/home/pi/.local/lib/python3.7/site-packages/Adafruit_IO/client.py", line 108, in _handle_error
    raise RequestError(response)
Adafruit_IO.errors.RequestError: Adafruit IO request failed: 422 Unprocessable Entity - request failed - Record invalid. Failed to save data to Rcayot/feeds/temperature, data missing required value


So evidently the data stream from the BLE-UART is a 'string' already, but is missing something. Note, in addition the 'print(temperature) command did not result in the value being printed.

If I comment out the print(type(temperature)) line and the aio.send line, the program prints the data:
Code: Select all | TOGGLE FULL SIZE
pi@raspberrypi:~/mu_code $ python3 temp_humidity_ble.py
Trying to connect...
Connected




21.96




25.46




990.7


One more thing, if I change:
Code: Select all | TOGGLE FULL SIZE
temperature = uart_service.readline().decode("utf-8")


to:
Code: Select all | TOGGLE FULL SIZE
temperature = uart_service.readline()


then I get:
Code: Select all | TOGGLE FULL SIZE
pi@raspberrypi:~/mu_code $ python3 temp_humidity_ble.py
Trying to connect...
Connected
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'22.05 ')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'25.23 ')


Which to me is strange. And actually, if I then un-comment my aio.send line I get:
Code: Select all | TOGGLE FULL SIZE
pi@raspberrypi:~/mu_code $ python3 temp_humidity_ble.py
Trying to connect...
Connected
bytearray(b'')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'22.09 ')
<class 'bytearray'>
bytearray(b'25.30 ')
<class 'bytearray'>
bytearray(b'')
<class 'bytearray'>
bytearray(b'990.7 ')


Interestingly, there is no aio error, and my temperature feed contains:
Code: Select all | TOGGLE FULL SIZE
2021/03/05 12:37:31PM   bytearray(b'')      
2021/03/05 12:37:28PM   bytearray(b'990.7 ')      
2021/03/05 12:37:25PM   bytearray(b'')      
2021/03/05 12:37:22PM   bytearray(b'25.30 ')      
2021/03/05 12:37:18PM   bytearray(b'22.09 ')      
2021/03/05 12:37:15PM   bytearray(b'')      
2021/03/05 12:37:12PM   bytearray(b'')   



AIO is taking the values!

So, what teh heck?

Please, if you have any more suggestions, please help!

Roger

Rcayot
 
Posts: 138
Joined: Sat Feb 08, 2020 6:48 pm

Re: AIO error: data missing required value

by brubell on Mon Mar 15, 2021 10:53 am

Roger, was this solved already in another thread, or are you still having this issue (you have a few threads open and I want to make sure I'm not dropping anything).

brubell
 
Posts: 1325
Joined: Fri Jul 17, 2015 10:33 pm

Re: AIO error: data missing required value

by Rcayot on Tue Mar 16, 2021 9:38 am

Brubell,

Thanks for the follow-up. Yes, this particular issue has been resolved. I have had a few other issues with my AIO project, and have been helped in a very prompt and helpful manner.

I continue to have a few issues, and am about to post another request, but I think I have a few more things to try before I do that.

Thanks again.

Roger

Rcayot
 
Posts: 138
Joined: Sat Feb 08, 2020 6:48 pm

Please be positive and constructive with your questions and comments.