0

Getting weird data on PyPortal Pynt
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

Getting weird data on PyPortal Pynt

by oldblackcrow on Fri Mar 12, 2021 6:18 pm

Hello all!

I'm trying to display my data from AdafruitIO to my PyPortal. I would love it to be graphical, but I think text based is a good place to start. So, I had some code modified by a wonderful Adafruit admin for this to even work on my Portal.... but it is returning cryptic data that I can't figure out.

The data is a from an SGP30, so it has eCO2 and TVOC. The code is as such:

Code: Select all | TOGGLE FULL SIZE
import time
import board
import busio
from digitalio import DigitalInOut
import neopixel
from adafruit_esp32spi import adafruit_esp32spi
from adafruit_esp32spi import adafruit_esp32spi_wifimanager

print("ESP32 SPI webclient test")

# Get wifi details and more from a secrets.py file
try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)
"""Use below for Most Boards"""
status_light = neopixel.NeoPixel(
    board.NEOPIXEL, 1, brightness=0.2
)  # Uncomment for Most Boards

wifi = adafruit_esp32spi_wifimanager.ESPSPI_WiFiManager(esp, secrets, status_light)

counter = 0

while True:
    try:
        print("Posting data...", end="")
        data = counter
        feed = "rx"
        payload = {"value": data}
        response = wifi.post(
            "https://io.adafruit.com/api/v2/"
            + secrets["aio_username"]
            + "/feeds/"
            + feed
            + "/data",
            json=payload,
            headers={"X-AIO-KEY": secrets["aio_key"]},
        )
        print(response.json())
        response.close()
        counter = counter + 1
        print("OK")
    except (ValueError, RuntimeError) as e:
        print("Failed to get data, retrying\n", e)
        wifi.reset()
        continue
    response = None
    time.sleep(15)


It's returning the following:

OK
Posting data...{'created_at': '2021-03-12T22:08:39Z', 'id': '0EP6ADG8E1ERK0X61Y88991DEB', 'expiration': '2021-04-11T21:08:39Z', 'created_epoch': 1615586919, 'feed_id': 1437718, 'value': '5', 'feed_key': 'rx'}

I have no idea what any of that is or even where to begin to parse it.

If you want to see my feeds page I'm trying to display, it is: https://io.adafruit.com/oldblackcrow/feeds/rx

Anyone know what's going on? Thanks so much!

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by dastels on Fri Mar 12, 2021 7:44 pm

That's the response (as JSON) from Adafruit IO from the post of your data. The more notable things would be when it was created, when it will expire (and be removed?) what feed (internal id as well as key) and the value. It looks like the data is getting posted fine.

The code is posting a sequence of numbers to your rx feed.

As I said it seem to be doing what it's intended to do.

I can't access your feed page, btw. It probably has to be logged into your account.

Dave

dastels
 
Posts: 5960
Joined: Tue Oct 20, 2015 3:22 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Fri Mar 12, 2021 7:59 pm

Oh! Well, it's doing the opposite of what I want to do. I'll have to look for another example that displays the data *from* AdafruitIO to the PyPortal. Thanks for your help!

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by dastels on Fri Mar 12, 2021 8:45 pm


dastels
 
Posts: 5960
Joined: Tue Oct 20, 2015 3:22 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Sat Mar 13, 2021 8:47 pm

Ok... I figured out 1/3 of it... I used the following code:

Code: Select all | TOGGLE FULL SIZE
# SPDX-FileCopyrightText: 2021 ladyada for Adafruit Industries
# SPDX-License-Identifier: MIT

# adafruit_circuitpython_adafruitio usage with an esp32spi_socket
from random import randint
import board
import busio
from digitalio import DigitalInOut
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
import adafruit_requests as requests
from adafruit_io.adafruit_io import IO_HTTP, AdafruitIO_RequestError

try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

print("Connecting to AP...")
while not esp.is_connected:
    try:
        esp.connect_AP(secrets["ssid"], secrets["password"])
    except RuntimeError as e:
        print("could not connect to AP, retrying: ", e)
        continue
print("Connected to", str(esp.ssid, "utf-8"), "\tRSSI:", esp.rssi)

socket.set_interface(esp)
requests.set_socket(socket, esp)

aio_username = secrets["aio_username"]
aio_key = secrets["aio_key"]

# Initialize an Adafruit IO HTTP API object
io = IO_HTTP(aio_username, aio_key, requests)

try:
    # Get the 'temperature' feed from Adafruit IO
    gas_feed = io.get_feed("rx")
except AdafruitIO_RequestError:
    # If no 'temperature' feed exists, create one
    gas_feed = io.create_new_feed("rx")

# Retrieve data value from the feed
print("Retrieving data from gas feed...")
received_data = io.receive_data(gas_feed["key"])
print("Data from gas feed: ", received_data["value"])


What I got back was half the information:
Retrieving data from gas feed...
Data from gas feed: TVOC = 0 ppb

Two things... 1. The eCO2 is not showing as it shows in my actual feed:
gas-feed001.jpg
gas-feed001.jpg (152 KiB) Viewed 134 times


And 2. I would like it to be continuously updated (maybe every second or so). Any help here? Thanks!

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Sun Mar 14, 2021 10:15 am

Ok... I figured out how to get the feed coming in on a loop, but it's still only pulling the TVOC data, not the eCO2...

My new code is:

Code: Select all | TOGGLE FULL SIZE
# SPDX-FileCopyrightText: 2021 ladyada for Adafruit Industries
# SPDX-License-Identifier: MIT

# adafruit_circuitpython_adafruitio usage with an esp32spi_socket
from random import randint
import board
import busio
from digitalio import DigitalInOut
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi
import adafruit_requests as requests
from adafruit_io.adafruit_io import IO_HTTP, AdafruitIO_RequestError

try:
    from secrets import secrets
except ImportError:
    print("WiFi secrets are kept in secrets.py, please add them there!")
    raise

# If you are using a board with pre-defined ESP32 Pins:
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

spi = busio.SPI(board.SCK, board.MOSI, board.MISO)
esp = adafruit_esp32spi.ESP_SPIcontrol(spi, esp32_cs, esp32_ready, esp32_reset)

print("Connecting to AP...")
while not esp.is_connected:
    try:
        esp.connect_AP(secrets["ssid"], secrets["password"])
    except RuntimeError as e:
        print("could not connect to AP, retrying: ", e)
        continue
print("Connected to", str(esp.ssid, "utf-8"), "\tRSSI:", esp.rssi)

socket.set_interface(esp)
requests.set_socket(socket, esp)

aio_username = secrets["aio_username"]
aio_key = secrets["aio_key"]

# Initialize an Adafruit IO HTTP API object
io = IO_HTTP(aio_username, aio_key, requests)

while True:
    try:
    # Get the 'gas' feed from Adafruit IO
        gas_feed = io.get_feed("rx")
       
# Retrieve data value from the feed
        print("Retrieving data from gas feed...")
        received_data = io.receive_data(gas_feed["key"])
        print("Data from gas feed: ", received_data["value"])
    except AdafruitIO_RequestError:
    # If no 'gas' feed exists, create one
        gas_feed = io.create_new_feed("rx")

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by dastels on Sun Mar 14, 2021 10:59 am

The receive_data method return the most recent value in the feed. The receive_all_data method (also takes just the feed key) returns everything. It says "in reverse order" which I take to mean most recent first.

Using the RESTful API directly you can filter the data you get in various ways (by count, date range, etc. See https://io.adafruit.com/api/docs/#adafruit-io-http-api. Alas, that level of capability doesn't seem to be exposed in the library.

Dave

dastels
 
Posts: 5960
Joined: Tue Oct 20, 2015 3:22 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Sun Mar 14, 2021 11:32 am

dastels wrote:The receive_data method return the most recent value in the feed. The receive_all_data method (also takes just the feed key) returns everything. It says "in reverse order" which I take to mean most recent first.

Using the RESTful API directly you can filter the data you get in various ways (by count, date range, etc. See https://io.adafruit.com/api/docs/#adafruit-io-http-api. Alas, that level of capability doesn't seem to be exposed in the library.

Dave


Thank you for your quick response!

When I substituted the "receive_data" to "receive_all_data" I got the following:

code.py output:
Connecting to AP...
Connected to Ross Family RSSI: -20
Retrieving data from gas feed...
Traceback (most recent call last):
File "code.py", line 57, in <module>
File "code.py", line 53, in <module>
File "adafruit_io/adafruit_io.py", line 572, in receive_all_data
File "adafruit_io/adafruit_io.py", line 528, in _get
File "adafruit_requests.py", line 347, in json
File "adafruit_requests.py", line 343, in json
File "adafruit_requests.py", line 67, in readinto
File "adafruit_requests.py", line 235, in _readinto
File "adafruit_requests.py", line 122, in _recv_into
File "adafruit_esp32spi/adafruit_esp32spi_socket.py", line 135, in recv
File "adafruit_esp32spi/adafruit_esp32spi.py", line 751, in socket_read
MemoryError: memory allocation failed, allocating 65 bytes

I'm reading through the link you sent, so once I parse that, maybe I can get this thing working. Again, thank you!

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by dastels on Sun Mar 14, 2021 12:49 pm

Looks like that pulls down too much json. If you don't want to miss anything (which fetching just the last item will likely do) you'll probably have to add some smarts to it to keep track of the last you got and get everything since then... not sure of the details without digging in more. But there is a get-next looking capability in the API that might do the trick.

Dave

dastels
 
Posts: 5960
Joined: Tue Oct 20, 2015 3:22 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Sun Mar 14, 2021 10:14 pm

Thanks Dave... I will try during this week. Much to learn with an old, dusty brain. :-)

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by brubell on Mon Mar 15, 2021 10:01 am

I'm somewhat confused by the screenshot and:
Ok... I figured out how to get the feed coming in on a loop, but it's still only pulling the TVOC data, not the eCO2...


Are you sending the TVOC and ECO2 data to one feed, or does each type of data get sent to a separate feed?

brubell
 
Posts: 1325
Joined: Fri Jul 17, 2015 10:33 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Mon Mar 15, 2021 3:14 pm

It's in a single feed, as that was the default when I set it up.

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Mon Mar 15, 2021 7:19 pm

Just so you have a better idea... I have an SGP30 connected to an ItsyBitsy nRF52840 Express and the Bluetooth connected to the Bluefruit App. That data is sent to AIO. I'm leaning towards ignoring the eCO2 data... it's not vital to my project, but if there is an easy solution, I would be tickled.

I'm integrating this and the Garmin LIDAR codes into the PyPortal GUI example. Thanks to Dave, I have the LIDAR working and a single data point showing on the GUI screen. It's just not giving me a live update... but that's another thread.

In other words, as a newbie, I've got a huge project here that is way over my head - but I have no impetus to stop. :-)

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Re: Getting weird data on PyPortal Pynt

by brubell on Wed Mar 17, 2021 11:22 am

connected to an ItsyBitsy nRF52840 Express and the Bluetooth connected to the Bluefruit App.


Ah, yeah. The BLE app is limited to one feed. I've recently with one of the people on adafruit's ios dev team and asked if we can expand it to more than one feed. You may want to try sending the BLE data to your computer or a Raspberry Pi, and using that as a bridge to Adafruit IO

brubell
 
Posts: 1325
Joined: Fri Jul 17, 2015 10:33 pm

Re: Getting weird data on PyPortal Pynt

by oldblackcrow on Wed Mar 17, 2021 4:29 pm

That's where I started at the beginning... But that did not turn out to be an efficient solution. Plus, this is going into a TOS Tricorder and decided against the RPi for a PyPortal Pynt and the Clue. I'm happy with one data stream.. I have a whole bunch of other sensor data to move through. Thanks for your help!

oldblackcrow
 
Posts: 190
Joined: Tue Jun 20, 2017 5:54 pm

Please be positive and constructive with your questions and comments.