🎁📬❄️📦Adafruit Holiday Shipping Deadlines 2019: Please place all UPS 2 Day orders by 11am Monday, December 16 📦❄️📬🎁
0

Newbie Problems (adafruit.io, JSON, MQTT, GPS)
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Fri Oct 25, 2019 1:27 pm

Hello,

I have been really interested in Adafruit products after having started studying Interactive Wearables course during my Master's of Design degree. I have previously had a bit of experience with Arduino + MAX/MSP. However, this project is quite different, and I feel like struggling "big time".

The project is a live-data tracking mobile dashboard. I have already ordered Flora & Circuit Playground (to learn more later on) and Bluetooth Module (for Flora). I read the 'Converse Weather Project' and watched 'Social Media Counters' coded on Arduino & Wi-Fi module. ...drumroll... Here are my problems:

1. I would like to get a social-account's (instagram/twitter/etc.) follower counts (e.g. https://www.instagram.com/"username"/?__a=1). I need to be able to use JSON (GET & POST functions) over Zapier to be able to receive the follower number of the uni, and continuously do this 10 times per minute (supposedly), but I can only do it for once manually.

I have managed to POST (instead of using Adafruit's Integration) but I get a string, not a number, and got stuck there. Adafruit's plug-in seems to receive it once and for all, I could not get updates on number. There seems to be 2 Adafruit's on Zapier, which is also a bit confusing. I used the lighter-coloured one. What am I doing wrong?

I am also trying out MQTT.fx but nothing is being pushed there either (which scares me).

2. Another issue I am going to face is getting my GPS position updated live, possibly with Zapier. If there's a guide that explains things with Webhooks by Zapier (as IFTTT seems to be providing no options any more) or there's a better way to fetch that live-info from Bluefruit App, that would be great. If not, I am also open to any other ideas (possibly without the GPS module). I have a fitbit if it makes things easier (maybe for zapier).

Thanks a lot (even for reading this long-post from a newbie)

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by brubell on Tue Oct 29, 2019 10:19 am

Hi Doruku,

Let's get this working, one issue at a time.

I would like to get a social-account's (instagram/twitter/etc.) follower counts (e.g. https://www.instagram.com/"username"/?__a=1). I need to be able to use JSON (GET & POST functions) over Zapier to be able to receive the follower number of the uni, and continuously do this 10 times per minute (supposedly), but I can only do it for once manually.

You should check out Adafruit IO's Webhook functionality. Instead of constantly connecting to the instagram API every minute to check if a new message has been posted, you can ask instagram to update a webhook URL on each post. That means instagram will contact you when there’s new data. But, as you can imagine, you need a webserver to listen for that posting. In this case, Adafruit IO can act as that webhook destination.

Once you have the instagram webhook hooked into an Adafruit IO feed, you can use the new multiline text block (whoo! perfect timing) to display the feed on a dashboard.

Please note that Adafruit IO only supports receiving data at this time, you'll need a service that will publish (POST) data into a feed.

Learn more about setting up a webhook here: https://learn.adafruit.com/all-the-inte ... o/webhooks, and let me know if you hit any road-bumps.

- brent r.

brubell
 
Posts: 649
Joined: Fri Jul 17, 2015 10:33 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Sun Nov 03, 2019 9:01 am

Hi,

Thanks for the answers on Discord. I have had the chance to re-watch and re-read documentation about MQTT & adafruit.io and tested out different services.

1. I have managed to catch (GET) JSON data and 'post' them to adafruit.io through webhooks of each specific feed. Currently, my Bluefruit LE App in UART mode shows:
{"feeds": {twitter: 49589}} or {"feeds": {insta: 391}}. I have twitter and insta in the same feed-group. This far, things look ok.

I have been struggling using Zapier's webhook 'POST' method which resulted with {"insta": "391"} instead of 391 on adafruit.io. Adafruit.io integration works cleaner than posting to webhook on Zapier.

2. For the GPS issue, I have installed OwnTracks. I can push the location to adafruit.io with a format that needs some love (has lat & long, missing 'ele'). I'll try to learn more about it before asking a more specific question. Do you know a better alternate to send GPS data, continuously to adafruit.io (other than GPS module)?

At the moment, my plan is to use my iPhone to continuously feed all the data that comes to doruku/g/# to Bluetooth LE module as an MQTT broker. Current Bluefruit LE App disconnects quite often, and does not run in the background. Is there a way to keep it alive (while screen-locked)?

Thanks a lot.

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Sun Nov 03, 2019 11:09 am

Summary:

I can see large arrays of information on UART mode. Current examples I found, mostly involve wi-fi module + MQTT or they're pinging information with a single letter.

1. How can I continuously receive data from adafruit.io through UART (Bluefruit LE App time-out issue) without wi-fi?
2. How can I receive these arrays as numbers (var1, var2, lat, long) and map them to LED colours (my plan is to assign variables on hex)?

Thanks a lot

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Mon Nov 04, 2019 11:41 am

Current Progress:
1. I have managed to run bleuart_cmdmode with the correct configuration.
2. I have not been able to get adafruit.io strings/numbers in the serial monitor for the connection. I could not manage to receive the data that comes to iPhone App in the serial module. Current examples (e.g. adafruitio_03_multiple_feeds in Arduino examples) require wifi_ssid and pass... Do I really need this?
3. If step 2 is possible, i'd like to learn how I can work with large arrays to assign variables {'var1' :'120'} (if string == var1, etc.)

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Tue Nov 05, 2019 8:39 pm

Thanks for bearing with my progressive-monologue.

I've managed to 'get' data to 'serial monitor'. However, things are trickier. Maybe I will have to close this issue and try getting help with a more specific 'subject'.
Here's the problem.

I can get my twitter count from adafruit.io to Serial Monitor, through changing the Subscribe Action from 'Local Only' to 'Transmit'. I could not find this in documentation, found this out by trial & error. The behaviour though, is a bit different.

1. My 'zap' from Zapier sends {"twitter": "12345" }
2. adafruit.io feeds tab shows 12345
3. adafruit.io Dashboard shows twitter 12345
4. Bluefruit LE App shows {"feeds":{"twitter": "12345"}}twitter,12345
5. Serial Monitor shows [Recv] {"feeds":{"twitter":"12345"}}twitter,12345

This gets even more confusing with OwnTracks lat/long information. Bluefruit LE shows the large array, in a different format than Dashboard again. On serial monitor, the array is too large and gets cut...

Apparently I have 2 simpler problems:

1. Is there a way to receive a large array to Bluefruit LE serial monitor?
2. How can I process {"feeds":{"twitter":"12345"}}twitter,12345 to assign 12345 to my twitter variable in Arduino code to hue-shift LEDs?

I will appreciate any help that offers an alternate solution as well. Thanks

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Wed Nov 06, 2019 12:00 pm

I solved question number 2 with ArduinoJson (arduinojson.org).

I got stuck with MQTT data length, I have tried to change MQTT-related values on my main cmdmode document and i seem to get stuck at the character-limit. Is there a solution to this? I looked at Git as well as documentation for 2 hours (only for character limit issue) without a break so far, and cannot seem to find a solution.

Is there a common way to approach MQTT data-length issue. I found a resolved issue on git, but it seems to require extra action?

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Wed Nov 06, 2019 8:21 pm

This might be the place where I am struggling at the moment

Code: Select all | TOGGLE FULL SIZE
// how much data we save in a subscription object
// and how many subscriptions we want to be able to track.
#if defined  (__AVR_ATmega32U4__) || defined(__AVR_ATmega328P__)
  #define MAXSUBSCRIPTIONS 5
  #define SUBSCRIPTIONDATALEN 20
#else
  #define MAXSUBSCRIPTIONS 15
  #define SUBSCRIPTIONDATALEN 100
#endif


I'll try to comment-out the if, and try out if things work, possibly not very stable though. Has anybody experienced this?

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Thu Nov 07, 2019 12:44 pm

I have had my hands on the Circuit Playground Express, with the hopes of not having the same issue (thinking that I was being held by Flora due to default settings). No luck, any advices? or ideas why?

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Fri Nov 08, 2019 12:39 am

My forum-logging goes on..
I really wish to get some answers at some point, though as my project gets close to a deadline.

A workaround can be using the Bluefruit LE Connect iOS App to push GPS data from control mode.
However, this does not really work simultaneously with UART mode. I guess the App (or UART mode) does not really support running the GPS-feed simultaneously, or in the background.

I still have not been progressing with the MQTT character limitation issue, though. I could not reach any solved git issue or forum entry that solves the exact problem. I am using adafruit.io at the moment.

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by brubell on Mon Nov 11, 2019 11:37 am

Hi doruku,

in:

Code: Select all | TOGGLE FULL SIZE
// how much data we save in a subscription object
// and how many subscriptions we want to be able to track.
#if defined  (__AVR_ATmega32U4__) || defined(__AVR_ATmega328P__)
  #define MAXSUBSCRIPTIONS 5
  #define SUBSCRIPTIONDATALEN 20
#else
  #define MAXSUBSCRIPTIONS 15
  #define SUBSCRIPTIONDATALEN 100
#endif


If you're using a Circuit Playground express, try changing the code to:

Code: Select all | TOGGLE FULL SIZE
// how much data we save in a subscription object
// and how many subscriptions we want to be able to track.
#if defined  (__AVR_ATmega32U4__) || defined(__AVR_ATmega328P__)
  #define MAXSUBSCRIPTIONS 5
  #define SUBSCRIPTIONDATALEN 20
#else
  #define MAXSUBSCRIPTIONS 15
  #define SUBSCRIPTIONDATALEN 500
#endif

brubell
 
Posts: 649
Joined: Fri Jul 17, 2015 10:33 pm

Re: Newbie Problems (adafruit.io, JSON, MQTT, GPS)

by doruku on Tue Nov 12, 2019 7:55 am

Hi brubell,

Unfortunately it does not make any difference. I changed both the local and the library code. I am sure about the library: Adafruit_MQTT.h

No difference at all, unfortunately.

doruku
 
Posts: 13
Joined: Sun Oct 20, 2019 12:11 pm

Please be positive and constructive with your questions and comments.


cron