Voting resources, early voting, and poll worker information - VOTE. ... Adafruit is open and shipping.
0

Post CSV via REST
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

Post CSV via REST

by haku15 on Tue Feb 02, 2016 5:15 pm

Hello,
I am trying to post a CSV String to a feed using REST API. The hardware is an Arduino and a Fona808.
I've tried to modify the Adafruit IO library https://github.com/adafruit/Adafruit_IO_Arduino but it does not work, I see no data under the feed.
This what I did:
- in Adafruit_IO_Client.cpp: I just copied the bool Adafruit_IO_Client::send method, renamed it to sendcsv and modified "Content-Type: application/json\r\n" to "Content-Type: text/csv\r\n";
I've tried also "Content-Type: application/csv\r\n".

Code: Select all | TOGGLE FULL SIZE
bool Adafruit_IO_Client::sendcsv(const char* feed, const char* value,
                              const char* key, bool quoted) {
    // Make HTTP POST to send feed data as CSV object.

    // First make sure a connection to the service is available.
    if (!connected()) {
        DEBUG_PRINTLN(F("Failed to connect to service!"));
        return false;
    }

    // Compute size of request.
    uint16_t len = 10 + strlen(value);
    if (quoted) {
        len += 2;
    }

    // Send HTTP POST and headers.
    _client.print(F("POST /api/feeds/"));
    _client.print(feed);
    _client.print(F("/data/send HTTP/1.1\r\n"));
    sendHeaders(key);
    _client.print(F("Content-Type: text/csv\r\n"));
    _client.print(F("Content-Length: "));
    _client.print(len, DEC);
    _client.print(F("\r\n\r\n"));

    // Send HTTP POST data.
    _client.print(F("{\"value\":"));
    if (quoted) {
        _client.print('"');
        _client.print(value);
        _client.print('"');
    }
    else {
        _client.print(value);
    }
    _client.print('}');

    // Now wait to read response (up to the client's configured stream timeout).
    // First read the HTTP/1.1 response.
    char recvbuffer[IO_RECV_LENGTH] = {0};
    if ((_client.readBytesUntil(' ', recvbuffer, sizeof(recvbuffer)) != 8) ||
        (strcmp_P(recvbuffer, PSTR("HTTP/1.1")) != 0)) {
        DEBUG_PRINTLN(F("Failed to find expected HTTP/1.1 response!"));
        _client.stop();
        return false;
    }
    // Now read the status code and expect a 200-level response.
    memset(recvbuffer, 0, sizeof(recvbuffer));
    if ((_client.readBytesUntil(' ', recvbuffer, sizeof(recvbuffer)-1) != 3) ||
        (recvbuffer[0] != '2')) {
        DEBUG_PRINT(F("HTTP POST failed with error code: "));
        DEBUG_PRINTLN(recvbuffer);
        _client.stop();
        return false;
    }

    // Ignore parsing the response data for now.  Close connection and return
    // success.
    _client.stop();
    return true;
}

- in Adafruit_IO_Client.h: I added the sendcsv method
Code: Select all | TOGGLE FULL SIZE
virtual bool sendcsv(const char* feed, const char* value, const char* key, 
                      bool quoted);

- in Adafruit_IO_Arduino.h: I added the sendcsv method
Code: Select all | TOGGLE FULL SIZE
bool sendcsv(const char* value) {   
        return _adapter->send(_name, value, _key, true);

- in Adafruit_IO_Fona.h: I just copied the virtual bool send method, renamed to sendcsv and modified r &= _fona.HTTP_para(F("CONTENT"), F("application/json")); to r &= _fona.HTTP_para(F("CONTENT"), F("text/csv"));
Code: Select all | TOGGLE FULL SIZE
virtual bool sendcsv(const char* feed, const char* value, const char* key,
                      bool quoted) {
        // Make HTTP POST to send feed data as CSV object.

        // Make sure GPRS is connected.
        if (_fona.GPRSstate() != 1) {
            DEBUG_PRINTLN(F("GPRS is not enabled!"));
            return false;
        }

        // Setup FONA for HTTP POST.
        _fona.HTTP_term();  // Stop any pending request.
        bool r = _fona.HTTP_init();
        r &= _fona.HTTP_para(F("CID"), 1);
        _fona.HTTP_para_start(F("URL"));
        _fona.print(_serviceUrl);
        _fona.print(F("/api/feeds/"));
        _fona.print(feed);
        _fona.print(F("/data/send?X-AIO-Key="));
        _fona.print(key);
        r &= _fona.HTTP_para_end();
        r &= _fona.HTTP_para(F("CONTENT"), F("text/csv"));
        if (!r) {
            DEBUG_PRINTLN(F("HTTP POST parameters failed!"));
            return false;
        }

        // Send HTTP POST data.
        if (quoted) {
            // Handle if the value should be surrounded in quotes.
            if (!_fona.HTTP_data(12+strlen(value), 10000)) {
                DEBUG_PRINTLN(F("HTTP POST data failed!"));
                return false;
            }
            _fona.print(F("{\"value\":\""));
            _fona.print(value);
            _fona.print(F("\"}"));
        }
        else {
            // Handle if the value should not be in quotes.
            if (!_fona.HTTP_data(10+strlen(value), 10000)) {
                DEBUG_PRINTLN(F("HTTP POST data failed!"));
                return false;
            }
            _fona.print(F("{\"value\":"));
            _fona.print(value);
            _fona.print(F("}"));
        }

        // Verify OK response.
        if (!_fona.expectReply(F("OK"))) {
            DEBUG_PRINTLN(F("HTTP POST data response failed!"));
            return false;
        }

        // Send HTTP POST request.
        uint16_t status;
        uint16_t datalen;
        if (!_fona.HTTP_action(FONA_HTTP_POST, &status, &datalen)) {
            DEBUG_PRINTLN(F("HTTP POST failed!"));
            return false;
        }

        // Verify response succeeded (is a 200-level response).
        if (status < 200 || status >= 300) {
            DEBUG_PRINT(F("HTTP POST status failure: "));
            DEBUG_PRINTLN(status);
            return false;
        }

        // Ignore parsing the response data for now.
        return true;
    }

- in the Arduino sketch I declared the csv feed like this: Adafruit_IO_Feed gpsLocFeed = aio.getFeed("GPSloc.csv"); and then send the CSV String with gpsLocFeed.sendcsv(CSVString) where the CSVString = "0,1.198365,29.100436,135.000000"

Does anybody know what I am doing wrong or what I am missing?
I am not using MQTT because it does not work at all, maybe the TCP port is blocked by the 2G network provider.
Using REST I can post successfully other non-csv values.
Thanks in advance for any help

haku15
 
Posts: 72
Joined: Mon Jun 25, 2012 12:45 pm

Re: Post CSV via REST

by clemens on Wed Feb 03, 2016 8:34 am

I think you can not post multiple values via a single HTTP request with the current AIO API. It accept only one value at one time. As I understand it a single REST request can handle only one value and only one feed.

https://learn.adafruit.com/adafruit-io?view=all#feeds
> see section "Update an existing feed"

Also submitting many datapoints as CSV and so as a "single string" should not work in case the API makes a basic check of the input data and guarentees the the data is more or less valide (means here it is a kind of number). Don't think there is a method to post strings to AIO atm, but please correct me if I'm wrong.

For many, especially battery driven projects, it would be great to have a single HTTP request and you transmit a bunch of data, let it call a dataset in a single transaction. Expanding this example could mean to have--e.g.houerly / per minute saved datasets from let's say 10 sensors--over a broud time span collected local data and send it only once a day / every hour to a server. Any plans or hints about that? Unfortunatelly long term battery powerd projects have a stepmotherly subsistence.

Transmitting 10 single tiny int values with a HTTP request each.makes me crying about the battery power I have to spend. Even with MQTT it is a waste of power you can use to extend battery and so project life span.

clemens
 
Posts: 41
Joined: Fri Jan 06, 2012 12:21 am

Re: Post CSV via REST

by haku15 on Wed Feb 03, 2016 2:49 pm

Hi Clemens,
thanks for your reply!
I hope there is a way to post a CSV using REST, otherwise how can I push GPS coordinates to the map widget in AIO?
Besides that, REST is more appropriate than MQTT if the device is sleeping most of the time and HTTP is surely not blocked by the network provider.
I don't know if u have seen the tutorial "Track Your Treats: Halloween Candy GPS Tracker", but with MQTT you can post more than one value in a string (with CSV).
Hopefully somebody from Adafruit can clarify about the REST API.

haku15
 
Posts: 72
Joined: Mon Jun 25, 2012 12:45 pm

Re: Post CSV via REST

by clemens on Wed Feb 03, 2016 3:00 pm

I think there is a post from Adafruit / jwcooper about this, I noticed it also just right now:
viewtopic.php?f=56&t=88640#p447356

I don't think you can get the data using that library as it sets the content-type and uses the helper methods to get and set data. You'll want to either use our python/ruby/node libraries for REST, or try using MQTT as that can handle the csv format on an Arduino.


But I wonder why it is possible via MQTT and not via REST, must be different implementations.

clemens
 
Posts: 41
Joined: Fri Jan 06, 2012 12:21 am

Re: Post CSV via REST

by haku15 on Wed Feb 03, 2016 4:10 pm

Yes, that was my other thread. I wrote there that I'd open a new thread to discuss specifically about CSV and REST

haku15
 
Posts: 72
Joined: Mon Jun 25, 2012 12:45 pm

Re: Post CSV via REST

by haku15 on Sat Feb 06, 2016 8:25 pm

Just for info, I found out why MQTT was not working for me http://forums.adafruit.com/viewtopic.php?f=56&t=89410.
Anyway, the question for CSV via REST stays open

haku15
 
Posts: 72
Joined: Mon Jun 25, 2012 12:45 pm

Please be positive and constructive with your questions and comments.