aJSON Library and SD Card

[cxvjeff]

Looking to see if I'm recreating the wheel on this one or not...

I'm attempting to use my new EthernetShield as a webserver that is capable of reading and writing to JSON files that are accessed through the HTML pages served up using jQuery's $.get to retrieve the data and display it on the web page.

The first hurdle is to read in JSON data that is at first static on the SD card of the Ethernet shield so the Uno can know variables such as a MAX or MIN temperature to send out email alerts. It sends these email alerts by briefly opening a connection to an external website page that takes POST data to send out the email alerts.

For parsing the JSON data I'm looking to use the aJSON library provided at https://github.com/interactive-matter/aJson. This library is expecting a char* or string value to parse the JSON data. I'm using the demo at http://arduino.cc/en/Tutorial/ReadWrite to load or read the data from the SD card.

However, the myFile.read() is only returning one byte of data at a time from the file and not the whole file or chunks. So I need to be able to store this information in a char* to pass to aJSON for parsing. I haven't figured this out how to do this yet and that is my base question. I've tried numerous methods in which to buffer this data but it only seems to freak out the card or else the chars captured are strange ASCII codes that cause the output of the Serial to freak out as I echo it back via Serial.println(data);.

If I'm on the wrong track please let me know. I don't wish to take someone else's code for free and claim it as my own or anything, just looking to better understand the method or thought process I should be taking in getting over this hurdle. My best guess at this point is to refit the streamhelper.c located at https://github.com/interactive-matter/aJson/blob/master/utility/streamhelper.c to work with the data from the SD card if such a thing can be done.

Thanks in advance if you have any feedback to this wild rambling of confusion.

[JamesC4S]

2K of RAM doesn't lend itself parson JSON data structures.

[Dan Malec]

I'd agree with @JamesC4S - the available memory is going to make general JSON parsing difficult. I am wondering if you can redefine the problem a bit given the memory limits. Two thoughts:

1.) For parsing JSON data, take a look at the jsonParse method in the Gutenbird sketch from the IoT printer https://github.com/adafruit/Adafruit-Tweet-Receipt/blob/master/Gutenbird.ino. If you can identify a number of fields at the same depth in the structure, you can handle them differently than the bulk of the JSON.

2.) Consider writing a parser that doesn't know it's looking at JSON when generating the files, but just thinks in terms of substituting variables into files... so, if you define your static JSON file like so:

Code: Select all

{
  'temperature': '$temp',
  'foo': 'bar'
}


You could parse it with something like so:

Code: Select all

  char c;
  int i=0;
  char var_name[16];
  boolean is_var = false;
  double temp = 91.4;
  ...
  while (myFile.available()) {
    c = myFile.read();
    if (!is_var) {
      if (c == '$') {
        // switch to parsing variables
        is_var = true;
        i = 0;
      } else {
        // otherwise, pass character unchanged
        Serial.write(c);
      }
    } else {
      if (isalnum(c)) {
         // parse as variable name as long as character is alphanumeric
        if (i < 15) {
          var_name[i++] = c;
        }
      } else {
        // stop parsing as variable name
        is_var = false;
        // null terminate variable name
        var_name[i] = '\0';
        // substitute if variable name is known
        if (!strcasecmp(var_name, "temp")) {
          Serial.print(temp);
        }
        //  pass character unchanged
        Serial.write(c);
      }
    }
  }


You might want to add some extra logic to escape dollar signs, handle long variable names, etc. but this would be an alternate approach that would get dynamic content into static JSON files.

[philba]

I agree with Dan on point 2. I think the OP is trying to do something simple and a general solution to that is likely to be inappropriate for these dinky little microcontrollers we all love.

However, the OP appears to be fairly early on the learning curve. To answer the explicit question takes a bit of a tutorial. a char * doesn't save data, it points to data. cxvjeff, you need to take the individual characters and store them into an array. something like this

Code: Select all

char buf[20];  // make it as big as you need
char *bp;  // pointer into buf[]
bp = buf;  // get address of array

while (<more data>) {
  *bp++ = get_another_char();
}
*bp = '\0';  // null for string manipulation


Then buf holds the string and you can party with that like its C99.

I'm not sure why echoing the characters is causing serial to freak out because JSON is supposed to be human readable and thus uses only readable characters. Try printing a known good string ("hello cxvjeff" or what ever). Maybe you have the baud rate screwed up...

This page http://en.wikipedia.org/wiki/C_character_classification lists a number of functions (actually, macros, I believe) that you can use to classify your characters.

[cxvjeff]

Thanks to all for your replies and assistance. I'm a little disheartened to hear that JSON may be a leap too far for Arduino but I enjoy beating the proverbial dead horse as I'm just stubborn that way. Suffice it to say, what I'm trying to accomplish is going to be extremely simple in nature regarding JSON and the data stored on the SD card. This was my solution to provide a channel for the Arduino to trade data back and forth to the web page on the SD card. When the page POSTS data it will save to the SD card and allow the Arduino to make on-the-fly changes to the code within. When the JSON data is written to the SD card it would flag the code on Arduino to re-read the data to obtain the thresholds for temperature and email listings for alerts. I'm fairly certain that 2K of memory will handle it just fine but I don't have near the experience that you all who have replied do with this platform.

@Dan Malec: After I quit being hard-headed about this situation I will surely be given your suggestion a full run for its money and see what I come up with. Your example is well appreciated.

@philba: I did understand that * made a pointer but I still confuse it often by attempting to use it as a variable and not a reference to a variable. My thanks for your clarification!

Once I get somewhere I'll communicate back so you can all have a good laugh that I wasted so much time on a deadend. Until then, take care...

Note: No horses have been injured in the making of this project.

[JamesC4S]

The Arduino simply isn't built for processing strings. The author of the JSON library even admits this.

If you need to manipulate strings look to the BeagleBone, Raspberry Pi (if you can find one), or the NETDUINO (which has nothing to do with Arduino.)

These are probably better platforms for manipulating strings as they are based on microprocessors and not a microcontroller.

[philba]

While I agree that an arduino (actually, any ATMega) isn't an idea platform for string processing, for manipulating a few strings it's fine. The OP was just trying to pull a couple of items out of a file and it should be ok for that. I wouldn't try to parse XML or such, though.