Unreliable Parsing of xml tags

[adafruit_support_rick]

One of the library contributors has come up with an idea:

I think this is something related to the connection being closed by the server and read calls failing or blocking in some way (need to dig a little further to see exactly what's going on). I found you can actually make the failing pages work if you switch to HTTP/1.1 protocol and remove the Connection: close header--this will make the connection persistent so the server won't immediately close it.

Here's the code I used (notice protocol is now HTTP/1.1 and the connection close header is not sent):

Code: Select all

/* Try connecting to the website */
  Adafruit_CC3000_Client www = cc3000.connectTCP(ip, 80);
  if (www.connected()) {
    www.fastrprint(F("GET "));
    www.fastrprint(WEBPAGE);
    www.fastrprint(F(" HTTP/1.1\r\n"));
    www.fastrprint(F("Host: ")); www.fastrprint(WEBSITE); www.fastrprint(F("\r\n"));
    //www.fastrprint(F("Connection: close\r\n"));
    www.fastrprint(F("\r\n"));
    www.println();
  } else {
    Serial.println(F("Connection failed"));    
    return;
  }

Here's the response I get back from the w1.weather.gov page (notice it didn't close the connection and sent a keep alive header in the response headers indicating so):

Code: Select all

Hello, CC3000!

Free RAM: 1191

Initializing...
Started AP/SSID scan




Connecting to tdicola...Waiting to connect...Connected!
Request DHCP

IP Addr: 192.168.1.103
Netmask: 255.255.255.0
Gateway: 192.168.1.1
DHCPsrv: 192.168.1.1
DNSserv: 192.168.1.1
w1.weather.gov -> 23.59.189.83

Connect to 23.59.189.83:80
-------------------------------------
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Mon, 04 Nov 2013 22:02:15 GMT
Accept-Ranges: bytes
Content-Length: 2268
Content-Type: application/xml
Date: Mon, 04 Nov 2013 22:36:25 GMT
Connection: keep-alive

<?xml version="1.0" encoding="ISO-8859-1"?> 
<?xml-stylesheet href="latest_ob.xsl" type="text/xsl"?>
<current_observation version="1.0"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="http://www.weather.gov/view/current_observation.xsd">
    <credit>NOAA's National Weather Service</credit>
    <credit_URL>http://weather.gov/</credit_URL>
    <image>
        <url>http://weather.gov/images/xml_logo.gif</url>
        <title>NOAA's National Weather Service</title>
        <link>http://weather.gov</link>
    </image>
    <suggested_pickup>15 minutes after the hour</suggested_pickup>
    <suggested_pickup_period>60</suggested_pickup_period>
    <location>New York City, Central Park, NY</location>
    <station_id>KNYC</station_id>
    <latitude>40.783</latitude>
    <longitude>-73.967</longitude>
    <observation_time>Last Updated on Nov 4 2013, 4:51 pm EST</observation_time>
        <observation_time_rfc822>Mon, 04 Nov 2013 16:51:00 -0500</observation_time_rfc822>
    <weather>Fair</weather>
    <temperature_string>42.0 F (5.6 C)</temperature_string>
    <temp_f>42.0</temp_f>
    <temp_c>5.6</temp_c>
    <relative_humidity>38</relative_humidity>
    <wind_string>Northeast at 4.6 MPH (4 KT)</wind_string>
    <wind_dir>Northeast</wind_dir>
    <wind_degrees>60</wind_degrees>
    <wind_mph>4.6</wind_mph>
    <wind_kt>4</wind_kt>
    <pressure_string>1037.1 mb</pressure_string>
    <pressure_mb>1037.1</pressure_mb>
    <pressure_in>30.65</pressure_in>
    <dewpoint_string>18.0 F (-7.8 C)</dewpoint_string>
    <dewpoint_f>18.0</dewpoint_f>
    <dewpoint_c>-7.8</dewpoint_c>
    <windchill_string>39 F (4 C)</windchill_string>
        <windchill_f>39</windchill_f>
        <windchill_c>4</windchill_c>
    <visibility_mi>10.00</visibility_mi>
    <icon_url_base>http://forecast.weather.gov/images/wtf/small/</icon_url_base>
    <two_day_history_url>http://www.weather.gov/data/obhistory/KNYC.html</two_day_history_url>
    <icon_url_name>skc.png</icon_url_name>
    <ob_url>http://www.weather.gov/data/METAR/KNYC.1.txt</ob_url>
    <disclaimer_url>http://weather.gov/disclaimer.html</disclaimer_url>
    <copyright_url>http://weather.gov/disclaimer.html</copyright_url>
    <privacy_policy_url>http://weather.gov/notice.html</privacy_policy_url>
</current_observation>

My suspicion is that once the connection is closed the CC3000 socket APIs probably start failing or blocking and are tripping up the client read() function from getting data.

[jimwims]

Thanks very much for passing this along. It seems to be working so far. I will let you know after I test it further.

[adafruit_support_rick]

The WebClient example code has been updated. You might want to start working with that.

[jimwims]

Will do. Tnx.

[jimwims]

The new WebClient example is a huge step forward. Many thanks for the tip and to the folks who updated it.

But, another problem has surfaced. My sketch downloads whether information from a Yahoo web site, extracts and formats the data of interest and then displays it on one of your TFT 1.8 inch display shields. This process is repeated every ten minutes hopefully without any operator intervention for the rest of time or until I pull the power plug. With the new WebClient as the base, the program runs very reliably. Using faster reset rates to test it out, it commonly runs a few hundred iterations flawlessly without the TFT shield installed and using only the serial monitor as an output. But when I install the TFT shield, the exact same program runs at most about 30 iterations, and often less than 10, and then just hangs.

It seems there is some interaction between the TFT shield and the Uno/CC3000 which is causing this problem. Do you have any suggestions as to how I can fix this hanging problem?

Tnx.

[adafruit_support_rick]

Are you using the string library, or doing anything with malloc, or anything else that might do dynamic memory allocation?

[jimwims]

No Strings -- was tempted but then scarred off. No memalloc because I'm not smart enough. Don't believe I am using anything else with dynamic memory allocation.

[adafruit_support_rick]

Sounds like a memory issue nevertheless. There are some things you can do to reduce SRAM usage, like using the F() macro. See this tutorial:
http://learn.adafruit.com/memories-of-an-arduino

[jimwims]

I have been making some progress with this. One issue I have is that occasionally when the sketch attempts to retrieve data from the web page the error messages "Failed to open socket Connection failed" or "Connection error Connection failed" appear on the Serial Monitor Once the error occurs, it is simply repeated and the sketch ceases to function. I have assumed (perhaps incorrectly) that these messages result from the vagaries of the internet and not something that can be corrected by changes to the sketch. Am I right about this? And, if so, is there some way to recover from these errors so that the sketch will resume functioning?

Tnx.

[adafruit_support_rick]

I think those errors are the result of memory corruption. I forget - you're running this on a Uno? You might want to try a Leonardo or a Mega.

[jimwims]

Thanks for your suggestion. I have a few questions.

My most basic problem is the one discussed in my Nov. 9 post. I am using a Uno with your CC3000 and 1.8" TFT shields stacked. The exact same program runs very differently depending on whether the TFT shield is in place. When the TFT is removed, the program will usually run hundreds of iterations and often thousands. When it is put back in place, it runs many fewer iterations before hanging, usually well less than a hundred. I am no expert, but it would seem to me that if my problem is lack of SRAM and/or SRAM corruption, the program would run the same way regardless of whether the TFT is in place. Would you explain why think my problem may be memory/corruption?

Also, I have spent a fair amount of time reading prior posts. Others have reported problems in applications where the CC3000 shares the SPI bus with another shield or board. Is it possible that I am having a similar problem? If so, is there anything I can do about it? FYI -- I am not using the ICSP header for SPI between the Uno and the CC3000; I could change that if it would help.

My current preferred solution is an in-line switch in the power line to the Uno. Not very satisfying, but it does make it easy to power cycle the system.

Thanks for your continued help.

[adafruit_support_rick]

Ohhhhhh... the SPI thing. Ick. I should have thought of that.

I don't know if there's anything you can do about it - there are some coding issues in the CC3000 library, and there may be some electrical issues.

Using the ICSP on a Uno won't help, since those pins are shared with Digital 11, 12, 13. Again, your salvation would be a Leonardo or a Mega, where you can use the ICSP header for CC3000 SPI, while using software SPI for the display.

[jimwims]

The tutorial for your 1.8" TFT display describes two ways to connect them, the more flexible, any pin way and the SPI pin way. I have taken a hard look at my 1.8" TFT shield. It looks to me like it would be possible to cut the traces on the shield to the SPI pins 11-13 and reroute those connections to unused pins 2,6 & 7. I would then adjust the tft initialize command to the new pins.

Two questions:

Do you see any problems with using the TFT shield this way? (I understand it will be slower, but I can live with that.)

Do you think this will solve any problems caused by the CC3000 and TFT shields sharing the SPI bus?

Tnx.

[adafruit_support_rick]

That works. I think it has a good chance of fixing your problems.

[jimwims]

I made the change. WOW! You are absolutely correct. It made a huge improvement. Thanks for all your help and patience.