Black Lives Matter - Action and Equality. ... Adafruit is open and shipping.
0

Adafruit io Arduino WIFI reconnect issue
Moderators: adafruit_support_bill, adafruit

Forum rules
If you're posting code, please make sure your code does not include your Adafruit IO Active Key or WiFi network credentials.
Please be positive and constructive with your questions and comments.

Adafruit io Arduino WIFI reconnect issue

by jdamelio on Sun May 24, 2020 9:27 pm

Hello. I'm having an issue with Adafruit IO not properly reestablishing the connection after losing wifi.

This is for a simple home humidity monitor. The original issue was that the system would run for a few days (3-7 typically), and then Adafruit IO would stop receiving data and I would reset the system manually. The system always reconnects fine after a reset.

I've spent some time messing around trying to fix this. I've tried using io.status() to call wifi.disconnect() and io.connect() when the connection is lost. I removed all the application code and switched to a wifi hotspot on my phone to more easily turn the network on and off. Eventually I determined the io.run() function can actually reconnect to wifi on its own. I had assumed this functionality was not implemented due to my connection losses and some old posts about similar issues, but it looks like I was wrong.

I think there may be a state machine issue or something similar somewhere in the Adafruit IO library. My test sketch can reconnect, but only occasionally. In this case io.run() has to be called a few times, but eventually the connection is restored and my test feed starts receiving data again.

However, other times when wifi cuts out, after a few calls io.run() will start returning 21 (AIO_CONNECTED) and the sketch proceeds to execute like normal, but the data is not received by the test feed. I don't think there is much I can do about this within the sketch because the library is reporting that the connection is healthy.

The only thing that stands out to me is when watching the connections section in the Adafruit IO monitor, the only time the connection is reestablished is when I see a disconnect event and a connect event after wifi has been reset. If there is no disconnect and reconnect, the data is not received even when io.run() returns 21.

I'm using the Metro M4 Airlift Lite, with Arduino. Code is below:

Code: Select all | TOGGLE FULL SIZE
#include "config.h"

AdafruitIO_Feed *TestFeed = io.feed("test-feed");

int testValue = 0;

void setup()
{
  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
 
  Serial.begin(9600);
  while ((!Serial) && (millis()<5000));
  Serial.println("Serial Connected");

  Serial.print("Connecting to Adafruit IO");

  // connect to io.adafruit.com
  io.connect();

  // wait for a connection
  while(io.status() < AIO_CONNECTED) {
    Serial.print(".");
    delay(500);
  }

  // we are connected
  Serial.println();
  Serial.println(io.statusText());

}

void loop()
{
  //check status with the run function
  Serial.print("Adafruit io status (run):");
  Serial.println(io.run());

  //sends test value to adafruit io
  testValue++;
  digitalWrite(13, HIGH);
  Serial.print("testValue:");
  Serial.println(testValue, 1);
  TestFeed->save(testValue);
  digitalWrite(13, LOW);
 
  Serial.println();
 
  delay(3000);
}

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by Systembolaget on Wed May 27, 2020 1:33 pm

This https://forum.arduino.cc/index.php?topi ... msg4617267 is allegedly reliable, although I have not implemented it (I am fighting the same problem as you, but first try to use a different WLAN reconnection routine).

Systembolaget
 
Posts: 251
Joined: Wed Mar 08, 2017 1:01 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed May 27, 2020 6:19 pm

Thanks for the link! That issue sounds similar but I'm not sure if it is the same.

I think my issue is with the Adafruit IO library, since it reports that the connection is healthy, but the data is not received by the feeds. It seems like whatever heartbeat the IO library sends to Adafruit IO would have to be working to report a healthy connection, and prevent the IO monitor page from showing a disconnect. This would mean that it IS reconnecting to wifi, but that something goes wrong after the wifi reconnect. I suppose it's also possible that the wifi isn't actually reconnecting and io.run() is returning 21 in error, but wouldn't that eventually result in a disconnect on the IO monitor page?

I would explore using a watchdog to just reset when the system goes down, but io.run() is reporting a healthy connection.

Are others able to run their airlift devices for more than a few days at a time without manual reset?

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Sun Jun 14, 2020 9:40 pm

I'm still having this issue. I can't keep my device connected to Adafruit IO for more than about 7 days. The test sketch I posted is about as simple as possible. Are others not having this issue?

I'm now trying the same device on thingspeak to see if I can improve reliability there, we'll see what happens.

If anyone has suggestions for what to do I'd love to hear them, all the rest of this project is working excellently, I just can't keep the connection!

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by Systembolaget on Mon Jun 15, 2020 4:40 am

First, WiFi connection/reconnection does not necessarily mean also Internet connection/reconnection. When you think you have not only WiFi connection but you also have Internet connection, you should be able to ping your device. This is how I troubleshooted my random disconnections. No ping = no Internet = no AIO. I did not realise at first that even when I had a WiFi connection (in code WL_CONNECTED = true each loop() execution), the router had not yet established an Internet connection after a reset or unplugging.

Second, one's particular router model can interfere, as it did in my case after weeks of troubleshooting, and a powerline WiFi extender also caused random disconnects. That I found out by looking at my router's system message and error log and looking at the real traffic with Wireshark. There are loads of ESP32 and ESP8622 related "no connection" or "no reconnection" discussions on GitHub. My set-up worked out of the box with a prehistoric Nokia smartphone used as WPA2 WiFi hotspot. That told me part of the story.

Third, I also noticed that the Adafruit IO library https://github.com/adafruit/Adafruit_IO ... afruitIO.h uses the Adafruit MQTT library https://github.com/adafruit/Adafruit_MQ ... uit_MQTT.h under the hood. Also handy to know for troubleshooting. The problem with all of that is one cannot hook up one's computer via USB for two weeks to capture serial monitor output. And later, the set-up will have to reliably connect/reconnect, away from a computer USB tether, entirely on its own. Thus, I used an RGB LED for visual feedback where my code actually is.

It only makes sense to fiddle with code if the router or other network related hardware cannot interfere. That is where I wasted many weeks, thinking it was either bad code or a temporary AIO service failure.

To cut a long story short: First, I removed the powerline WiFi extender. Second, I bought a better router. Third, I changed my connection/reconnection code approach (whether a WiFi function and a MQTT/AIO function called from loop, or an all-in-one state machine called from loop):

1a. WLAN_DOWN_MQTT_DOWN = WLAN DOWN (set-up <-> router), MQTT DOWN (router <-> Internet/AIO) = RGB LED red

1b. WLAN_STARTING_MQTT_DOWN = wait 15 seconds for WiFi.status() = 3, otherwise start at 1a. next loop()

2a. WLAN_UP_MQTT_DOWN = WLAN UP (set-up <-> router), MQTT DOWN (router <-> Internet/AIO) = RGB LED yellow

2b. WLAN_UP_MQTT_STARTING = wait 3 seconds for mqtt. connected() = 0, otherwise start at 2a. next loop()

3. WLAN_UP_MQTT_UP = WLAN UP (set-up <-> router), MQTT UP (router <-> Internet/AIO) = RGB LED green

Now I can pull the plug on my set-up or pull the plug on my router or simulate router failure, and always, the set-up reconnects itself.

Maybe that helps or at least gives you new ideas.

Systembolaget
 
Posts: 251
Joined: Wed Mar 08, 2017 1:01 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Mon Jun 15, 2020 11:27 pm

Thank you for such a detailed answer. Are you using the Adafruit IO Arduino library? How are you determining which state you are in?

I think I have done much of the debugging you described while connected to serial and using my phone's wifi hotspot to quickly simulate outages. What I found was that the library would report that BOTH WLAN and MQTT were healthy, and adafruit io would still not receive data. I didn't feel like I knew enough about MQTT to go further.

I should try using wireshark to monitor router data and confirm what is being sent in this problem state. It really seems like something in the library can occasionally get into a bad state, without it being recognized by the library.

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by Systembolaget on Tue Jun 16, 2020 4:15 am

Maybe start with simple bare-bones code as shown below, using MQTT, which is what AIO uses behind the scenes anyway, and when (re)connecting works for you, after unplugging the power both on your set-up and router to simulate failures, change to using the AIO library. But you can see below that using MQTT directly is not complicated at all. Plus you have access to other functions at a lower level, which may come in handy later.

I am using a Metro Mini with an external AirLift FeatherWing, so you omit the #include SPI and the five AirLift FeatherWing pin definitions, as well as the WiFi.setPins() in setup(), as your board is "all-inclusive".

Code: Select all | TOGGLE FULL SIZE
#include <SPI.h>

#include <WiFiNINA.h> // Adafruit's WiFiNINA fork, use version 1.4.0
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

// Variables that remain constant
#define SPIWIFI SPI // SPI port
#define SPIWIFI_SS 10 // AirLift ESP32 chip select pin
#define ESP32_RESET 4 // AirLift ESP32 reset pin
#define SPIWIFI_ACK 3 // AirLift ESP32 ready pin
#define ESP32_GPIO0 -1 // AirLift ESP32 pin not used

#define WLAN_SSID "#"
#define WLAN_PASS "#"
//#define WLAN_SSID "Smartphone" // Alternative router
//#define WLAN_PASS "qvd75ogz97gouo7l" // Alternative router
#define AIO_SERVER "io.adafruit.com" // MQTT broker/server host
#define AIO_SERVERPORT 8883 // Secure port, 1883 insecure port
#define AIO_USERNAME "#"
#define AIO_KEY "#"

const int intervalWLAN = 11000; // WLAN (re-)connection interval 11s
const int intervalMQTT = 3000; // MQTT (re-)connection interval 3s

enum : byte {
  WLAN_DOWN_MQTT_DOWN,
  WLAN_STARTING_MQTT_DOWN,
  WLAN_UP_MQTT_DOWN,
  WLAN_UP_MQTT_STARTING,
  WLAN_UP_MQTT_UP
} connectionState;

// Variables that can change
unsigned long timeNowWLAN = 0; // Timestamp that updates each loop() iteration
unsigned long timeNowMQTT = 0; // Timestamp that updates each loop() iteration

bool AIOconnected = false; // Flag to enable publish and subscribe

// Instances an object from the WiFiNINA library to connect and
// transfer data with SSL/TLS support
WiFiSSLClient client;

// Instances a client object from the MQTT_Client library with the
// WLAN client, MQTT server, port and login credentials
Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

// Instance publishing objects from the MQTT_Client library; a feed
// is an Adafruit IO specific MQTT topic
Adafruit_MQTT_Publish mytestfeed = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/Mytestfeed");

void setup()
{
  // Serial monitor printing is only needed for debugging
  Serial.begin(9600);
  while (!Serial); Serial.println();

  // Override default pins with the AirLift's breakout board pins
  WiFi.setPins(SPIWIFI_SS, SPIWIFI_ACK, ESP32_RESET, ESP32_GPIO0, &SPIWIFI);
}

void loop()
{
  // A call to this function (re)connects to the WLAN router and MQTT broker
  connectToWLANAndMQTT();

  if (AIOconnected)
  {
    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");

    // To avoid hitting AIO rate limit; later better publish with millis() timer
    // to not block the program
    delay(5000);
  }
}

void connectToWLANAndMQTT()
{

  static byte connectionState = WLAN_DOWN_MQTT_DOWN;

  switch (connectionState)
  {
    case WLAN_DOWN_MQTT_DOWN:
      if (WiFi.status() != WL_CONNECTED)
      {
        Serial.println("(Re)start WLAN connection");
        WiFi.setLEDs(0, 255, 0); // Red = no connection to anything
        WiFi.begin(WLAN_SSID, WLAN_PASS);
        timeNowWLAN = millis();
        connectionState = WLAN_STARTING_MQTT_DOWN;
      }
      break;

    case WLAN_STARTING_MQTT_DOWN:
      if (millis() - timeNowWLAN >= intervalWLAN)
      {
        Serial.println("Wait for WLAN connection");
        if (WiFi.status() == WL_CONNECTED)
        {
          connectionState = WLAN_UP_MQTT_DOWN;
        }
        else
        {
          Serial.println("Retry WLAN connection");
          WiFi.disconnect();
          connectionState = WLAN_DOWN_MQTT_DOWN;
        }
      }
      break;

    case WLAN_UP_MQTT_DOWN:
      if ((WiFi.status() == WL_CONNECTED) && !mqtt.connected())
      {
        Serial.println("WLAN connected. Start MQTT connection");
        WiFi.setLEDs(160, 255, 0); // Yellow = connection to router but not MQTT/AIO (= Internet)
        printWLANStatus();
        timeNowMQTT = millis();
        connectionState = WLAN_UP_MQTT_STARTING;
      }
      break;

    case WLAN_UP_MQTT_STARTING:
      if (millis() - timeNowMQTT >= intervalMQTT)
      {
        Serial.println("WLAN connected. Wait for MQTT connection");
        if (mqtt.connect() == 0)
        {
          connectionState = WLAN_UP_MQTT_UP;
        }
        else
        {
          Serial.println("Retry MQTT connection");
          connectionState = WLAN_UP_MQTT_DOWN;
        }
      }
      break;

    case 4:
      Serial.println("WLAN and MQTT connected");
      WiFi.setLEDs(255, 0, 0); // Green = connection to router and MQTT/AIO (= Internet)
      AIOconnected = true;
      break;
  }
}

void printWLANStatus()
{
  // Print the ESP32's MAC address
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");

  // A call to this function fetches the MAC address
  printMacAddress(mac);

  // Print the SSID of the WLAN network connected to
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // Print the ESP32's IP address assigned by the router
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // Print the router's subnet mask, usually 255.255.255.0
  Serial.print("Subnet mask: ");
  Serial.println((IPAddress)WiFi.subnetMask());

  // Print the rounter's IP address
  Serial.print("Gateway IP: ");
  Serial.println((IPAddress)WiFi.gatewayIP());

  // Print the WLAN router's signal strength received
  long rssi = WiFi.RSSI();
  Serial.print("Signal strength (RSSI): ");
  Serial.print(rssi);
  Serial.println(" dBm");
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}

Skärmbild 2020-06-16 10.05.24.png
Two power/Internet outages forced
Skärmbild 2020-06-16 10.05.24.png (184.59 KiB) Viewed 80 times

Systembolaget
 
Posts: 251
Joined: Wed Mar 08, 2017 1:01 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed Jun 17, 2020 1:06 am

Thank you for the example code. I worked with this a bit to turn it into a full state machine for navigating disconnects.

The first pass through always works great and results in a MQTT connection that successfully sends data to the test feed. If there is a disconnect it will reconnect to WiFi, and then attempt to reconnect MQTT. The MQTT reconnect always fails. The call to mqtt.connect() returns -1.

After failing to reconnect MQTT, the wifi connection also fails and cannot be re-established. From this point on WiFi.begin() always returns 4, even after restarting the network. The system will stay in this state until a manual reset.

If I comment out mqtt.connect(), so mqtt never starts, this sketch can constantly recover from WiFi outages. I can turn the router on and off as many times as I want and it always reconnects. But, after failed calls to mqtt.connect(), wifi.begin() stops working. I did notice that wifi.begin() returns immediately in this state. It seems like the mqtt library should not be able to do this to the wifi library, but it does...

I have tried using mqtt.disconnect(), and wifi.disconnect() before repairing each of the connections, hoping it would clear out whatever is getting messed up, but it doesn't seem to make much difference.

Code: Select all | TOGGLE FULL SIZE
#include <SPI.h>

#include <WiFiNINA.h> // Adafruit's WiFiNINA fork, use version 1.4.0
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

// Variables that remain constant
//#define SPIWIFI SPI // SPI port
//#define SPIWIFI_SS 10 // AirLift ESP32 chip select pin
//#define ESP32_RESET 4 // AirLift ESP32 reset pin
//#define SPIWIFI_ACK 3 // AirLift ESP32 ready pin
//#define ESP32_GPIO0 -1 // AirLift ESP32 pin not used

//#define WLAN_SSID "X"
//#define WLAN_PASS "X"
#define WLAN_SSID "X" // Alternative router
#define WLAN_PASS "X" // Alternative router
#define AIO_SERVER "io.adafruit.com" // MQTT broker/server host
#define AIO_SERVERPORT 8883 // Secure port, 1883 insecure port
#define AIO_USERNAME "X"
#define AIO_KEY "X"

const int intervalWLAN = 5000; // WLAN (re-)connection interval 5s
const int intervalMQTT = 5000; // MQTT (re-)connection interval 5s

enum : byte {
  WLAN_DOWN_MQTT_DOWN,
  WLAN_STARTING_MQTT_DOWN,
  WLAN_UP_MQTT_DOWN,
  WLAN_UP_MQTT_STARTING,
  WLAN_UP_MQTT_UP
} connectionState;

// Variables that can change
unsigned long timeNowWLAN = 0; // Timestamp that updates each loop() iteration
unsigned long timeNowMQTT = 0; // Timestamp that updates each loop() iteration

bool AIOconnected = false; // Flag to enable publish and subscribe

// Instances an object from the WiFiNINA library to connect and
// transfer data with SSL/TLS support
WiFiSSLClient client;

// Instances a client object from the MQTT_Client library with the
// WLAN client, MQTT server, port and login credentials
Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

// Instance publishing objects from the MQTT_Client library; a feed
// is an Adafruit IO specific MQTT topic
Adafruit_MQTT_Publish mytestfeed = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/test-feed");

void setup()
{
  // Serial monitor printing is only needed for debugging
  Serial.begin(9600);
  while (!Serial); Serial.println();

  // Override default pins with the AirLift's breakout board pins
  //WiFi.setPins(SPIWIFI_SS, SPIWIFI_ACK, ESP32_RESET, ESP32_GPIO0, &SPIWIFI);
}

void loop()
{
  // A call to this function (re)connects to the WLAN router and MQTT broker
  connectToWLANAndMQTT();

  if (AIOconnected)
  {
    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");

    // To avoid hitting AIO rate limit; later better publish with millis() timer
    // to not block the program
    delay(5000);
  }
}

void connectToWLANAndMQTT()
{

  static byte connectionState = WLAN_DOWN_MQTT_DOWN;

  switch (connectionState)
  {
    case WLAN_DOWN_MQTT_DOWN:
      if (WiFi.status() != WL_CONNECTED)
      {
        Serial.print("(Re)start WLAN connection:");
        //WiFi.setLEDs(0, 255, 0); // Red = no connection to anything
        //WiFi.disconnect();
        Serial.println(WiFi.begin(WLAN_SSID, WLAN_PASS));
        timeNowWLAN = millis();
        connectionState = WLAN_STARTING_MQTT_DOWN;
        Serial.println("Wait for WLAN connection");
      }
      else{
        connectionState = WLAN_UP_MQTT_DOWN;
      }
      break;

    case WLAN_STARTING_MQTT_DOWN:
      if (millis() - timeNowWLAN >= intervalWLAN)
      {
        if (WiFi.status() == WL_CONNECTED)
        {
          Serial.println("WLAN Connection Successful");
          connectionState = WLAN_UP_MQTT_DOWN;
          printWLANStatus();
        }
        else
        {
          Serial.println("Retry WLAN connection");
          connectionState = WLAN_DOWN_MQTT_DOWN;
        }
      }
      break;

    case WLAN_UP_MQTT_DOWN:
      if(WiFi.status() != WL_CONNECTED){
        connectionState = WLAN_DOWN_MQTT_DOWN;
      }
      else
      {
        Serial.print("WLAN connected. Start MQTT connection:");
        //mqtt.disconnect();
        Serial.println(mqtt.connect());
        connectionState = WLAN_UP_MQTT_STARTING;
        timeNowMQTT = millis();
        Serial.println("WLAN connected. Wait for MQTT connection");
      }
      break;

    case WLAN_UP_MQTT_STARTING:
      if (millis() - timeNowMQTT >= intervalMQTT)
      {
        if (mqtt.connected() == true)
        {
          Serial.println("Successful MQTT connection");
          connectionState = WLAN_UP_MQTT_UP;
        }
        else
        {
          Serial.println("Retry MQTT connection");
          connectionState = WLAN_UP_MQTT_DOWN;
        }
      }
      break;

    case 4:
      Serial.print("mqtt status:");
      Serial.println(mqtt.connected());
      if(WiFi.status() != WL_CONNECTED){
        Serial.println("Lost WLAN");
        connectionState = WLAN_DOWN_MQTT_DOWN;
        AIOconnected = false;
      }
      else if(mqtt.connected() == false){
        Serial.println("Lost MQTT");
        connectionState = WLAN_UP_MQTT_DOWN;
        AIOconnected = false;
      }
      else{
        AIOconnected = true;
        Serial.println("WLAN and MQTT connected");
      }
      break;
  }
}

void printWLANStatus()
{
  // Print the ESP32's MAC address
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");

  // A call to this function fetches the MAC address
  printMacAddress(mac);

  // Print the SSID of the WLAN network connected to
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // Print the ESP32's IP address assigned by the router
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // Print the router's subnet mask, usually 255.255.255.0
  Serial.print("Subnet mask: ");
  Serial.println((IPAddress)WiFi.subnetMask());

  // Print the rounter's IP address
  Serial.print("Gateway IP: ");
  Serial.println((IPAddress)WiFi.gatewayIP());

  // Print the WLAN router's signal strength received
  long rssi = WiFi.RSSI();
  Serial.print("Signal strength (RSSI): ");
  Serial.print(rssi);
  Serial.println(" dBm");
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}


And an example of what I get on the serial monitor during a simulated outage:

21:46:19.249 ->
21:46:20.011 -> (Re)start WLAN connection:4
21:46:25.106 -> Wait for WLAN connection
21:46:30.126 -> Retry WLAN connection
21:46:30.126 -> (Re)start WLAN connection:4
21:46:35.131 -> Wait for WLAN connection

Turn on WiFi hotspot

21:46:40.155 -> Retry WLAN connection
21:46:40.155 -> (Re)start WLAN connection:3
21:46:45.179 -> Wait for WLAN connection
21:46:50.172 -> WLAN Connection Successful
21:46:50.172 -> MAC address: 3C:71:BF:77:9D:08
21:46:50.172 -> SSID: AndroidAP_1929
21:46:50.205 -> IP Address: 192.168.43.136
21:46:50.205 -> Subnet mask: 255.255.255.0
21:46:50.205 -> Gateway IP: 192.168.43.155
21:46:50.205 -> Signal strength (RSSI): -47 dBm
21:46:50.205 -> WLAN connected. Start MQTT connection:0
21:46:55.249 -> WLAN connected. Wait for MQTT connection
21:47:00.274 -> Successful MQTT connection
21:47:00.274 -> mqtt status:1
21:47:00.274 -> WLAN and MQTT connected
21:47:00.274 -> Data published to AIO
21:47:05.282 -> mqtt status:1
21:47:05.282 -> WLAN and MQTT connected
21:47:05.282 -> Data published to AIO

Turn off hotspot to simulate outage

21:47:10.300 -> mqtt status:1
21:47:10.300 -> Lost WLAN
21:47:10.300 -> (Re)start WLAN connection:4
21:47:15.324 -> Wait for WLAN connection
21:47:20.327 -> Retry WLAN connection
21:47:20.327 -> (Re)start WLAN connection:4
21:47:25.377 -> Wait for WLAN connection
21:47:30.373 -> Retry WLAN connection
21:47:30.373 -> (Re)start WLAN connection:4
21:47:35.388 -> Wait for WLAN connection
21:47:40.394 -> Retry WLAN connection
21:47:40.394 -> (Re)start WLAN connection:4
21:47:45.439 -> Wait for WLAN connection
21:47:50.464 -> Retry WLAN connection
21:47:50.464 -> (Re)start WLAN connection:4
21:47:55.484 -> Wait for WLAN connection
21:48:00.496 -> Retry WLAN connection
21:48:00.496 -> (Re)start WLAN connection:4
21:48:05.526 -> Wait for WLAN connection
21:48:10.513 -> Retry WLAN connection
21:48:10.513 -> (Re)start WLAN connection:4
21:48:15.556 -> Wait for WLAN connection
21:48:20.544 -> Retry WLAN connection
21:48:20.544 -> (Re)start WLAN connection:4
21:48:25.580 -> Wait for WLAN connection
21:48:30.588 -> Retry WLAN connection
21:48:30.588 -> (Re)start WLAN connection:4
21:48:35.637 -> Wait for WLAN connection

Turn hotspot back on

21:48:40.627 -> Retry WLAN connection
21:48:40.627 -> (Re)start WLAN connection:3
21:48:45.646 -> Wait for WLAN connection
21:48:50.672 -> WLAN Connection Successful
21:48:50.672 -> MAC address: 3C:71:BF:77:9D:08
21:48:50.672 -> SSID: AndroidAP_1929
21:48:50.672 -> IP Address: 192.168.43.136
21:48:50.672 -> Subnet mask: 255.255.255.0
21:48:50.672 -> Gateway IP: 192.168.43.155
21:48:50.672 -> Signal strength (RSSI): -50 dBm
21:48:50.672 -> WLAN connected. Start MQTT connection:-1
21:49:19.323 -> WLAN connected. Wait for MQTT connection
21:49:24.298 -> Retry MQTT connection
21:49:24.298 -> WLAN connected. Start MQTT connection:-1
21:49:34.418 -> WLAN connected. Wait for MQTT connection
21:49:39.445 -> Retry MQTT connection

I always see two failed MQTT attempts before WiFi fails and can't be recovered

21:49:39.445 -> (Re)start WLAN connection:4
21:49:39.855 -> Wait for WLAN connection
21:49:44.876 -> Retry WLAN connection
21:49:44.876 -> (Re)start WLAN connection:4
21:49:44.876 -> Wait for WLAN connection
21:49:49.874 -> Retry WLAN connection
21:49:49.874 -> (Re)start WLAN connection:4
21:49:49.874 -> Wait for WLAN connection
21:49:54.888 -> Retry WLAN connection
21:49:54.888 -> (Re)start WLAN connection:4
21:49:54.888 -> Wait for WLAN connection

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by Systembolaget on Wed Jun 17, 2020 2:01 am

I don't have your "all-inclusive" Metro, but a Metro Mini with AirLift FeatherWing, so in theory, it's exactly the same tech, and thus, what works perfectly all the time on my end should "in theory" work on your end, too. Is there a difference when you use a smartphone as router instead of your regular router? If so, then you found out something. On GitHub there are some fairly knowledgeable ESP32 users that found out that on some routers, it all ever only works the second time upload/reset, as bizarre as that seems. Some even reverted to an ESP8266 based solution.

It is hard to say without having the original device(s). Maybe someone from Adafruit can chip in.

Systembolaget
 
Posts: 251
Joined: Wed Mar 08, 2017 1:01 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed Jun 17, 2020 2:30 am

Identical behavior from hotspot and router.

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by brubell on Wed Jun 17, 2020 11:33 am

The disconnection from the hardware (esp32) to your router makes me think it's a potential issue with the firmware on the ESP32 itself. Have you tried updating the firmware on your ESP32 to the latest nina-fw? https://learn.adafruit.com/upgrading-esp32-firmware

brubell
 
Posts: 946
Joined: Fri Jul 17, 2015 10:33 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed Jun 17, 2020 2:20 pm

I have updated the esp32 firmware from 1.2.2 to 1.6.1. No change in behavior.

Did a bit more experimenting and noticed that:
1) If mqtt.connect() is called during the first wifi connection it will always succeed.
2) If mqtt.connect is called after wifi has disconnected and reconnected, it will always fail, and cause future attempts to reconnect wifi to fail. Most of my tests saw mqtt.connect() fail after getting a successful mqtt connection and then losing WiFi, but this happens even if there was never a prior mqtt connection.

To be extra clear, the following case never makes an mqtt connection:
-connect to wifi
-simulated outage
-reconnect wifi (succeeds)
-attempt first mqtt connection(fails, disconnects wifi, and cannot reconnect wifi)

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by Systembolaget on Wed Jun 17, 2020 2:33 pm

Ok, how about a good old "two functions permanently called from loop()" approach?

Could you use a smartphone as router and try the code below to see if it publishes every five seconds as the other code above? If I deactivate my smartphone's router function and reactivate it a minute later, my set-up starts publishing again. Again, you'd have to omit the #include SPI and the five AirLift FeatherWing pin definitions, as well as the WiFi.setPins() in setup(), as your board is "all-inclusive". Does this code work for you?

Code: Select all | TOGGLE FULL SIZE
#include <SPI.h>

#include <WiFiNINA.h> // Adafruit's WiFiNINA fork, use version 1.4.0
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

// Variables that remain constant
#define SPIWIFI SPI // SPI port
#define SPIWIFI_SS 10 // AirLift ESP32 chip select pin
#define ESP32_RESET 4 // AirLift ESP32 reset pin
#define SPIWIFI_ACK 3 // AirLift ESP32 ready pin
#define ESP32_GPIO0 -1 // AirLift ESP32 pin not used

//#define WLAN_SSID "#"
//#define WLAN_PASS "#"
#define WLAN_SSID "Smartphone"
#define WLAN_PASS "s39hk48fn2349lly"
#define AIO_SERVER "io.adafruit.com" // MQTT broker/server host
#define AIO_SERVERPORT 8883 // Secure port, 1883 insecure port
#define AIO_USERNAME "#"
#define AIO_KEY "#"

const int intervalWLAN = 1000; // WLAN (re-)connection interval

// Variables that can change
unsigned long timeNowWLAN = 0; // Timestamp that updates each loop() iteration

bool AIOconnected = false; // Flag to enable publish and subscribe

// Status must be set to idle for when WiFi.begin() is called for
// the first time; it remains so until the WLAN connection attempt
// fails, succeeds or is re-established
int status = WL_IDLE_STATUS;

// Instances an object from the WiFiNINA library to connect and
// transfer data with SSL/TLS support
WiFiSSLClient client;

// Instances a client object from the MQTT_Client library with the
// WLAN client, MQTT server, port and login credentials
Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

// Instance publishing objects from the MQTT_Client library; a feed
// is an Adafruit IO specific MQTT topic
Adafruit_MQTT_Publish mytestfeed = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/Mytestfeed");

void setup()
{
  // Serial monitor printing is only needed for debugging
  Serial.begin(9600);
  while (!Serial); Serial.println();

  // Override default pins with the AirLift's breakout board pins
  WiFi.setPins(SPIWIFI_SS, SPIWIFI_ACK, ESP32_RESET, ESP32_GPIO0, &SPIWIFI);

  // Indicate there is no WLAN router connection yet; GRB colour order!
  WiFi.setLEDs(0, 255, 0); // Red

  // Only to better see the status indicating RGB LED's state
  delay(2000);
}

void loop()
{
  // A call to this function connects to the WLAN router
  connectToWLAN();

  // A call to this function connects to the MQTT broker
  connectToMQTT();

  if (AIOconnected)
  {
    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");

    // To avoid hitting AIO rate limit; later better publish with millis() timer
    // to not block the program
    delay(5000);
  }
}

void connectToWLAN()
{
  // Check if the AirLift FeatherWing's WLAN module works
  if (WiFi.status() == WL_NO_MODULE)
  {
    Serial.println("Connection to WLAN module failed");

    // If it doesn't, trap code execution here forever
    while (true)
    {
      // And indicate there is a WLAN module malfunction
      WiFi.setLEDs(0, 255, 0); // Red
    }
  }

  // Return to loop() if already connected to the WLAN router
  if (WiFi.status() == WL_CONNECTED)
  {
    return;
  }

  // Indicate there is neither WLAN router nor Internet connection yet
  WiFi.setLEDs(192, 255, 0); // Yellow

  if (millis() - timeNowWLAN >= intervalWLAN)
  {
    // Update the timestamp for the next loop() execution
    timeNowWLAN = millis();

    Serial.println("Trying to connect to WLAN router");

    WiFi.disconnect();

    // Start connection to WLAN router
    status = WiFi.begin(WLAN_SSID, WLAN_PASS);

    // WL_NO_SHIELD       = 255
    // WL_IDLE_STATUS     = 0
    // WL_NO_SSID_AVAIL   = 1
    // WL_SCAN_COMPLETED  = 2
    // WL_CONNECTED       = 3
    // WL_CONNECT_FAILED  = 4
    // WL_CONNECTION_LOST = 5
    // WL_DISCONNECTED    = 6
    Serial.println(WiFi.status());

    Serial.println("Connection to WLAN router successful");

    // Indicate that the WLAN router connection was established
    WiFi.setLEDs(255, 0, 0); // Green

    // A call to this function prints various connection details
    printWLANStatus();
  }
}

void connectToMQTT()
{
  // Return to loop() if already connected to the MQTT broker
  if (mqtt.connected())
  {
    return;
  }

  if (WiFi.status() == WL_CONNECTED)
  {

    Serial.println("Trying to connect to Adafruit IO");

    // Stores a printable string version of the error code returned by
    // connect()
    int8_t MQTTerrorString;

    // In case the error code is not 0 = successful connection, then
    while ((MQTTerrorString = mqtt.connect()) != 0)
    {
      // Print an error message that matches the error code
      switch (MQTTerrorString)
      {
        case 1: Serial.println("Wrong protocol"); break;
        case 2: Serial.println("ID rejected"); break;
        case 3: Serial.println("Server unavailable"); break;
        case 4: Serial.println("Bad username/password"); break;
        case 5: Serial.println("Not authenticated"); break;
        case 6: Serial.println("Failed to subscribe"); break;
        default: Serial.println("Connection failed"); break;
      }

      // In case of an error
      if (MQTTerrorString >= 0)
      {
        // Indicate that the connection failed
        WiFi.setLEDs(0, 255, 255); // Magenta

        // Only to better see the status indicating RGB LED's state
        delay(2000);

        // Send a MQTT disconnect packet and break the connection
        mqtt.disconnect();

        Serial.println("Retry to connect to Adafruit IO");

        return;
      }
    }

    // Indicate that the MQTT broker connection was established
    WiFi.setLEDs(255, 0, 255); // Cyan

    // Only to better see the status indicating RGB LED's state
    delay(2000);

    AIOconnected = true;

    Serial.println("Connection to Adafruit IO successful");
  }
}

void printWLANStatus()
{
  // Print the ESP32's MAC address
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");

  // A call to this function fetches the MAC address
  printMacAddress(mac);

  // Print the SSID of the WLAN network connected to
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // Print the ESP32's IP address assigned by the router
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // Print the router's subnet mask, usually 255.255.255.0
  Serial.print("Subnet mask: ");
  Serial.println((IPAddress)WiFi.subnetMask());

  // Print the rounter's IP address
  Serial.print("Gateway IP: ");
  Serial.println((IPAddress)WiFi.gatewayIP());

  // Print the WLAN router's signal strength received
  long rssi = WiFi.RSSI();
  Serial.print("Signal strength (RSSI): ");
  Serial.print(rssi);
  Serial.println(" dBm");
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}

Systembolaget
 
Posts: 251
Joined: Wed Mar 08, 2017 1:01 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed Jun 17, 2020 3:47 pm

No that doesn't work for me, the order of events that produces is roughly:
-Wifi connects
-Aio connects
-send some feed data successfully
simulated outage
-Wifi reconnects
-sketch happily keeps sending data to Aio but data is not received. Sketch doesn't know mqtt connection is down.
-wait 5 minutes for Aio timeout
-sketch recognizes mqtt connection is lost (I guess this must happen via a local timeout since the mqtt connection is down?)
-sketch tries to reconnect mqtt and fails
-wifi fails

this is basically the same as above. It works at first, after an outage wifi reconnect still works, but mqtt fails to connect and takes wifi down with it.

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Re: Adafruit io Arduino WIFI reconnect issue

by jdamelio on Wed Jun 17, 2020 4:18 pm

This is even simpler:

Code: Select all | TOGGLE FULL SIZE
#include <SPI.h>

#include <WiFiNINA.h> // Adafruit's WiFiNINA fork, use version 1.4.0
#include "Adafruit_MQTT.h"
#include "Adafruit_MQTT_Client.h"

// Variables that remain constant
//#define SPIWIFI SPI // SPI port
//#define SPIWIFI_SS 10 // AirLift ESP32 chip select pin
//#define ESP32_RESET 4 // AirLift ESP32 reset pin
//#define SPIWIFI_ACK 3 // AirLift ESP32 ready pin
//#define ESP32_GPIO0 -1 // AirLift ESP32 pin not used

//#define WLAN_SSID "#"
//#define WLAN_PASS "#"
#define WLAN_SSID "#"
#define WLAN_PASS "#"
#define AIO_SERVER "io.adafruit.com" // MQTT broker/server host
#define AIO_SERVERPORT 8883 // Secure port, 1883 insecure port
#define AIO_USERNAME "#"
#define AIO_KEY "#"

const int intervalWLAN = 1000; // WLAN (re-)connection interval

// Variables that can change
unsigned long timeNowWLAN = 0; // Timestamp that updates each loop() iteration

bool AIOconnected = false; // Flag to enable publish and subscribe

// Status must be set to idle for when WiFi.begin() is called for
// the first time; it remains so until the WLAN connection attempt
// fails, succeeds or is re-established
int status = WL_IDLE_STATUS;

// Instances an object from the WiFiNINA library to connect and
// transfer data with SSL/TLS support
WiFiSSLClient client;

// Instances a client object from the MQTT_Client library with the
// WLAN client, MQTT server, port and login credentials
Adafruit_MQTT_Client mqtt(&client, AIO_SERVER, AIO_SERVERPORT, AIO_USERNAME, AIO_KEY);

// Instance publishing objects from the MQTT_Client library; a feed
// is an Adafruit IO specific MQTT topic
Adafruit_MQTT_Publish mytestfeed = Adafruit_MQTT_Publish(&mqtt, AIO_USERNAME "/feeds/test-feed");

void setup()
{
  // Serial monitor printing is only needed for debugging
  Serial.begin(9600);
  while (!Serial); Serial.println();

  delay(2000);
}

void loop()
{
    //First wifi connection
    Serial.print("Wifi connecting:");
    Serial.println(WiFi.begin(WLAN_SSID, WLAN_PASS));
    delay(5000);
    printWLANStatus();

    //First mqtt
    Serial.print("MQTT connecting:");
    Serial.println(mqtt.connect());
    delay(5000);
   
    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");
    delay(5000);

    //User should simulate an outage here
    Serial.println("test operator disable wifi");
    delay(5000);
    Serial.println("test operator enable wifi");
    delay(10000);
   
    //Second wifi connection
    Serial.print("Second Wifi connecting:");
    Serial.println(WiFi.begin(WLAN_SSID, WLAN_PASS));
    delay(5000);
    printWLANStatus();

    //Second mqtt
    Serial.print("Second MQTT connecting:");
    Serial.println(mqtt.connect());
    delay(5000);

    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");
    delay(5000);

    //Third wifi connection
    Serial.print("Third Wifi connecting:");
    Serial.println(WiFi.begin(WLAN_SSID, WLAN_PASS));
    delay(5000);
    printWLANStatus();

    //Third mqtt
    Serial.print("Third MQTT connecting:");
    Serial.println(mqtt.connect());
    delay(5000);

    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");
    delay(5000);

    //Fourth wifi connection
    Serial.print("Fourth Wifi connecting:");
    Serial.println(WiFi.begin(WLAN_SSID, WLAN_PASS));
    delay(5000);
    printWLANStatus();

    //Fourth mqtt
    Serial.print("Fourth MQTT connecting:");
    Serial.println(mqtt.connect());
    delay(5000);

    // Dummy data for testing AIO feed and dashboard
    mytestfeed.publish(random(1024));
    Serial.println("Data published to AIO");
    delay(5000);

    while(true){
      Serial.println("Test complete");
      delay(1000);
  }
}


void printWLANStatus()
{
  // Print the ESP32's MAC address
  byte mac[6];
  WiFi.macAddress(mac);
  Serial.print("MAC address: ");

  // A call to this function fetches the MAC address
  printMacAddress(mac);

  // Print the SSID of the WLAN network connected to
  Serial.print("SSID: ");
  Serial.println(WiFi.SSID());

  // Print the ESP32's IP address assigned by the router
  IPAddress ip = WiFi.localIP();
  Serial.print("IP Address: ");
  Serial.println(ip);

  // Print the router's subnet mask, usually 255.255.255.0
  Serial.print("Subnet mask: ");
  Serial.println((IPAddress)WiFi.subnetMask());

  // Print the rounter's IP address
  Serial.print("Gateway IP: ");
  Serial.println((IPAddress)WiFi.gatewayIP());

  // Print the WLAN router's signal strength received
  long rssi = WiFi.RSSI();
  Serial.print("Signal strength (RSSI): ");
  Serial.print(rssi);
  Serial.println(" dBm");
}

void printMacAddress(byte mac[]) {
  for (int i = 5; i >= 0; i--) {
    if (mac[i] < 16) {
      Serial.print("0");
    }
    Serial.print(mac[i], HEX);
    if (i > 0) {
      Serial.print(":");
    }
  }
  Serial.println();
}


It produces this result:

13:03:36.341 ->
13:03:38.341 -> Wifi connecting:3
13:03:54.225 -> MAC address: 3C:71:BF:77:9D:08
13:03:54.225 -> SSID: AndroidAP_1929
13:03:54.225 -> IP Address: 192.168.43.136
13:03:54.225 -> Subnet mask: 255.255.255.0
13:03:54.225 -> Gateway IP: 192.168.43.88
13:03:54.225 -> Signal strength (RSSI): -66 dBm
13:03:54.225 -> MQTT connecting:0
13:04:03.787 -> Data published to AIO
13:04:08.784 -> test operator disable wifi
13:04:13.794 -> test operator enable wifi
13:04:23.779 -> Second Wifi connecting:3
13:04:33.837 -> MAC address: 3C:71:BF:77:9D:08
13:04:33.837 -> SSID: AndroidAP_1929
13:04:33.837 -> IP Address: 192.168.43.136
13:04:33.837 -> Subnet mask: 255.255.255.0
13:04:33.837 -> Gateway IP: 192.168.43.88
13:04:33.837 -> Signal strength (RSSI): -55 dBm
13:04:33.837 -> Second MQTT connecting:-1
13:04:49.086 -> Data published to AIO
13:04:54.088 -> Third Wifi connecting:3
13:05:04.151 -> MAC address: 3C:71:BF:77:9D:08
13:05:04.151 -> SSID: AndroidAP_1929
13:05:04.151 -> IP Address: 192.168.43.136
13:05:04.151 -> Subnet mask: 255.255.255.0
13:05:04.151 -> Gateway IP: 192.168.43.88
13:05:04.151 -> Signal strength (RSSI): -53 dBm
13:05:04.151 -> Third MQTT connecting:-1
13:05:19.261 -> Data published to AIO
13:05:24.271 -> Fourth Wifi connecting:4
13:05:29.823 -> MAC address: 3C:71:BF:77:9D:08
13:05:29.823 -> SSID:
13:05:29.823 -> IP Address: 192.168.43.136
13:05:29.823 -> Subnet mask: 255.255.255.0
13:05:29.823 -> Gateway IP: 192.168.43.88
13:05:29.823 -> Signal strength (RSSI): 0 dBm
13:05:29.823 -> Fourth MQTT connecting:No Socket available
13:05:29.823 -> -1
13:05:34.853 -> Data published to AIO
13:05:39.843 -> Test complete
13:05:40.833 -> Test complete
13:05:41.858 -> Test complete
13:05:42.846 -> Test complete
13:05:43.849 -> Test complete


You can see the sequence is:
-wifi connection succeeds
-mqtt connection succeeds
-publish succeeds
user simulates an outage
-wifi connection succeeds
-mqtt connection fails
-publish fails
-wifi connection succeeds
-mqtt connection fails (it's this second mqtt fail that seems to break wifi)
-publish fails
-wifi connection fails (and I notice that WiFi.begin() returns immediately here, where it usually takes a couple of seconds.)
-mqtt connection fails
-publish fails

Are you able to get more than one good publish from this sequence?

If I don't cycle wifi at all during this sketch, I get 4 publishes, from 4 different mqtt connections, and they all work. So this sketch is good otherwise.

I should mention that I have no shields or other hardware connected. Just a bare metro m4 airlift lite.
Last edited by jdamelio on Wed Jun 17, 2020 4:31 pm, edited 1 time in total.

jdamelio
 
Posts: 15
Joined: Mon Mar 30, 2020 10:18 pm

Please be positive and constructive with your questions and comments.