GPS + millis() Function Discovery

Adafruit Ethernet, Motor, Proto, Wave, Datalogger, GPS Shields - etc!

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

GPS + millis() Function Discovery

Post by lucgallant »

Hi there,

just wanted to post a bit of learnings here. I got my GPS to work, and for all intents and purposes I'm trying to make my Arduino Uno a clock, using the GPS.

After a few hours of debugging/testing, I found that it took 70 seconds for millis() to reach the 60 second mark.

The GPS code that runs every millisecond with TIMER0 (bundled into that interrupt), slows millis() down by ~16%.

I only noticed this because the Time library I'm using would drift from the actual GPS time. Time would reach 60 seconds but GPS was already at 70.

Anyway, what I'm doing now is putting the GPS to sleep (not so much sleep as not running any reads/gets) and only awaking it every hour (eventually might be days even).

I'm pretty impressed with this chip, I'm in my basement and it still gets 8 satellites!

Hopefully this is useful to someone, and may have already been brought up. Thanks!

P.S., I just read the last FAQ on the GPS breakout, looks like its RTC will keep the time - though I think I'll keep the Time library due to great UTC - Mountain Time Conversion.

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

Interesting..

It's generally known that making TIMER0 do anything else can throw off the millis() function, but this is the firs time I've heard of a specific connection to the GPS module. Thank you for characterizing it and posting the information!

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

No worries.

Though, I do have a question. In principle I'm making a clock. I want the clock to update from the GPS every once in a while. Could be every 30 minutes or even 2 days, depending how accurate I deem the Time library to be.

Because of what I said in the OP, I want to hush the GPS such that when I want an update, it comes right away. I don't want to lose fix because then it could take me 30 minutes to fix to get my update.

The way I've done it so far, is to disable the code in the interrupt as such:

Code: Select all

// Interrupt is called once a millisecond, looks for any new GPS data, and stores it
SIGNAL(TIMER0_COMPA_vect) {
  if(GPSFUNCTION)
  {
    char c = GPS.read();
    // if you want to debug, this is a good time to do it!
#ifdef UDR0
    if (GPSECHO)
      if (c) UDR0 = c;  
    // writing direct to UDR0 is much much faster than Serial.print 
    // but only one character can be written at a time. 
#endif
  }
}
I've also disabled the following code:

if(GPSFUNCTION)
{
// if a sentence is received, we can check the checksum, parse it...
if (GPS.newNMEAreceived()) {
//Serial.println(GPS.lastNMEA()); // this also sets the newNMEAreceived() flag to false

if (!GPS.parse(GPS.lastNMEA())) // this also sets the newNMEAreceived() flag to false
return; // we can fail to parse a sentence in which case we should just wait for another
}

So for testing purposes, GPSFUNCTION is false for 1 minute, and then true for 5 seconds.

The main issue I have is that when I switch it to true, some old times come back from the GPS, from before I put it to sleep.

So, the workaround I put in, is to read through some data and junk it, and then proceed, as such:

Code: Select all

    //just woke up, let's go through a bit of garbage data first    
   //just woke up, let's go through a bit of garbage data first    
    for(long i = 0; i<50000; i++)
    {
      // if a sentence is received, we can check the checksum, parse it...
      if (GPS.newNMEAreceived()) {
        if (!GPS.parse(GPS.lastNMEA())) 
       {
         break;
       }
      }
      j = i;
    }   
    Serial.println("Broke after: " + String(j) + " iterations!"); 
This isn't very efficient, and takes about 2 seconds to process. Isn't very reliable either because my "50000" may or may not be enough... I was using 5000 earlier and it isn't enough. The loop finishes and the GPS time values are still from before we stopped polling.

Any suggestions as to how to make this reliable? I tried standy() and wakeup(), but that loses fix entirely. I don't have a battery in my GPS at the moment.

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

Adding a battery will improve the restart fix time dramatically.

The GPS module works by calculating the positions of the satellites from the radio signals it receives, predicting where they'll be at the next update, then comparing that prediction to the next reading. If you have a battery, the module can work in 'reduced precision' mode where it just extrapolates the satellite positions until it gets enough valid readings. When you cold-start, it has to assemble all that information from scratch.

To keep your clock honest, I'd suggest doing a simplified version of what the global time server network does: iterative estimation.

Set up a pin-change interrupt on one of the Arduino's pins and connect it to the GPS module's PPS output. Those pulses are aligned to the beginning of a GPS second to within 10ns, so the time between any two pulses will be an integer +/-10ns.

The ISR can be dead-simple.. just set a flag named "PPS_ARRIVED" and test that at the beginning of each pass through your main loop. If it's true, clear it and record the current millis() value.

Keep a running average of millis() counts per PPS_ARRIVED times and use that average to estimate seconds. The longer you collect values, the more accurate the average will get. Then once a day you can check the time against the NMEA sentences.

Alternatively, you can use an RTC to handle the timekeeping for you. The ChronoDot is accurate to within a few seconds a month: https://www.adafruit.com/product/255

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

Thanks for this detail.

I guess the question remains though, how to reliably put the thing to sleep? I don't want to be running that code every millisecond because that's what's slowing down millis().

As in, how to effectively check against the NMEA sentences once per day?

When I start reading again, is the way that I'm dumping the "garbage" data the most effective way or is there a quicker or more efficient way of doing that? This ties up my Uno for a few seconds straight, which mean my motor will stop spinning. I can do that in the night, no biggie, I'd just rather the efficiency instead...

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

There's a hardware way and a software way.

- The hardware way is to take the GPS module's power from one of the Arduino's pins. The GPS module wants 20-25mA for normal operation. I have one hooked up to an ammeter right now and it's running just a hair below 24mA, with occasional bursts up to 34mA when getting a fix from a new satellite.

An Arduino pin's upper limit is 40mA, so it should be possible to power the module from a single pin. If you want to be extra sure, you can use the Arduino to control a transistor which control's the GPS module's power.

- The software way is to send the command "$PMTK161,0*28" with a newline/carriage-return line ending ("Both NL and CR" in the Arduino IDE's Serial Monitor). That puts the module into "standby mode", where it draws a steady 1.63mA by my measurement.

The module will remain in standby mode until you wake it up again by sending any character (even an "\n\r" terminator) through the serial connection. The low current is enough to keep the satellite tables in memory, but you'll still have to wait a while for the module to get a fix again. In my tests using sleep periods of a few minutes, it takes maybe 5-10 seconds to reaquire a fix after waking up, compared to a couple of minutes after power cycling without a battery.

Since you only plan to calibrate against the module once a day, you can turn the module on and let it run for 5-10 minutes while ignoring (or suppressing) its output. After you've given it time to acquire a fix, you can start reading the serial input again and check for a good fix.

This document has a list of the commands the module accepts: http://www.adafruit.com/datasheets/PMTK_A08.pdf

Among the options are faster/slower refresh rates, and control over which NMEA sentences are emitted and how often they're emitted.

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

Thanks for the feedback.

Until I can obtain a battery, I'm not going to put the GPS in standby officially. Just going to ignore what it has to say and then wake it up, but give it some time to wake. I'll leave that for a project optimization at a later date. What about using the Arduino 3V output into the VBAT input? Is there a link to to the breakout board circuit diagram?

Anyway I've decided to do a state machine with gpsAwake, gpsWakingUp, gpsSleeping. It's nothing really fancy, but here are the main blocks:

Globals declared:

Code: Select all

unsigned long gpsSleepTime = 3600000; // gps to sleep for one hour
unsigned long gpsAwakeTime = 10000; //gps to be on for 10 seconds
unsigned long gpsWakeUpTime = 20000; //take 20 seconds to wake up parsing the data
boolean gpsWakingUp = false;
boolean gpsAwake = true;
boolean gpsSleeping = false;
Code in loop():

Code: Select all

  
if (timer2 > millis())
    timer2 = millis();

  //state machine with three possible states
  //start waking up GPS
  if (gpsSleeping && ((millis() - timer2) > gpsSleepTime))
  {
    //need to wake the GPS up slowly so that it doensn't give us bad data
    GPSFUNCTION = true;
    gpsSleeping = false;
    gpsAwake = false;
    gpsWakingUp = true;
    timer2 = millis();    
    Serial.println(F("Waking GPS Up!"));    
    lcd.setCursor(0, 3);
    lcd.print(F("GPS Wake (Auto) "));
  }
  //time to put the GPS to sleep
  else if (gpsAwake &&  ((millis() - timer2) > gpsAwakeTime))
  {
    GPSFUNCTION = false;
    gpsSleeping = true;
    gpsAwake = false;
    gpsWakingUp = false;
    timer2 = millis();
    Serial.println(F("Turning GPS Off!"));    
    lcd.setCursor(0, 3);
    lcd.print(F("GPS Off (Auto)"));  
    //GPS.standby();
  }
  //GPS is fully awake, turn it on, allow the data to be used
  else if (gpsWakingUp && ((millis() - timer2) > gpsWakeUpTime))
  {
    gpsSleeping = false;
    gpsAwake = true;
    gpsWakingUp = false;
    timer2 = millis();    
    Serial.println(F("Turning GPS On!"));    
    lcd.setCursor(0, 3);
    lcd.print(F("GPS On (Auto) "));
  }
Then, from the Time library, I don't let the "Time Update" function return unless I know we're fully awake. Anyway it seems to work well - if anyone needs any more detail please let me know.

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

Also I wanted to add more detail.

I implemented the PPS strategy suggested.

With the GPS on (code in the interrupt running, sentences polled, etc), the Time per pulse is ~840-860 ms.

With the GPS off (code in the interrupt not running, all GPS reads bypassed), the time per pulse is ~950 - 990 ms.

I think my only next step is to figure out how to use this data to bias the Time library on a continuous basis, but make it look natural.

I think the best way is to modify the time library to use the PPS signal to increment its time value every second, instead of how it uses millis() currently.

Or I could just dump it all and use an RTC which I'm getting pretty close to doing . . .

Thanks again!

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

If you're using Paul Stoffregen's Time library (https://github.com/PaulStoffregen/Time) there's a function named 'adjustTime()' that will tweak the counter. Every time you identify a new second, call:

Code: Select all

    adjustTime( 1000 - PPSAverage );
and it will bump the counter up. The next PPS pulse should arrive when the counter hits 1000.

That will throw off your absolute count of milliseconds per PPS pulse though, so it will be easier to track the offset than the actual duration. There are some subtleties to that when the target can both jitter (small changes) and jump (large changes), but this tests out well:

Code: Select all

    int gap = millis() - millisAtLastPPS;
    int step = gap - offset;
    
    offset = (( $step * $step ) < 10000 ) 
        ? (( offset * 2 ) + gap ) / 3
        : gap
    ;
It averages the offset over small changes but starts fresh for large ones.
lucgallant wrote:Or I could just dump it all and use an RTC which I'm getting pretty close to doing . . .
Hey, you're among people who think building a 555 chip or 741 op amp out of discrete transistors is awesome. ;-)

https://www.adafruit.com/products/1526
https://www.adafruit.com/product/2086

There's a lot of "the journey is its own reward" 'round here. ;-)

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

Thanks,

I'll keep this in my back pocket. I am using that Time library. One thing is that the sysTime in library is only accurate to 1 second, and so if I'm off by 50 ms, it isn't much use to use the adjustTime function.

I let the clock run for a day using my new strategy of using the GPS PPS to increase my time, and it is exactly dead on with the NIST clock online (http://time.gov/HTML5/).

Thanks for all the help, I'll report back in a few months as this project reaches completion.

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

Glad to hear it's working the way you want.

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

One question about efficiency.

My program is on the borderline of memory issues...

This code, from the GPS example:

Code: Select all

void useInterrupt(boolean v) {
  if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    ////Block both these out since using timer0Func
    OCR0A = 0xAF;
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  } 
  else {
    // do not call the interrupt function COMPA anymore
    TIMSK0 &= ~_BV(OCIE0A);
    usingInterrupt = false;
  }
}
When I run the freeRam function on my program, I get 303 free bytes.

In my sketch I'm using the interrupt. So, If I comment out all this code as such:

Code: Select all

void useInterrupt(boolean v) {
  //if (v) {
    // Timer0 is already used for millis() - we'll just interrupt somewhere
    // in the middle and call the "Compare A" function above
    ////Block both these out since using timer0Func
    OCR0A = 0xAF;
    TIMSK0 |= _BV(OCIE0A);
    usingInterrupt = true;
  //} 
  //else {
    // do not call the interrupt function COMPA anymore
  //  TIMSK0 &= ~_BV(OCIE0A);
 //   usingInterrupt = false;
//  }
}
The free memory lists as 264 bytes! I'm just wondering why in the world I'd lost 30 bytes by commenting out a bunch of stuff. Any ideas?

I need to make this program as efficient as possible otherwise I'll have to buy a Mega...

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

Couple of things:

First, the freeRAM() function reports heap space, and isn't directly related to the size of the code stored in Flash memory.

Second, the compiler optimizes the code during compilation, and compiler optimization is something of a dark art. One possibility is that changing that chunk of source code made it less attractive to hold some value in one of the microcontroller's registers, so that was shoved out to SRAM to make some other operation faster during execution or smaller in terms of Flash footprint.

If you haven't already moved all your string variables out to program memory with the F() macro, start there.. it's an easy way to free up a few bytes because they'll have to be stored there anyway. Next, try eliminating as many String objects as you can and replace them with a single, global char buffer. That will prevent repeated memory allocation every time you need a new string, which in turn prevents heap fragmentation.

After that, try converting as much of your math as possible to 16-bit integer operations. The ATmega328 doesn't have a floating point unit, so it has to do all that math the hard way. On top of that, floats have a 32-bit representation (which is only good for about 7 significant figures) so the ALU has to store the pieces it isn't working with in registers or SRAM while it performs each operation.

lucgallant
 
Posts: 39
Joined: Sat Mar 15, 2014 12:21 pm

Re: GPS + millis() Function Discovery

Post by lucgallant »

Thanks for the tips.

I've F()'ed all my strings already, though I can only do it with ones that don't change.

In terms of replacing Strings with global char buffers, do you mean a single char buffer for each object or one char buffer for the whole program? Since I've got a 20x4 LCD, I've got a lot of Strings on the go.

From reading the one Adafruit page on memory, I see that I should be using String.reserve().

So just for clarity, if I've got the following:

String s = "";

And I know that the string won't ever be more than 20 characters long, are you suggesting I should make it an array as such?

char c[20] = "";

I was going to just try s.reserve(20) which would have similar benefits.

Or something of the sort?

If even there's a resource or page I could be pointed to that would be great. Thanks.

User avatar
adafruit_support_mike
 
Posts: 67446
Joined: Thu Feb 11, 2010 2:51 pm

Re: GPS + millis() Function Discovery

Post by adafruit_support_mike »

You want one global buffer for the whole program, adjusted in any way that makes sense.. if you have a 16x2 LCD, you can use two char[17] buffers named lcdLine1 and lcdLine2. You might even give yourself a couple of extras to make things like copying easier.

The malloc() routine will always try to find the smallest contiguous block of memory that's big enough to hold the latest request, which leads to problems if you allocate and release lots of items of different sizes. If you allocate space for "8-chars!", create another 1-byte object, then release "8-chars!", you'll have a 9-byte hole in the heap (8 characters plus the 0 terminator). If you create the string "10-chars!!", the 9-byte hole isn't big enough to hold it, so you'll lose another 11 bytes off the top of your heap.

If you create a new string "4chr", malloc() is smart enough to put it into the 9-byte hole, but that still leaves a 4-byte hole of unused memory.

The overall process is called 'heap fragmentation'.

The two strategies for avoiding heap fragmentation are garbage collection (where you periodically re-pack all the storage and re-map all the variables) and using malloc() as little as possible.

Garbage collection is a theoretically Hard problem, but we've done it enough to have techniques that are less than perfect but work really well in practice. Thing is, all the techniques require more processing resources than an Arduino has.

Global buffers move the process of managing memory out of malloc()'s hands and into the programmer's. If you give yourself four 17-byte line buffers or a 68-byte scratchpad, you can decide which bytes need to be kept and which ones can be discarded at any point in the program. A programmer who's watching memory carefully will always be able to manage it more efficiently than malloc()'s one-size-fits-all approach.

WRT general information about memory management, hit the search engines with "memory managment for embedded systems" and read everything you can find. It's more a collection of strategies and principles than "use this code" solutions. It also tends to depend heavily on the CPU that's executing the code. Techniques that work well on one CPU can make problems worse on another.

Locked
Please be positive and constructive with your questions and comments.

Return to “Arduino Shields from Adafruit”