Synchronizing two SD Data Logger Shields

For RTC breakouts, etc., use the Other Products from Adafruit forum

Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.
Locked
joschen
 
Posts: 12
Joined: Wed Jan 01, 2014 1:59 pm

Synchronizing two SD Data Logger Shields

Post by joschen »

Hello!

I am trying to make two wearable sensors, each "module" with an inertial measurement unit and an SD Data Logger Shield. The sensors also communicate through I2C. When the measurements are being taken, they are not connected to each other or to the computer. Measurements will be taken for about 30 minutes. They will be powered by a battery pack throughout the measurements.

Each module will be taking measurements every 10 miliseconds or so, and I would like to compare the values from both sensors at a specific time, for which I would need that the clocks of both shields are synchronized accurately enough.
The RTC_Milis function from the library offered by adafruit seems like a great solution to be able to retrieve the timestamp at a milisecond accuracy, but the main problem for me is to synchronize both clocks to ensure that their miliseconds are synchronized (or at least as close as possible).

I haven't had any trouble implementing the inertial measurement unit and logging the data on the SD Card, but I am unsure how the Shield sets the time from the computer.

Take the following example:
Let's say I am setting the clock for the first SD shield. I compile+upload the sketch to set the time, and at that moment my computer time is:
0 Hours
0 Minutes
0 Seconds
10 Miliseconds
What time will the data logger shield be set on? Does it account (and read) for the miliseconds from the computer?
Would the resulting time being set be:
1) 0 hours, 0 minutes, 0 seconds and 0 miliseconds?
2) 0 hours, 0 minutes, 1 second and 0 miliseconds?
3) 0 hours, 0 minutes, 0 seconds and 10 miliseconds?

Now let's say I try to set the time of the second data logger. When I upload and compile the program, the computer time is:
0 Hours
0 Minutes
5 Seconds
70 Miliseconds
In cases 1 and 2, miliseconds are not being read from the computer and logged into the RTC. The second data logger would be unsynchronized by an X amount of miliseconds (60 miliseconds for case 1, or 20 miliseconds for case 2), which I may not be able to account for. My measurements would then be unsynchronized, and the comparison between both values will not be accurate.

This of course, brings no trouble when using a single "module" for data acquisition. But if I am trying to use another data logger and compare the data between both shields, it brings some trouble.

How can I ensure that both clocks are synchronized as accurate as possible?
Any solutions to this?

EDIT: Can I use the data logger shield with another I2C sensor? Does the data logger shield use the arduino as a master or as a slave? If the shield is the master, then I am assuming that the sensor would connect as a slave to the Shield? Would the shield then transfer the data from the sensors and the timestamp to the arduino? How does this work?

I am planning on using the MPU6050 inertial measurement unit, which communicates through I2C. I will be pulling the AD0 pin high such that the I2C address is 0x69 for the MPU6050 and not conflict with the SD logger address.
https://www.sparkfun.com/products/11028

EDIT2: I think that the MPU6050 breakout board that I am planning on using uses 3.3V (https://www.sparkfun.com/products/11028). Can I connect it directly with the latest version of the Data Logger shield from adafruit(http://www.adafruit.com/products/1141), or do I need a level shifter?
ARGH, this is so confusing!!

Thank you!!!!!!!

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_rick »

The RTCs on each shield will be set to the compile time of the sketch when you download it. Between the two boards, the times will be close, but not down to millisecond resolution.
Offhand, I'm not sure what the best approach to synchronization would be. I can imagine a pair of special synchronization sketches, master and slave, that use, say, the NTP algorithm to synchronize the clocks. You could do that over a serial link between the two.

Regarding I2C, the Arduinos are each masters, and the DS1307 RTCs are slaves, so you can use other I2C devices at the same time.

The Data Logger Shield has pull-up resistors to 5V on the I2C lines, so you can't attach non-5V-friendly 3.3v I2C devices without a level-shifter.

joschen
 
Posts: 12
Joined: Wed Jan 01, 2014 1:59 pm

Re: Synchronizing two SD Data Logger Shields

Post by joschen »

Thank you for your quick reply!

I am definitely getting a level shifter then.

Would you have any idea the worst resolution that they will be synchronized to?
10 miliseconds? 100 miliseconds?
Are the miliseconds accounted for at all when reading the time from the computer? If not I would assume that the worst case scenario would be a 1 second shift.

I really don't care about the "real world time", I only care about the time drift between both shields' clocks. Also, it doesn't have to be a resolution of exactly 1ms, as close as possible with an acceptable amount of work would be good too.
If I could somehow intitialize them both to 0 at the same time with a sketch, that would be more than enough. Perhaps Serial communication between both Arduinos would be the solution here, as long as they can pass the "initialization" to each other?
Any tips or advice on where I could start looking into this?

My main question is what accuracy (worst case scenario) do you think I would achieve with:
1) Setting the time through the computer on two different times?
2) Setting the time through the NTP algorithm?

Thank you again!!

UPDATE:
I've been reading about the NTP algorithm, and I think I understand what you meant:
1. Set the time of both Modules (arduino + shield) normally with the computer time
2. Connect both modules through Serial Communication.
3. Use the NTP algorithm to determine the offset in time between both modules.
4. Update the time of Module 2 with the offset to sync.


The NTP algorithm already takes into consideration the time it takes to send the data, right?
Is this what you meant?

This is what I understand from the NTP algorithm:
First define modules as client (module 2) and server (module 1).
The client will then try to be synched to the server.

1. Send data through serial communication from the Client to the server. Save the client timestamp when the data is sent (obtained from RTC_milis).
2. The server receives the data through serial communication. Save the server timestamp when the data is received.
3. Once the server receives the data through serial communication, the server sends another data, indicating that it was received. Save the server timestamp when the data is sent.
4. The client receives the data through serial communication. Save the client timestamp when the data is received.
5. Repeat this several times and take the average, and use that.

Then, the offset and delay can be calculated from the following formulae:
offset = [(T2 - T1) + (T3 - T4)] / 2,
delay = (T4 - T1) - (T3 - T2)



I would only use the "offset", as the delay is of little (or no) importance to me.
A positive offset X would mean the client clock is X seconds ahead respect to the server (e.g. server 10 seconds, client 8 seconds).
A negative offset Y would mean the client is Y seconds late respect to the server (e.g. server 10 seconds client 12 seconds)

Is this how it works? Is everything I wrote up there right?
How big must the data packet that is send by the client and server be? Is the smaller the better?
And must both data packets be the same size? (the one the client sends, and the one the server sends?)
Can I just send a single character, e.g. 'H' or 'B'?

But I think this algorithm assumes that the delay to send it one way is the same as the delay to send it back!Hopefully this is a safe assumption for two arduinos?

Thank you!

User avatar
adafruit_support_bill
 
Posts: 88087
Joined: Sat Feb 07, 2009 10:11 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_bill »

I really don't care about the "real world time", I only care about the time drift between both shields' clocks.
If that is the case, then I think the simplest solution is to program it so that a digital input will reset both clocks to an arbitrary time. Have a reset button that plugs into both systems. Press the button to sync the times, then unplug it and start logging.

joschen
 
Posts: 12
Joined: Wed Jan 01, 2014 1:59 pm

Re: Synchronizing two SD Data Logger Shields

Post by joschen »

Thank you for your reply!

I was thinking about that too, using a single button plugged into both Arduino's reset pin. And then running the same sketch to set the time on both clocks to any arbitrary time, resetting them both with a single button should theoretically give me the exact same time set on both clocks.
I could then unplug them and let the shield keep track of time.

But I read somewhere that using a single reset pin for both would give me bouncing, and even with debouncing techniques not be as accurate?
What resolution do you think I would be able to achieve with this technique?
I think a very difficult thing here is that I won't be able to "check" whether they are completely synchronized to a few ms accuracy or not.

Thank you again!

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_rick »

adafruit_support_bill wrote:If that is the case, then I think the simplest solution is to program it so that a digital input will reset both clocks to an arbitrary time. Have a reset button that plugs into both systems. Press the button to sync the times, then unplug it and start logging.
That works. Just be aware that milliseconds are counted on the arduino, not on the RTC. The RTC only counts seconds. There will be drift between the two arduinos, so the longer you wait between syncing and sampling, the more the two will drift apart.

For that matter, if you don't care about real-world time, then you don't need the RTCs at all. Since they don't count milliseconds, they aren't providing any real value.

An alternative would be to use a Chronodot RTC. The Chronodot can be configured to output a 1Hz "square wave" signal. Once you get two Chronodots synchronized with Bill's synchronization switch, you can then use the 1Hz square wave signal to keep the arduino millisecond clocks synchronized. I'd expect you would be able to keep them to within a couple of ms that way.

Regarding the 'bounce', if both computers use a 5ms debounce time, then they should both complete the debounce within a millisecond of each other, so that really isn't a concern.

User avatar
adafruit_support_bill
 
Posts: 88087
Joined: Sat Feb 07, 2009 10:11 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_bill »

I think a very difficult thing here is that I won't be able to "check" whether they are completely synchronized to a few ms accuracy or not.
Assuming that they will get synchronized to within much less than a second, you could have each one output a pulse periodically (say every 10 seconds) and watch them with a logic analyzer or scope and measure the time difference between the pulses.

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_rick »

I just checked the data logger shield schematic, and it looks like the DS1307 also has a square-wave signal output. So you wouldn't need a chronodot after all.

joschen
 
Posts: 12
Joined: Wed Jan 01, 2014 1:59 pm

Re: Synchronizing two SD Data Logger Shields

Post by joschen »

Thank you for your help again!!

What I was thinking was using the RTC second counter to initialize the arduino milisecond counter every second (or so). That way if the RTCs can be synchronized up to a milisecond, the arduino milisecond counter will not be able to drift longer than a second. But that I will figure out after I have both clocks synchronized :lol:

I was thinking about the following:
Connecting a reset switch to both units(arduino+sd logger). The reset switch can have a simple debouncing circuit (and maybe the software as well), and the modules will already be powered by batteries.
Upload the same program after compiling, that simply sets the real world computer time at the time of compilation. Let's say 8:37. The drift between the real world time will then be the time it takes to initialize both units.

The exact same code that I will be uploading to both modules will be:

Code: Select all

#include <Wire.h>
#include "RTClib.h"
 
RTC_DS1307 RTC;

void setup () {
Serial.begin(57600);
Wire.begin();
RTC.begin();
 
if (! RTC.isrunning()) {
Serial.println("RTC is NOT running!");
// following line sets the RTC to the date & time this sketch was compiled
// uncomment it & upload to set the time, date and start run the RTC!
RTC.adjust(DateTime(__DATE__, __TIME__)); //it will initialize at whatever time the sketch was compiled
}
 
}
For this example, when I let go of the reset switch, both sd data loggers will initialize at 8:37. I can then unplug them, (leaving the rtc battery connected), and then just upload my new logging sketches, etc..
This should initialize them both to about milisecond accuracy, right?


I was thinking another option that *may* be easier, but I am unsure if it works.
I am aware I can't plug both shields on the same arduino UNO, since they both have the same I2C address, and the UNO only has a single I2C port.
However, if I use the DUE (which has 2 I2C ports), wouldn't I be able to plug both shields at the same time?

And then just use the following modificiation to the sketch?

Code: Select all

#include <Wire.h>
#include "RTClib.h" //modify this library to allow for second I2C port from due
 
RTC_DS1307 RTC1;
RTC_DS1307_1 RTC2; //exact same code but replaced wire to wire1 to use wire1.xxx commands

void setup () {
Serial.begin(57600);
Wire.begin();
Wire1.begin();
RTC1.begin();
RTC2.begin();
 
if (! RTC.isrunning()) {
Serial.println("RTC is NOT running!");
// following line sets the RTC to the date & time this sketch was compiled
// uncomment it & upload to set the time, date and start run the RTC!
RTC1.adjust(DateTime(__DATE__, __TIME__)); //it will initialize at whatever time the sketch was compiled
RTC2.adjust(DateTime(__DATE__, __TIME__)); //it will initialize at whatever time the sketch was compiled
}
 
}
Do you think this would work better, worse, or the same as the reset switch to synchronize them?
The only downside would be that I have to find an arduino Due...

Thank you again!

User avatar
adafruit_support_rick
 
Posts: 35092
Joined: Tue Mar 15, 2011 11:42 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_rick »

joschen wrote:For this example, when I let go of the reset switch, both sd data loggers will initialize at 8:37. I can then unplug them, (leaving the rtc battery connected), and then just upload my new logging sketches, etc..
This should initialize them both to about milisecond accuracy, right?
Unfortunately, no. The reason why is that the Arduino IDE will always do a recompile before an upload. So you'll always wind up with different compilation times.

Instead, when you release the reset switch, set the clock to some constant time, so it will be the same on both arduinos.
joschen wrote:Do you think this would work better, worse, or the same as the reset switch to synchronize them?
The only downside would be that I have to find an arduino Due...
The problem there is that you'll have the overhead of parsing the date/time strings and communicating with the RTC. Offhand, I don't know what that would be, but it's still probably within 2ms.

Still, I think the reset switch idea is the better approach. This is something you'll have to do periodically, since even the RTCs will drift a bit. I guess whichever way you go depends on what's easier for you and how often you think you'll have to do this synchronization.
joschen wrote:What I was thinking was using the RTC second counter to initialize the arduino milisecond counter every second (or so). That way if the RTCs can be synchronized up to a milisecond, the arduino milisecond counter will not be able to drift longer than a second. But that I will figure out after I have both clocks synchronized
That's what the square-wave signal will do for you. You'll get an interrupt on the exact 1-second tick, and you can adjust your millisecond count in the interrupt service routine.

joschen
 
Posts: 12
Joined: Wed Jan 01, 2014 1:59 pm

Re: Synchronizing two SD Data Logger Shields

Post by joschen »

Perfect!! Thank you again! I'm sorry for all the questions :oops:

I will be using the reset switch option to synchronize both SD Data Loggers then! I will make a de-bouncing circuit for the reset switch, connect both Arduinos to the reset switch, and upload the same code to both Arduinos:

Code: Select all

#include <Wire.h>
#include "RTClib.h"
 
RTC_DS1307 RTC;

void setup () {
Serial.begin(57600);
Wire.begin();
RTC.begin();
 
if (! RTC.isrunning()) {
Serial.println("RTC is NOT running!");
// following line sets the RTC to the date & time this sketch was compiled
// uncomment it & upload to set the time, date and start run the RTC!
RTC.adjust(DateTime("MAY 21 2013","20:40:45")); //you can use this to set initial date/time as you wish
}
Assuming I manage to synchronize both SD data Loggers to exactly the same time using the reset switch, how much should I expect both Shields to drift per hour?
How long do you think they'd be able to stay synchronized within a millisecond accuracy?
I guess I could buy a DS3231 RTC which has better accuracy, and design a new SD data logger shield, but that would be troublesome...

This is the sketch I plan on running to keep the millisecond count, it's an adaptation of a code from John Boxall:

Code: Select all

#include <Wire.h>
#include "RTClib.h"
 
RTC_DS1307 RTC;

void setup () {
Serial.begin(57600);
Wire.begin();
RTC.begin();
if (! RTC.isrunning()) 
   {
     Serial.println("RTC is NOT running!");
   }

Serial.println("Enabling RTC 1Hz Square Wave");
Wire.beginTransmission(0x68); //DS1307 address
Wire.send(0x07); // move pointer to SQW address
Wire.send(0x10); //  sends 0x10 (hex) 00010000 (binary)
Wire.endTransmission();
Serial.println("RTC 1Hz Square Wave Enabled");

Serial.println("Configuring interrupt");
pinMode(2, INPUT); //pin 2 arduino uno int.0
attachInterrupt(0, Second_counter, RISING); //Seconds are counted for voltage changes in SQW shield pin
Serial.println("Interrupt configured for Arduino Uno Pin 2");

uint32_t start_ms=0; //initialize millisecond counters
uint32_t end_ms=0;
DateTime now = RTC.now();
}

void Second_counter()
{
start_ms=millis();
DateTime now = RTC.now();
}

void loop()
{
end_ms = millis() - start_ms; //milliseconds that have passed since last second
    
    Serial.print(now.year(), DEC);
    Serial.print('/');
    Serial.print(now.month(), DEC);
    Serial.print('/');
    Serial.print(now.day(), DEC);
    Serial.print(' ');
    Serial.print(now.hour(), DEC);
    Serial.print(':');
    Serial.print(now.minute(), DEC);
    Serial.print(':');
    Serial.print(now.second(), DEC);
    Serial.println();
    Serial.print('.');
    Serial.print(end_ms, DEC); //milliseconds
    Serial.println();
}
I am unsure how the SQW output works. Does it rise every second, fall every second, or simply changes from low to high every second?
Should the interrupt mode be RISING (edge), a FALLING(edge) or a CHANGE?
The datasheet also says that the SQW pin needs a pull-up resistor to the 5V. In this case I would use a 10kohm pull-up resistor. Would this be Ok?

I am also aware that the millis function doesn't keep counting during the interrupt. Is the time it takes to run the interrupt routine negligible?

Thank you again for all your help!!!! You've all been awesome!!! I love Adafruit and the great support, I'm definitely buying again from here :lol: :D

User avatar
adafruit_support_bill
 
Posts: 88087
Joined: Sat Feb 07, 2009 10:11 am

Re: Synchronizing two SD Data Logger Shields

Post by adafruit_support_bill »

Assuming I manage to synchronize both SD data Loggers to exactly the same time using the reset switch, how much should I expect both Shields to drift per hour?
How long do you think they'd be able to stay synchronized within a millisecond accuracy?
My experience with the DS1307 is that they can drift by a couple minutes per month. This translates roughly into a couple milliseconds per minute.
I guess I could buy a DS3231 RTC which has better accuracy, and design a new SD data logger shield,
You could easily hack one into an existing shield. For the ultimate in accuracy, you could go with GPS shields. That way they would all be synced to the same atomic clock.
I am unsure how the SQW output works. Does it rise every second, fall every second, or simply changes from low to high every second?
a 1Hz square wave will rise and fall once per second. You can trigger on either the rising or falling edge. A 10K resistor should work fine for a pullup. Your interrupt service routine looks short enough that it would not affect millis() timing. But it is really irrelevant given that you are taking the interrupt time as your new start time.

Locked
Please be positive and constructive with your questions and comments.

Return to “Clock Kits (discontinued)”