TUNE your Ice Tube CRYSTAL: HARDWARE or J. Archie software

[jarchie]

Maybe a surface mount could be squeezed on the bottom? Provided something can be found to squeeze in, would that work to fine tune the clock if the other cap is left at 10pf?

I know this question is directed at Russell, but I can't resist commenting.

The crystal load already is already asymmetric, even though the ATmega328p datasheet recommends a symmetric load. XTAL1 has a pin capacitance of 18 pF; XTAL2, 8 pF. XTAL1 is the input to the inverting amplifier; XTAL2, the output. I'm no expert on oscillator circuits, but I think that using more capacitance on XTAL1 (crystal output, inverter input) means that the output of the crystal on XTAL1 is dampened a bit by the additional capacitance, and using less capacitance on XTAL2 (crystal input, inverter output) means that the microcontroller's inverting amplifier uses less current to drive the crystal. Making the crystal load more symmetric might be a good idea. Maybe try a a 1-10 pF varcap on XTAL1 and a 15 pF cap on XTAL2?

You generally say that your firmware initially needs to be kick started in order to begin to work for drift correction and this is done by the user making a manual correction after a period of time of about a week or two or after greater than a 15 second difference is noted.

I give this advice for people who would like the drift correction to kick in as soon as possible. I suspect most users won't want to go through the trouble. They'll just wait until their clock has drifted for a few minutes, correct the time like they normally would, and probably never notice that their clock somehow became more accurate. If those users set the time exactly, the drift correction will be similarly exact. If those users set the time within a minute or so, the drift correction might not be perfectly exact, but it should still be an improvement.

Is it possible to kick start the drift correction automatically by something along the lines of the following?

If crystal speed is being measured, how would the clock determine the length of an hour without using the crystal? Are you thinking of temporally attaching an external time reference such as the PPS output of a GPS module?

You've probably already seen this option, but there is way to kickstart the drift correction immediately if you already know--or are willing to measure--how fast or slow the clock runs. Defining the AUTODRIFT_PRELOAD macro in config.h preloads the specified drift correction value as the first element in the 7-element EEPROM array that holds the automatically calculated drift correction values; that makes it equivalent to going through the first let-the-clock-drift-by-at-least-15-seconds-and-correct-the-time cycle.

This is probably more accurate than someone manually making a time correction. I can only get two clocks to within about a quarter to half second of each other and the time base when manually setting them or initially correcting time.

The situation is a bit worse than that because there is up to a second of rounding error as well, so given the human error, I'd place accuracy at about two seconds of total time-setting error. But that really isn't so bad. Let me describe a plausible use scenario:

A typical Adafruit clock (unmodified, except for installing an ATmega328p and xmas firmware) might drift by 20 seconds in a week, so a user anxious to enable drift correction might set the time after a week. Two seconds of time-setting error means that the automatically calculated drift correction factor will have an accuracy of 2 seconds / (7 days * 24 hours/day * 60 minutes/hour * 60 seconds/minute) = 3.15 ppm. So with temperature-related error, the user might expect an accuracy within 5 ppm.

It will take a month or two for the clock to drift by more than fifteen seconds, so the next automatically calculated drift correction factor will have an accuracy of 2 seconds / (30 days * 24 hours/day * 60 minutes/hour * 60 seconds/minute) = 0.74 ppm. Factor in temperature variation, and the error of the automatically calculated drift correction might be more like 3ppm.

Two automatically calculated drift correction values are now stored in the 7-element EEPROM array for drift correction values. The clock is supposed to use the median value, but since there are only two values, there is no single median. Instead of averaging the two values, the clock again uses the first estimate. (I think... actually I don't remember which one is used, but it doesn't really matter.)

After another month and another time correction, another 3 ppm drift correction value is added to the 7-element array. At this point, the median drift correction value in EEPROM will be accurate to about 3 ppm. It will take a couple months for the clock to drift by another 15 seconds, and the accuracy of the next automatically calculated drift correction factor will be even better. At that point, the drift correction has pretty much converged in terms of accuracy.

Eventually, the 7-element array of drift correction factors will be full, and the oldest value--typically the least accurate--will be overwritten. The fact that old values eventually get overwritten helps the automatic drift correction adapt to gradual crystal aging.


I know Russell and Phil have already seen this information, but the gory details of the automatic drift correction algorithm are described elsewhere. The implementation itself is spread across several functions in time.c: time_init, time_autodrift, time_settime, time_newdrift, and time_loaddriftmedian. The time_autodrift function is the one that is called once per second to perform 1/128 second adjustments as needed.

[phild13]

I can program industrial robots and plc's, but I'm not really much of a software guy when it comes to this stuff.

Don't think I don't appreciate the work you have done, I do very much. The firmware and research to get to this point is a great assett to the community. The firmware along with some minor hardware modifications makes the clock plenty accurate for everyday use as an alarm or desk clock and I for one don't really need anything any more accurate than I have made my clocks. This is especially true when non GPS commercial clocks can be off by as much or more as the default Adafruit clock is per week. I generally don't mind a clock (or watch for that matter) that runs a bit fast as it tends to keep me on time. I don't like time to be off by more than 10 minutes though, so I generally will reset the clock to the correct time if it gets off by more than this amount or if there is a power outage as I have to stop the blinking (on most clocks) anyway..

Since all icetube clocks built have unknown variables in them in the form of manufacturing tolerances of the components used and what components are actually used (22pf vs 10pf, 10% vs 0.5%) then all clocks will be off by some order in their time keeping. Provided the clock is using your software and no GPS, then we have several choices.

We can do nothing and when the clock gets off by an undesirable amount of time, make a correction which will in most circumstances provide the initial value needed to start auto correction. This value will be off by whatever the end user decides is close enough when correcting the time.

We can try to direct the end user to make a correction after a period of time or if the clock time varies by a certain amount. This value will also be off by whatever the end user decides is close enough when correcting the time.

We can apply a fixed value when setting up parameters in the firmware as an educated guess to start the correction as a preload. This will be off by some order of magnitude for the particular clock.

We can try to determine what to preload automatically and that will also be off by some order of magnitude for the particular clock.

If crystal speed is being measured, how would the clock determine the length of an hour without using the crystal?

I know a preload can be loaded into the clock to essentially start off the auto drift correction at a fixed point of correction, but the firmware does not (last I looked, I don't think it does) load a preload value as a default. What I was getting at was the clock already knows when an hour has passed so we use that as the time base not the crystal. The clock just does not know if the hour was a fast hour or a slow hour. Now it doesn't really matter if the check value recorded is accurate, just that there is enough resolution in the recorded number value over a period of time to make a comparison after the check value is recorded for a second hour. The tested number value will be greater than or less than itself (the initial recorded value) and would give an initial indication if the crystal is fast or slow in a particular clock.

You need a time period and a number value that is based on the crystal in some way that can provide sufficient resolution to see the amount of variation in the time period.

So after a suitable warm up period to somewhat stabilize the clock components, record a suitable resolution of a number value (cycles, counts, ticks, clock interrupts, ms, or whatever) preferably based on the crystal for a known, even if inaccurate, time period. This is the first check value to record. I suggested something like an hour for a time period but any time period of an hour or more should be great enough. Even if inaccurate the time period will vary and this is what we want to find out and by how much. You need a high resolution on the check number but you don't want a time period so great as to cause an overflow error condition on the check number. If you can pick up milliseconds somewhere, then that resolution in an hour would probably be good for this as we are not really looking for wildly accurate, just a < or > comparison, as we already know the clock is not accurate. Then record again for the same time period and resolution a second check value. Then compare the two values recorded. If a suitable resolution is used as the measurement then the two recorded counts will be different because of the crystal running fast or slow.
Convert this value to what you need to use in the adjustment algorithm and apply the appropriate adjustment as the initial adjustment. From that point on, the clock should start performing the auto drift correction.

You can probably even alter the existing auto-drift to stuff a value after 24 hours as the initial preload value if the preload value is zero which would indicate that the end user did not preset a preload value in firmware. This might be much simpler to implement. Again, we don't really care if the initial preload value stuffed is actually incorrect, as the auto-drift will begin to correct for that as it works on accuracy.

[jarchie]

Don't think I don't appreciate the work you have done, I do very much. The firmware and research to get to this point is a great assett to the community.

Thank you!

What I was getting at was the clock already knows when an hour has passed so we use that as the time base not the crystal.

I'm sorry... I still don't understand. How would the clock detect that an hour had passed without using the crystal?

Consider the simple case where the clock has no software temperature compensation, and drift correction has yet to enable itself. The hour advances every 117964800 crystal oscillations (60 minutes * 60 seconds/minute * 32768 oscillations/second). The clock could count the number of ~1/128 second ticks--which happen every 256 crystal oscillations--but in one hour, the result will always be 460800 (117964800 / 256). So counting the ~1/128 second timer2 ticks would not be useful in establishing an initial drift correction estimate, but I may simply misunderstand what you're proposing.

You can probably even alter the existing auto-drift to stuff a value after 24 hours as the initial preload value if the preload value is zero which would indicate that the end user did not preset a preload value in firmware. This might be much simpler to implement. Again, we don't really care if the initial preload value stuffed is actually incorrect, as the auto-drift will begin to correct for that as it works on accuracy.

With the default settings, the existing algorithm is already equivalent to having the clock use an initial drift correction of 0 ppm, and then updating the amount of drift correction over time.

I'd like to make sure that we're on the same page as far as how the drift correction algorithm works, so please forgive me if you know all this already:

The only way to tell if a clock is running fast or slow is to compare it to a more exact reference. When people set clocks, they generally correct the time to match a more exact reference, so the automatic drift correction algorithm uses time-sets not only to set the new time, but also to compare the old time to a more exact reference. So only when someone corrects the time will the clock determine or update the amount of drift correction to be applied in the future.

[russell 27]

Crystal DATA.zip

(942.12 KiB) Downloaded 64 times

Phil, read some of this data, It should help you make some choices. The overall complexity here is quite astonishing. If load capacitance is too large, crystal will oscillate too slow, if load capacitance is too small crystal will oscillate too fast. Capacitance is considered series, so C1 X C2/ C1 + C2 or two equal capacitors would be cut in half. IE: 2 20 Pf in series = 10 Pf. You also have to consider stray capacitance and circuit capacitance, and crystal character. 2 10 Pf tuned my Archie clock to 32767.9, so I finished off with the dreaded software correction; if I set it manually it does work good. In another clock 2 8 Pf gave the same 32767.9 Hz. Some information tells to use two equal capacitors, some information suggests an unbalanced load can be of benefit in some instances with negative feedback. After I changed the crystal, either a 10 Pf and an 8 Pf or two sevens were as close as I could get. To really hone you would benefit from a variable capacitor. Don't forget Pin 9 is the input, and usually the smaller of the two capacitors seemed to be placed there. You don't need a scope or a frequency meter, but it's a big help. If nothing else it will show the actual frequency for accurately defining software setting. Accuracy is of course a must, and a .1 hertz resolution is really needed, this pretty much rules out the high dollar fluke, with only a 1 hertz resolution at that frequency. My oscilloscope must be fairly accurate, I've had good success in tuning with it. If you are going to experiment with values, I recommend soldering the leads on long, on the bottom of the board, much easier to get off to try another combination. If you want to know how to send crystal frequency to pin 14 so it's buffered and can be read, I can go over the sequence, may already know that. As mentioned, It is also useful to do this even if you just want an accurate place to start with software correction. You can always just solder on a stub wire from Atmega pin 9 (crystal in) and bug on the external oscillator that way. By building it on a bread board, three jumper wires can attach circuit to clock; just after the battery diode for Vcc; Ground to breadboard ground; and output to the stub that was soldered. This is how I added mine to test in my Adafruit clock. If you can't find a 14Dip 4069, I could probably send one to you.

[phild13]

Russell, I will be happy to read up on the information you posted when I have some time.

John, it is probably I who lacks enough understanding of the firmware or else I lack a way to properly convey my thoughts at this point.

At any rate I will do some reading and research when I have some time to devote.

[jarchie]

At any rate I will do some reading and research when I have some time to devote.

If it helps, the drift correction implementation is described in this post, and the code itself is in time.c--in particular, the time_init, time_autodrift, time_settime, time_newdrift, and time_loaddriftmedian functions.

[phild13]

Thanks, I will take a look at that also.

[russell 27]

CLOCK TEST.JPG (121.82 KiB) Viewed 789 times

JOHN

Earlier, I mentioned that I found that the software deviation correction did not define itself during battery backup/sleep. I was met with opposition, so I put together this trial. Clock on the left is my test clock. It uses an Atmega 328 with Archie software, a tuned crystal and no software correction. Clock on the right is an Archie clock, it of course uses an Atmega 328 with Archie software. The 10 Pf crystal load capacitors tuned the crystal to 32767.9 hertz; at a -.1 hertz deviation, I would expect to see a 1 second loss in about 3.5 days, or 2 seconds a week. To add software correction, I set to AUTODRIFT_CONSTANT 2100, defined in config.h file. I then synchronized the clocks to the exact second. I'm sure seconds were not exactly synchronized, but were visually. I let the clocks run together for a week and a half, continuing on synchronized, Then I unplugged the Archie clock for a week. When I plugged back transformer power, result in image is what I found, Archie clock 2 seconds behind. Does this mean software correction does not work during battery backup? Test clock did not gain two seconds, clocks ran synchronized together for a week and a half under normal transformer power. Two second loss is exactly what I would expect to find with a -.1 hertz crystal deviation and no software correction.

To clarify; this is not a shot on software correction, nor do I find software correction inferior. This is not a trick, I did not merely photograph two clocks together detuned 2 seconds. I took the time and effort, just shy of three weeks, to make this trial. This is completely separate from the issue I had with auto deviation correction. I stand by my statement that software correction is not defined during battery backup/ sleep. Unless you have done your own test to prove this wrong, simply telling me it works is not enough.

[jarchie]

Nice work, Russell! And thank you for taking the time to test my hardware design and firmware.

Were you comparing both clocks to an atomic/GPS-synchronized time standard? Even if you're comparing to a known-accurate commercial quartz clock, I still believe your results... I would just have added faith with a high accuracy time standard.

Were you using the software-based DS18B20 temperature compensation on my rev d board, but not on your design?

Were you running my rev d board in the case and your design in open air throughout the entire test?

After both clocks woke from sleep, did you notice if the rev d board ran fast for the first five to ten minutes?

Does this mean software correction does not work during battery backup?

I hope it does! Because if that's the problem, there would be an easy software fix.

But if the problem lies elsewhere, the workaround might not be so straightforward. For example, your tunable oscillator hardware might be less prone to frequency changes as a result of changes in voltage.

This is not a trick, I did not merely photograph two clocks together detuned 2 seconds.

I know you well enough to respect you, your work, and your integrity. The idea that you would intentionally fabricate data is simply absurd!

[russell 27]

To answer all these questions you would really need to do the test yourself. I understand how the software works and what it does, but I do not know how to define the statements in the code. If I did I would have experimented there. If the processor doesn't see the correction during sleep, I would think only the standard ASSR define value would be used for prescaler reset. I may not even be using the right terms here. Not all firmware's work the same. While I used JSGF's for some of my tests, that firmware would not reliably go into sleep mode for me. Sometimes battery draw was about 48 Ua, sometimes it was 350 Ua, sometimes clock would just lock up going from battery backup/sleep to standard power and I would have to reset it. With the tuned oscillator circuit and Archie software in my test clock, clock would always reliably go into sleep at about 18 Ua and wake up just as reliably. The Archie clock I tested was the board you sent, the temp chip is in there but that should not be defined, only manual time correction. I think the version of the software is the latest with the manual time correction in config h. I also wanted to add the toggle alarm switch to that, and should just be a matter of adding the alarm.c from the earlier code. I compiled it, just have not tried it yet. I also defined SUBDIGIT_MULTIPLEXING, instead of DIGIT_MULTIPLEXING in that hex because the test clock seemed to have a more even display using SUBDIGIT_MULTIPLEXING. I also soldered in 2 Murata variable capacitors to the ADAFRUIT clock, if that works out decent I'll start a new post about that.

[jarchie]

To answer all these questions you would really need to do the test yourself.

The important questions regarding the validity of the test are really just the three below. But am I understanding correctly that you don't remember the answer to any of these questions?

Were you comparing both clocks to an atomic/GPS-synchronized time standard? [...]

Were you using the software-based DS18B20 temperature compensation on my rev d board, but not on your design? [i.e., was the TEMPERATURE_SENSOR macro defined in config.h?]

Were you running my rev d board in the case and your design in open air throughout the entire test?

[russell 27]

TEST CLOCK.JPG (223.79 KiB) Viewed 744 times

This is my test clock, not the best Image, I had to compress file to enable upload. Basically the Archie hardware built this way, with a few additions. Board on right, where power comes in; I used these 220 Field Effect Transistors for AC inverter, they work a little better than the TO-92 type, no need for the 317. Top right board houses battery backup and tuned oscillator. Board below that, remote switching, this is where the toggle alarm really comes in handy. This clock wakes me everyday, Obviously I'm not married because this would never fly on top of my dresser. I can turn off the alarm or go to snooze, without getting out of bed, using my remote. I never encountered the clock race you spoke of that I can recall. I poked fun at the software correction, actually it's pretty impressive, I wanted to go this route. Eventually I hope to build this finished version, the hardest part left is finding a decent finished aesthetic, some designs I have seen aren't so visually appealing. I greatly appreciate all the help and insight you passed along. The only way you will be satisfied is if you do the test yourself. Again I did not use temperature compensation.

[russell 27]

You never mentioned a possible cause. During sleep, is it possible correction variable is not read or defined the same as during standard operating mode and Vcc.

[jarchie]

If the processor doesn't see the correction during sleep, I would think only the standard ASSR define value would be used for prescaler reset.

Software drift correction does not reset the prescaler. Instead, it sets the timer2 top value (OCR2A) at the start of each second to set the duration of the following second in 1/128 second increments. In the code below, "time.drift_adjust" is the drift correction value, and "time.drift_adjust_timer" is a countdown timer controlling the intervals between drift adjustments.

Code: Select all

    ATOMIC_BLOCK(ATOMIC_RESTORESTATE) {
        if(time.drift_adjust_timer) {
            // seconds until next drift correction
            --time.drift_adjust_timer;
        }
    }

    ATOMIC_BLOCK(ATOMIC_RESTORESTATE) {
        // set timer2 top value to adjust duration of next second
        if(!time.drift_adjust_timer && time.drift_adjust > 0) {
            // reset drift correction timer
            time.drift_adjust_timer = time.drift_adjust;

            // clock is slow: make next "second" faster
            OCR2A = 126;  // 127 values, including zero
        } else if(!time.drift_adjust_timer && time.drift_adjust < 0) {
            // reset drift correction timer
            time.drift_adjust_timer = -time.drift_adjust;

            // clock is fast: make next "second" longer
            OCR2A = 128;  // 129 values, including zero
        } else {
            // make next "second" of normal duration
            OCR2A = 127; // 128 values, including zero
        }
    }

The only way you will be satisfied is if you do the test yourself.

Perhaps, but I was hoping to learn a bit more about your testing method.

The difference observed was 2 seconds over a week and a half--on the order of 2 ppm or 0.07 Hz. I am convinced that your result is correct; there is certainly a difference in timekeeping during sleep. But I'm not yet convinced that the cause is the drift correction correction code. It seems like there are other possible explanations:

1. The frequency of oscillator circuits do change with voltage, and voltage falls when running on the battery backup. The difference in the oscillator circuits could mean that they respond differently to voltage changes.
2. Crystal oscillation slows as the temperature moves away from the crystal center (around 25 °C), so software temperature compensation works by speeding up the clock to compensate. But software temperature compensation doesn't really work during sleep.
3. When in the acrylic case and under external power, the clock temperature is significantly above room temperature, but when sleeping, the temperature falls to room temperature. There's a fair bit of crystal-to-crystal variation in the center temperature (plus or minus 5 °C), making it even more difficult to figure out exactly what's going on.

I never encountered the clock race you spoke of that I can recall.

Clock race? I'm not sure I understand what you mean here.

[russell 27]

I think we are in a different place here. As an example of my point, you asked this a few days back " After both clocks woke from sleep, did you notice if the rev d board ran fast for the first five to ten minutes?" I used the term clock race to describe this, and your response was " Clock race? I'm not sure I understand what you mean here." My point here is I don't know if I am conveying my thought on software correction being defined during battery/sleep. Bug is a coined word to describe a situation or problem that arises undesirably at certain times, while not being present at other times. I think that comes at a cost of the micro controller understanding hex and us written word, and some things get lost in translation. Ironically the case here sometimes. If you've ever used a PC you know about BUGS. I read the comment once, only at MICROSOFT is 60% good enough. That one's really close to my heart and funny bone. I don't know where else to go from here. From my calculations and test, 2 seconds a week would amount to a .1 hertz deviation, exactly what I found.