Gemma M0 - save persistant values across reboots?

[jharris1993]

Greetings!

I am working on a LED headband project that - for the time being - is using the software from the Gemma Color Touch Pendant located at https://learn.adafruit.com/gemma-color- ... e/software

I'm playing around with it, trying to get a feel for the way Neopixels work.

One feature I'm thinking would be cool would be to save certain runtime values in non-volatile memory so they wold be available on subsequent runs.

Example:
As written, the pendent always starts at the color specified by the initial value of "hue". It would be interesting if I could save either the last value for hue, or some random value for hue, so that every time the program starts, the starting color will be different. Likewise other parameters can be saved across reboots to cause other aspects to appear somewhat random at startup.

1. Is this possible?
2. If so, how would it be done in
(a) Arduino
(b) Circuit Python

Thanks!

Jim "JR"

[adafruit_support_mike]

You can use the FlashStorage library to save information in the microcontroller's program memory:

https://github.com/cmaglie/FlashStorage

[jharris1993]

Interesting - Thanks!

I'm going to wander over there tomorrow and take a look at that.

On that topic, what is your opinion about using flash memory to save the state of a prior run so that each run doesn't start in the same place? Are there any "gotcha's" you see that I don't?

The reason I'm interested in this is for display purposes - it will help the display look more "random" each time it's run. There might be only five or seven different display states, but if it starts in a different place each time, it should look like there's more than that.

Another thing I can see it being useful for is for saving calibration info - for a compass or accelerometer, etc. - so you don't have to re-calibrate it every time you power up.

Thanks!

Jim "JR"

Just thought of something. . .

In order to use this effectively, it will have to save the state at the point the program ended - or generate a random point in the display cycle when the program starts.

In order to save the state at program end, the program will have to know that it's ending. Does the Gemma M0 have any way of detecting a power-fail with enough time to save?

[kcl1s]

In order to save the state at program end, the program will have to know that it's ending. Does the Gemma M0 have any way of detecting a power-fail with enough time to save?

Not really. You could save the current place to flash each time it changes and it would always be available.

If you want random you can just use the random() and randomSeed() functions built into Arduino. Maybe you have not stumbled across the Arduino Reference page yet https://www.arduino.cc/reference/en/ It is a great resource.

Keith

[jharris1993]

Not really. You could save the current place to flash each time it changes and it would always be available.

Not sure that's a good idea here. . .

The program I'm working with here makes a number of display changes per second, continuously, at about 30 to 50 times a second. Assuming the flash has a 100,000 lifetime writes, it would take less than an hour to wear it out. Even at 10 updates a second it would wear out in less than 3 hours use.

Probably the best idea is to randomly generate a new starting point when the program starts, save it, and then on with the show! That would give something like a hundred-thousand use-cycles before it wore out.

Or, maybe I should skip this idea for the time being. . . . (grin!)

Jim "JR"

Thanks anyway - that's good information!

Just had a thought - (laughing) - the two granddaughters fighting over who's toy is whose - if I program each girl's name into the flash ROM, I can settle that argument easy by plugging into the serial console and have it print the owners name each time the toy starts!

[adafruit_support_mike]

On that topic, what is your opinion about using flash memory to save the state of a prior run so that each run doesn't start in the same place? Are there any "gotcha's" you see that I don't?

The major downside is eventual burnout of the Flash array, but that isn't an all-or-nothing thing. The Flash array is organized in 64-byte 'pages' which are erased and rewritten as a group. The SAMD21G has 512 pages of Flash, so wearing out one would leave 511 pages still working.

You can burn through the 10k lifespan using the Flash as a memory scratchpad, but at one update per hour, each page would last about a year.

In order to save the state at program end, the program will have to know that it's ending. Does the Gemma M0 have any way of detecting a power-fail with enough time to save?

Not by itself. You could add a small LiPo and watch the voltage on the 5V line, or simply monitor the LiPo voltage to see when it's getting low, but in general, the microcontroller learns about a power failure when it happens.

If you just need randomness, you can use the input from a floating analog pin to generate true random numbers. Here's a C sketch with my own random seed generator:

Code: Select all

void setup() {
    while ( ! Serial ) { delay( 1 ); }
    Serial.begin( 115200 );
}


uint32_t seed() {
    uint32_t result = 0x5EF348B6;  // any 32-bit value
    uint32_t b;

    for ( uint8_t i=40 ; i ; i-- ) {
        b = (( result >> 13 ) & 1 ) ^ (( result >> 8 ) & 1 ) ^ ( analogRead( A3 ) & 1 );
        result = ( b << 31 ) | ( result >> 1 );
    }
    return( result );
}

void loop() {
    Serial.println( seed(), HEX );
    delay( 100 );
}

The alphabet soup of numbers and logic operators is an efficient pseudo-random number generator known as a Linear Feedback Shift Register (LFSR). It marches through a sequence of 2^32 possible values in the same order every time, but the values have the statistical properties that we want from 'random' sequences.. each bit is high or low about 50% of the time, each pattern of N bits appears about 1/2^N of the time, and so on. It also has the computationally useful property that changing any bit in the 32-bit pattern will change about half of the bits in the next pattern.

Adding the truly random output from an analog pin makes the LFSR jump around in its sequence in unpredictable ways. My routine collects 40 bits of truly random input, which is enough to completely overpower the constant starting value.

[jharris1993]

If you just need randomness, you can use the input from a floating analog pin to generate true random numbers. Here's a C sketch with my own random seed generator:

Code: Select all

void setup() {
    while ( ! Serial ) { delay( 1 ); }
    Serial.begin( 115200 );
}


uint32_t seed() {
    uint32_t result = 0x5EF348B6;  // any 32-bit value
    uint32_t b;

    for ( uint8_t i=40 ; i ; i-- ) {
        b = (( result >> 13 ) & 1 ) ^ (( result >> 8 ) & 1 ) ^ ( analogRead( A3 ) & 1 );
        result = ( b << 31 ) | ( result >> 1 );
    }
    return( result );
}

void loop() {
    Serial.println( seed(), HEX );
    delay( 100 );
}

The alphabet soup of numbers and logic operators is an efficient pseudo-random number generator known as a Linear Feedback Shift Register (LFSR). It marches through a sequence of 2^32 possible values in the same order every time, but the values have the statistical properties that we want from 'random' sequences.. each bit is high or low about 50% of the time, each pattern of N bits appears about 1/2^N of the time, and so on. It also has the computationally useful property that changing any bit in the 32-bit pattern will change about half of the bits in the next pattern.

Adding the truly random output from an analog pin makes the LFSR jump around in its sequence in unpredictable ways. My routine collects 40 bits of truly random input, which is enough to completely overpower the constant starting value.

Wicked cool!

Thanks for both the code and the explanation of what it does for us mere mortals, (laughing!), who have to use it.

This little trick is handy enough to be a "library" function all by itself - which leads to another question (sorry):

Assume that, as I work with these projects, I accumulate a "candy bag" or "tool-kit" of these cute little helpers.
Assume that I want to either take them individually, (or as a group), and make them into one, (or more), #include-able external libraries so that I can re-use them without having to drop them in, in toto every time I want a random number (or whatever).

What do I have to do to create libraries of my own? The ones that I import into my Arduino environment are wonderfully complex, with .h, .c, .cpp, (and others that I don't remember), files scattered all over. At risk of sounding like a Java advertisement, it would be convenient to "write once, use everywhere".

Obviously in this particular use case, I could simply include it in-line and call it as needed, but, (as I have learned to my regret), cutting-and-pasting code is the fast train to mismatched curly braces, syntax errors, and other subtleties that make one feel like a noob idiot. (Not that I'm not a noob idiot anyway. . .)

Can I assume that variables declared inside the library function are specifically local to that library? (That is, unless you need a global variable, which would be declared in-line in the main routine, right?)

Corollary question: Do local variables defined in a function go on the stack, or in the heap? I read an interesting article you folks wrote that talks about heap fragmentation and using the stack to avoid that when possible. Maybe not a big deal here, but learning programming "best practices" right from the get-go is a good thing.

Again, thank you - and all your colleagues! - for all the noob help you've given me. I appreciate it more than you can possibly imagine!

Likewise, thank Lady Ada for both contributing AND allowing you folks the time away from more constructive pursuits to help noobs like me. I should buy you all pizza and bear sometime. Aside from million-dollar orders, is there a mechanism for that?

Jim "JR"

[adafruit_support_mike]

What do I have to do to create libraries of my own?

At minimum, you need a folder in sketchbook/libraries that contains a header file.

When the Arduino IDE launches, it collects a list of folders in sketchbook/libraries and looks inside them for header files. Then it uses that list of headers to generate compiler directives for the build process. The IDE won't see any new files you put into sketchbook/libraries until it reboots, which can be a source of confusion.

Some people have put their entire library in the header file, function definitions and all. That's syntactically legal, but is like the sound of a kitten climbing a blackboard to anyone who's used to writing libraries. By convention, the header contains your constants, #define'd macros, and function declarations. Inline functions can also live in the header if they're simple. The function definitions live in a separate .c or .cpp file.

As a side point, don't worry about memory consumption from unused library functions. The IDE's compiler settings include a fragmentation option that turns each function in a file into its own standalone block of executable instructions. When the linker comes through to build the final executable, it only copies the pieces the program actually uses.

At risk of sounding like a Java advertisement, it would be convenient to "write once, use everywhere".

Nah.. every language since the 1980s has leaned heavily on the idea of code reuse. Sometimes it's even possible. ;-)

The problem with code reuse is that, by definition, reused code doesn't do anything novel. The ideal case for code reuse is the one where you don't need to write any code at all because the program to do what you want already exists. We write new programs because we want something that doesn't already exist.

Obviously in this particular use case, I could simply include it in-line and call it as needed, but, (as I have learned to my regret), cutting-and-pasting code is the fast train to mismatched curly braces, syntax errors, and other subtleties that make one feel like a noob idiot. (Not that I'm not a noob idiot anyway. . .)

Cut-and-paste is high on the list of bad habits experienced programmers cultivate a loathing for. It's just too easy to miss a detail that will come back and bite you.

In terms of measurable time economy, it's almost always faster and easier to re-type the code by hand. That way you have to think about what it's doing.

Can I assume that variables declared inside the library function are specifically local to that library? (That is, unless you need a global variable, which would be declared in-line in the main routine, right?)

C and C++ use file-level scope by default. A variable you declare outside any function is visible to all functions defined in the same file, but invisible to functions in any other file.

If you want to declare a top-level variable that's visible in more than one file, you use the 'static' modifier. Any other file that declares a top-level static variable with the same type and name will have that name bound to the same storage location.

C++ both simplifies and complicates things by adding the concept of namespaces: named chunks of scope where names are visible. The identifier 'foo::varname' refers to the same entity in any file where it appears. You can also use namespace specifiers for function definitions, if it happens to be useful to split a block of interrelated code across several files. The variable 'foo::varname' will be visible to the function 'foo::func()' no matter which files they're in.

Do local variables defined in a function go on the stack, or in the heap? I read an interesting article you folks wrote that talks about heap fragmentation and using the stack to avoid that when possible. Maybe not a big deal here, but learning programming "best practices" right from the get-go is a good thing.

That's kind of squishy.

Function parameters and return values are allocated in the stack. Primitive types declared inside a function (int, float, boolean, etc, plus structures composed of primitive types) also tend to be allocated on the stack, but the compiler might come up with its own ideas. In C++, objects are allocated in the heap because the construction process involves a call to malloc(). The Arduino convenience class String is a prime source of heap fragmentation, not least because it's so easy to create one.

It's possible to write code that only uses variables declared on the stack, but that kind of code can't solve certain problems. There's a large category of problems that you can only solve by allowing information to exist independently of the function-call hierarchy. The price of the additional power is a certain amount of mess and the capacity to make a much bigger mess than you actually need.

If heap fragmentation becomes a problem, one good strategy is to pre-allocate pools of the objects you plan to create and throw away a lot, then write functions to manage passing them out and marking them free for reuse. You basically end up making a simple garbage collection system, but fortunately that isn't too hard.

Likewise, thank Lady Ada for both contributing AND allowing you folks the time away from more constructive pursuits to help noobs like me. I should buy you all pizza and bear sometime. Aside from million-dollar orders, is there a mechanism for that?

*laugh* If you happen to run into us/them at a Maker Faire somewhere, come up and say hi.

Ladyada has described Adafruit as "an education company that happens to have a gift shop at the end." What we're doing here is the company's primary function. Selling hardware keeps the lights on and the bills paid, and gives Ladyada a personal workbench to rival Thomas Edison's, but it's a support activity for the central idea of getting people excited about working with technology.

We also have a policy of sharing happy posts in the forums and friendly messages to [email protected] with everyone. There are nearly a hundred people making the machine called Adafruit go, and we're big on high-fiving everyone as much as possible.

[jharris1993]

Re: Creating libraries for Arduino.

I am familiar with what is in a library - a gibberish summary that defines the detailed gibberish in the associated .c/.cpp files. (:laughing:)

Seriously, I know that the .h files contain function prototypes, #ifdef statements, some #defines, and a bunch of other stuff. Since I've never even compiled a .dll, let alone created libraries for Arduino, I'm afraid I'll need a little bit more detail than you provided - as in how do I know what goes where?

Second library file question:
You said that C/C++ enforce "file level" scoping.

• Assume I have a project that contains one program file and two or three included library files.
• Assume that - since all of this has to do with LEDs of some type or kind - it is very likely that each library may have a variable named "LED" (or LED_TYPE). such that the same variable name exists in each of the libraries AND in my own program module as well. Ergo, we have four instances of the exact same variable name that could be a dozen different things. Under normal conditions, you cannot have multiple different instances of the same variable within the same scope.

Am I correct in assuming that variable scope is contained within the header/library file itself, so that variable name collisions cannot occur? This is what makes the most sense.

Thanks again!

Jim "JR"

[kcl1s]

I believe most library variables are local in nature for the reasons you give. Data is usually exchanged with library functions via parameters passed to and data returned.

This tutorial takes a simple sketch and show how to turns it into a library. https://www.arduino.cc/en/Hacking/LibraryTutorial It give a good explanation of the unusual syntax found in library code.

This Adafruit tutorial gives a good explanation of C++ classes which most libraries use. https://learn.adafruit.com/multi-taskin ... 1?view=all

These will not make you a library expert but it helps a lot for understanding how a library works.

Keith

[adafruit_support_mike]

Since I've never even compiled a .dll, let alone created libraries for Arduino, I'm afraid I'll need a little bit more detail than you provided - as in how do I know what goes where?

You don't compile anything. All you have to do is place the header and source file where the IDE can find it (inside sketchbook/libraries).

The firmware that goes on a microcontroller is all statically linked. When you tell the IDE to upload firmware, the scanner looks at your code to find the #include statements, finds the .h and .c/cpp files associated with them, looks for any files #included by those, and so on. When it's done, it has a list of files to run through the compiler. The build system feeds those files to the compiler one by one, and the compiler creates a separate object file for each source file. Within those object files, each function is created as a stand-alone item. After all the object files have been created, the linker creates a single object file that holds all the function-chunks your sketch actually uses.

Microcontrollers don't use dynamic linking at all. The point of dynamic linking is to have a single file that contains executable code used by several different programs, primarily to save space on the hard drive and in RAM. The runtime environment for each program has a system that lets it load DLLs into memory when it needs them, or use DLLs that have already been loaded into memory by some other program. Microcontrollers only have one active process, and all of the firmware has to live in the Flash program memory anyway, so there's no need for them.

Some software build environments keep pre-compiled object files for library code to save the time and effort of re-compiling the same files over and over again, even when the linker will build a statically-linked executable in the end. It saves time when you're building something big, like a word processing suite. The Arduino IDE doesn't even do that (mostly), because the firmware for a microcontroller is small. As fast as computers are these days, the 'wasted effort' of compiling everything from source every time is negligible.

Having said that, the hardware support code for some of the more recent 32-bit microcontrollers can span a few hundred files, so recent versions of the Arduino IDE have taken to saving a copy of the low-level object code, and using that when you tweak the main code and recompile. That's all handled automatically by the IDE though, and is just a temporary thing. I think the IDE starts from scratch every time you open a new project file.

As far as anyone using the Arduino IDE is concerned, 'installing a library' means putting the header and source files where the IDE can find them. Nothing more.

Assume I have a project that contains one program file and two or three included library files.

Got it.

Assume that - since all of this has to do with LEDs of some type or kind - it is very likely that each library may have a variable named "LED" (or LED_TYPE). such that the same variable name exists in each of the libraries AND in my own program module as well.

To add a little more detail, let's call the library source files standard_led.c, neopixel.c, and dotstar.c, and say that each of those files has a global variable named LED_COUNT.. three instances of the same variable name being used in different source files.

Any function defined in neopixel.c will see the variable LED_COUNT declared in that file. They won't see the variables named LED_COUNT in standard_led.c or dotstar.c at all. The same is true for the other two files.

The compiler only cares about variable names as a stepping-stone toward generating an object file. Any names we use get converted to 'symbol table entry N' while the compiler is parsing the file. The build system compiles files one at a time, and the compiler creates a new symbol table for every file it sees. What we see as 'the variable named LED_COUNT in source file neopixel.c' ends up being 'symbol table item N in object file neopixel.o'. The others end up as 'symbol table item M in object file standard_led.o' and 'symbol table item L in object file dotstar.o'.

That only applies to variables declared in the .c/cpp files though. If you have three headers.. standard_led.h, neopixel.h, and dotstar.h.. and declare LED_count in -those-, you'll get error messages.

Functionally, the directive #include "foo.h" means "replace this line with the contents of the file foo.h". The scanner does that before sending files to the compiler, so having the same variable name in multiple #include'd headers is identical to declaring the same variable multiple times in the same source file.

[jharris1993]

Thanks a lot for the excellent and detailed explanation.

When I said I hadn't even compiled a DLL, that was, (supposed to be), a figure of speech - I've never worked with the creation of libraries at all. Actually, that's not literally true. I've created modular library-like subroutines for assembly language programming. What I haven't done is create C-like libraries/header files.

First, I totally understand the idea of an "include" statement - it inserts the contents of a file literally into the code at such-and-so place. When I used to program 6502 assembly language applications many years ago, I had developed a "tool-box" of necessary routines for things like "display", "read/write to disk", "I/O", somewhat like the C standard libraries. This assembler was referred to as a "macro assembler" because it allowed the use of "include" type statements to drop macro-like routines into the code directly.

They existed on the disk that my development environment was on - and as I wrote code, if I needed disk I/O, I included the associated "module" in the code. Actually, I included them as callable subroutines in the code and called the routines as needed passing either literal strings, pointers to things, or pointers to variables that needed to be used. . .

==========================

In this case, though variables in the individual libraries are private to the libraries, (i.e. local in scope), the variables declared in the header are global.

Why? Let me see if I understand. . .

Variables that exist only within the body of the library function do not need to be visible outside it. In fact, it is desirable that they are not visible outside it since the scope of a variable should exist for both the shortest time possible and within the smallest block of code possible to avoid scoping collisions. <== huge source of both bugs and headaches.

The variables declared in the .h files are global to everything because any routine might need to use them - like printf() - any and every routine in the program may need that statement. Having it local to a specific function would make no sense - you'd have to include/create the "printf()" everywhere you needed it.

So, if I understand rightly, since the .h file ONLY creates/declares variables visible outside the included library, they have to be unique. Right?

I read an article on variable scoping that talked about a useful naming convention - naming local versions of variables with a leading underscore. (i.e. I might pass "Brightness" as a parameter to a library function, and use "_Brightness" to reference how and where I use it within the routine itself.) Though not mandatory, and possibly not even necessary due to scoping rules, it's a useful convention to help the programmer understand the intended scope of a variable - as in, "What's that doing out here?!" <== bug/compiler error waiting to happen.

Thanks!

Jim "JR"

[adafruit_support_mike]

You're on the right track, but there are a couple of details to tweak.

the scope of a variable should exist for both the shortest time possible and within the smallest block of code possible to avoid scoping collisions

This part is exactly correct.

Variables that exist only within the body of the library function do not need to be visible outside it.

This part is true, but is a special case of file-level scope.

Using these as an example:

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.h
 */

int foo_up();
int foo_dn();
int get_foo();

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.c
 */

#include "foo.h"

int foo = 0;

int foo_up () {
    foo++;
    return( foo );
}

int foo_dn () {
    foo--;
    return( foo );
}

int get_foo () {
    return( foo );
}

Code: Select all

/*  MAIN CODE
 *  filename: main.c
 */

#include "foo.h"

int foo = -1; // same name, but independent of the global variable in foo.c

void main () {
    for ( int i=10 ; i ; i-- ) {
        foo_up();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
    }
    foo_dn();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
}

The variable 'foo' in foo.c needs to be global because all three functions use it. It's visible everywhere in foo.c, but isn't visible anywhere in main.c

When you compile the code, the macro preprocessor does its substitutions and strips the comments. The compiler itself gets this as the file foo.c:

Code: Select all

int foo_up();
int foo_dn();
int get_foo();

int foo = 0;

int foo_up () {
    foo++;
    return( foo );
}

int foo_dn () {
    foo--;
    return( foo );
}

int get_foo () {
    return( foo );
}

And that's straightforward, well-formed C. The macro-expanded-comment-stripped version of foo.c tells the compiler what functions are coming, then tells the compiler what they do. The compiler creates an object file with chunks of executable code for all three functions. It also creates a memory location for the global variable 'foo', and when it generates machine code, it replaces the name 'foo' with the address of that memory location.

At that point, the programmer-defined variable named 'foo' no longer exists, at least with that name. It's been replaced by the location where the value is stored and a set of executable statements that operate on that memory location.

Then the build system moves on to the file main.c, runs it through the preprocessor, and feeds this to the compiler:

Code: Select all

int foo_up();
int foo_dn();
int get_foo();

int foo = -1;

void main () {
    for ( int i=10 ; i ; i-- ) {
        foo_up();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
    }
    foo_dn();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
}

Once again, the function declarations from the header tell the compiler the functions foo_up(), foo_dn(), and get_foo() exist before any code tries to call those functions. The compiler also sees the global variable in that file named 'foo', creates a memory location for it, and places 'get the value stored at this location' instructions in the two printf() statements that use the variable.

And again, the name 'foo' no longer exists in the object file. It was just a placeholder so the compiler could generate the appropriate 'use this location' instructions.

And of critical importance to this discussion, the memory location in the object file foo.o, used by the executable instructions for foo_up(), foo_dn(), and get_foo(), has no relation to the memory location in the object file main.o

In fact, once you take away the names, there's no reason to think the two memory locations would ever be associated with each other. The object file main.o would be exactly the same if the global variable was named 'bar' instead of 'foo'.

The problem of dealing with variables that have the same name but live in different scopes is a philosophically difficult one. Lots of people have spent a long time thinking hard about it, and agreed, "yep, that's a right bugger that is." The critical question is what I just suggested: "if we just word-replace the variable name with a different name in one scope, does the whole problem go away?" The formal name for changing variable names to eliminate conflicts is 'alpha-conversion', and language designers sweat over doing it properly.

After the .c files have been compiled into .o files, the build system hands things over to the linker.

The linker looks at main.o and says:

The object file main.o contains the main() function, so let's copy that chunk of executable instructions to the final output file. The main() function also uses a memory location in main.o, so we'll copy that over too.

Now then.. I see a space for a function call labeled foo_up() that doesn't have a corresponding chunk of executable code in main.o. I do have an executable chunk with the name foo_up() in the file foo.o, so let's copy that into the output file. Looks like the executable chunk for foo_up() uses a memory location in foo.o, so we'll copy that over too. Now that the executable chunk for foo_up() is in the output file, I have an address and can paste a 'jump to this location' instruction into the function call labeled foo_up() that came from main.o

Moving on, I see a space for a function call labeled get_foo() that doesn't have a corresponding chunk of executable code in main.o. I do have an executable chunk with the name get_foo() in the file foo.o, so let's copy that into the output file. Looks like the executable chunk for get_foo() uses a memory location in foo.o and.. let me check my notes.. yep, I've already copied that location over to the output file. I don't need to copy the memory location to the output file again, but I do need to make sure the executale chunks for foo_up() and get_foo() in the output file use the same memory location. Now that the executable chunk for get_foo() is in the output file, I have an address and can paste a 'jump to this location' instruction into the function call labeled get_foo() that came from main.o

and so on.

So.. moving on to headers..

If we change the foo library files like so:

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.h
 */

int foo_up();
int foo_dn();
int get_foo();

int foo = 0;

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.c
 */

#include "foo.h"

int foo_up () {
    foo++;
    return( foo );
}

int foo_dn () {
    foo--;
    return( foo );
}

int get_foo () {
    return( foo );
}

The compiler will see exactly the same input when the preprocessor has done its macro substitution and comment stripping:

Code: Select all

int foo_up();
int foo_dn();
int get_foo();

int foo = 0;

int foo_up () {
    foo++;
    return( foo );
}

int foo_dn () {
    foo--;
    return( foo );
}

int get_foo () {
    return( foo );
}

But this is what the compiler will get for mainc:

Code: Select all

int foo_up();
int foo_dn();
int get_foo();

int foo = 0;  //  <-- problem: 
int foo = -1; //  <-- two variables with the same name in the same scope

void main () {
    for ( int i=10 ; i ; i-- ) {
        foo_up();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
    }
    foo_dn();
        printf( "the library's foo=%d, while the main code's foo=%d\n", get_foo(), foo );
}

The key insight here is that the header file doesn't have any scope of its own. The code in the header gets pasted into the scope of any .c file that #includes the header.

The compiler never has to deal with the problem of 'variables with the same name in different files' because it doesn't have a concept of 'more than one file'. Each time it runs, it processes exactly one macro-expanded source file. The linker is the part of the build system that knows how to work with multiple files, but by the time it's ready to play, there are no longer any 'named variables' in the sense that programmers use them. There are only memory locations created by the compiler, executable instructions that use specific memory locations, and some basic rules for turning 'all references to this memory location in this object file' into 'references to a common memory location in the final output file'.

It's impossible for 'a memory location in object file main.o' to refer to the same thing as 'a memory location in object file foo.o', and the linker uses that information when it does alpha-conversion on memory locations while building the output file.

So, if I understand rightly, since the .h file ONLY creates/declares variables visible outside the included library, they have to be unique. Right?

Again, you're very close. The complete version is, "[...] unique within all files that #include that .h file."

You can have two .h files with the same variable name as long as no .c file #includes both of them. Actually doing it would make the people who have to maintain the code hunt you down with mayhem in their hearts, but it's functionally legit.

The _variable convention is one widely known trick for letting people know you're using a variable whose scope is restricted. It would be good manners to write foo.c like so:

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.c
 */

#include "foo.h"

int _foo = 0;

int foo_up () {
    _foo++;
    return( _foo );
}

int foo_dn () {
    _foo--;
    return( _foo );
}

int get_foo () {
    return( _foo );
}

If you had some compelling reason to move the counter variable into the header (it can happen), you'd want to use something like this:

Code: Select all

/*  FOO COUNTER LIBRARY 
 *  filename: foo.h
 */

int foo_up();
int foo_dn();
int get_foo();

int LIB_FOO_COUNTER = 0;

with the appropriate substitutions in the .c files. Only the the most evil of of trolls would use a variable name like that in code that didn't have anything to do with the foo library.

Though not mandatory, and possibly not even necessary due to scoping rules, it's a useful convention to help the programmer understand the intended scope of a variable - as in, "What's that doing out here?!" <== bug/compiler error waiting to happen.

It isn't mandatory for the compiler or linker, but it's mandatory for writing decent code.

The entry-level description of source files is, "these tell the compiler what kind of executable code to generate."

The experienced programmer's description of source files is, "these tell other humans (including myself six weeks from now) how I solved this problem.. and, incidentally, are written in a form that a compiler can turn into executable code."

Source code is, first and foremost, a document for humans to read. There are *much* more efficient ways to tell a compiler what executable code to generate, but we've put a lot of effort into abandoning them because they're they're harder for humans to read and understand.

[jharris1993]

Another, (hopefully related), question:

What happens to code that is never used? Does it get optimized out? Does it sit there and consume memory?

Example:

Code: Select all

testfunction();  //  The one and only call to testfunction

[intervening lines of code that are possibly unrelated to "testfunction"]

void testfunction()
{
/*  Routine "testfunction"
    Requires:  Nothing.
    Returns:  Nothing.
    Depends on:  Standard I/O library.
    Is a dependency of:  Nothing.
    Description:
    testfunction does something that is used mainly for testing and debugging but *is not needed* for normal operation.
*/

[lines of code that implement "testfunction"]

}

[more source code that does something else, etc. etc. etc. . . . .]

Assume that "testfunction" is a diagnostic/debugging blurb - kind-of like a "Kilroy Was Here!" marker in the code - that is only used for testing/diagnostics/debugging - but is not needed in the final production release.
Also assume that you compile/link/download the code with the original call to "testfunction" commented out.
Also assume that there are no other references to "terstfunction" anywhere else, so that block of code is, in essence, orphaned.

What happens when the code is compiled, linked, created into an executable binary, and downloaded. Is that block of code still there, though orphaned?

Jim "JR"

[adafruit_support_mike]

What happens to code that is never used? Does it get optimized out?

Yep.. at least mostly.

What happens when the code is compiled, linked, created into an executable binary, and downloaded. Is that block of code still there, though orphaned?

For the specific example you described, the compiler will catch the orphaned function and throw it away.

A compiler starts by reading the source file and creating a tree-shaped data structure that represents the code as it's written. That structure is called the 'parse tree'.

The parse tree is usually kind of a mess, so the compiler goes through and rearranges things like 'a+b/c' into more formal units like 'divide(add(a,b),c)'. The end result of that is another tree-shaped structure called the 'abstract syntax tree'. One step in creating an abstract syntax tree involves getting rid of statements that will never be executed.

The abstract syntax tree goes to the optimizer, which rearranges statements so they use the ALU as efficiently as possible. If an orphaned chunk of code isn't removed while building the abstract syntax tree, the optimizer will get rid of it.

The optimizer's output goes to the code generator, which turns each optimized expression into a set of machine instructions in the output file.

The syntax analyzer (which creates the abstract syntax tree) can only find orphaned code in a file that contains a main() function though. That's a known starting point, and it's easy to find every function called in the body of main(). The process of finding all the functions called by main(), looking through each of those and finding the functions they call, finding the functions those call, and so on is called 'finding the closure of main()'. In that context, 'closure' basically means 'any piece of code you can possibly reach after starting in a given place'.

You can calculate the closures for every function in a library, and can find 'the set of functions that don't appear in the closure of any other function', which are likely to be orphans. You can't actually declare them as orphans unless you know the closure of main(), though. In theory, you could tell the compiler that the Arduino functions setup() and loop() are also known starting points, but it isn't strictly necessary.

When the compiler sees a source file that doesn't contain the known starting point, it has no choice but to create object code for every function in the file. Under those conditions, the linker is the part of the build system that actually finds the closure of main(). Only items in the closure of main() get copied into the final executable, then uploaded to the microcontroller.

It's still technically possible to create orphaned code that will never execute, but still gets uploaded. You just have to make it rely on facts the compiler and optimizer don't know, like this:

Code: Select all

#define BUTTON  12

void setup () {
    Serial.begin( 9600 );
    
    pinMode( BUTTON, OUTPUT );
    digitalWrite( BUTTON, HIGH );
}

void loop () {
    if ( digitalRead( BUTTON ) ) {
        do_one_thing();
    } else {
        do_another();
    }
}

The function do_another() will never be executed (unless someting goes badly wrong with the pin), but the compiler doesn't have the information necessary to make that decision. It will compile and upload object code for do_another() because the rules of C say that function is in the closure of loop().