0

Main.Py character limitation and importing functions
Moderators: adafruit_support_bill, adafruit

Please be positive and constructive with your questions and comments.

Main.Py character limitation and importing functions

by pixelicous on Tue Jun 12, 2018 1:40 pm

I have a Gemma M0 which has nothing on it besides neopixel library and init file, the other file is main.py.
A neopixel ring/jewel are connected and some functions that play different light sequences.

I added another function and tried saving, at that moment the gemma fails to run the code.
If i remove that function and save, the code run just fine.

So i took a different piece of working function and replaced between the two, just to see what happens, and the code runs after saving successfully.
After that i made a lot of checks and im sure my code is fine, but it just seems like after a certain amount of characters in the file the gemma just refuses to run the code.

So i tried using extra files, but this doesnt seem to work..
I tried "from secondfile.py import *" and i also tried specific function name. I also tried "import secondfile.py" and then use "secondfile.funcname" and i also tried using the __INIT__.py file and putting everything there..

So my questions are whether there is such limitation and how do i import functions from extra files :(

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by tannewt2 on Tue Jun 12, 2018 1:58 pm

There is definitely a limit to how much code can be loaded at once. Have you confirmed in the serial output that you are getting a MemoryError? The line it points to will be one with import on it if that's the case.

MemoryErrors while importing can be reduce by converting the Python to bytecode before loading using mpy-cross. Check out this page on Learn for more info and suggestions: https://learn.adafruit.com/circuitpytho ... emoryerror

tannewt2
 
Posts: 764
Joined: Thu Oct 06, 2016 8:48 pm

Re: Main.Py character limitation and importing functions

by pixelicous on Wed Jun 13, 2018 4:49 am

tannewt2 wrote:There is definitely a limit to how much code can be loaded at once. Have you confirmed in the serial output that you are getting a MemoryError? The line it points to will be one with import on it if that's the case.

MemoryErrors while importing can be reduce by converting the Python to bytecode before loading using mpy-cross. Check out this page on Learn for more info and suggestions: https://learn.adafruit.com/circuitpytho ... emoryerror


Hey..
  • No, i haven't connected it to serial to see what would be the output, can usb also be serial?
    I am pretty certain that is what i will see but i will check anyways, I am working with visual studio code and not arduino IDE, i'll try chekcing the serial with the arduino IDE
  • What detail on gemma m0 should i look at to understand how much code can i put in memory? RAM? Flash? I dont think im surpassing any of those..
  • I thought the fact that i have a python file on a disk like emulation means i can put how much code i want and gemma will be able to handle it. so basically all that space is just for nothing, the actual code space is the RAM??
  • Also, i asked about importing extra python files, is that possible? if so, how do i do it

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by tannewt2 on Wed Jun 13, 2018 2:20 pm

pixelicous wrote:Hey..
  • No, i haven't connected it to serial to see what would be the output, can usb also be serial?
    I am pretty certain that is what i will see but i will check anyways, I am working with visual studio code and not arduino IDE, i'll try chekcing the serial with the arduino IDE


There are instructions here for connecting to the serial output: https://learn.adafruit.com/welcome-to-c ... al-console

  • What detail on gemma m0 should i look at to understand how much code can i put in memory? RAM? Flash? I dont think im surpassing any of those..


RAM is limited to 20kB and the file system is 60kB or so. The RAM restriction will lead to a MemoryError in the serial output. The flash limit will be a copy error in your OS as you copy files over.

  • I thought the fact that i have a python file on a disk like emulation means i can put how much code i want and gemma will be able to handle it. so basically all that space is just for nothing, the actual code space is the RAM??


The text Python code on the disk is parsed into bytecode which is stored in RAM. As its parsed the memory tends to peak above the amount needed for the bytecode. Thats why mpy-cross can help. It reduces the memory needed during an import.

  • Also, i asked about importing extra python files, is that possible? if so, how do i do it


Yup! It works just like normal Python. You `import <filename>` where filename doesn't have the .py extension when its in the root of the disk or in `lib` folder. This does require a bit more memory than a single file because it has to store the module name too.

tannewt2
 
Posts: 764
Joined: Thu Oct 06, 2016 8:48 pm

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 8:06 am

Thanks so much for the elaborated reply, sorry it took me time to response, some personal matter that left me not being able to work on it.

I tried what you wrote, i converted to a mpy file but now the code doesnt execute, i just added "import demos" and put the demos.mpy in both root and lib, i tried ejecting the gemma after to make sure it can read the file (saw it in the guide).. i am running a neopixel jewel/ring from it, so i rely on the lights i see to get that its running or not.

I used vagrant to download the virtual machine, compile using make and then running the mpy creator

the only problem i had initially is when running make i got the error: "msgfmt: command not found"
fixed it with "apt-get install gettext"

"make" seemed to run successfully.

any idea what can make my code freeze?

I just tried using the most simple of python files:
Code: Select all | TOGGLE FULL SIZE
def starPowerDemo(wait,colorOne,colorTwo):
    i = 5


Used "import demos" and hardly put anything in the main python file, still getting stuck.. :|

Maybe the firmware i used for making the mpy file is older than the one i have on the gemma? i think i updated the gemma and downloaded upgraded neopixel.mpy file

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by aaronaverill on Fri Aug 10, 2018 10:25 am

Install Mu and open the REPL tab. It will tell you exactly the error you are experiencing. You're trying to diagnose your problems without any tools. You're basically just guessing what a problem is. That is going to be a very frustrating experience long term if you plan to work with microcontrollers.

https://learn.adafruit.com/welcome-to-c ... -mu-editor

You might also post your code here, in case there is some obvious error. It's strange if you are only running a jewel ring you would run out of memory. What the heck code are you doing, calculating your taxes too? XD

I suspect you have a problem with compatibility of your circuitpython version and the .mpy you created. If you pulled mpy-cross code from GitHub you're likely using CP 3. But it's very likely your gemma was installed with CP2.X. So that won't work. You can look at the bootlog file to see the exact version of CP you have on the gemma.

You can reflash your gemma to CP3, and use the mpy-cross.exe tools from GitHub. All the binaries are here:

https://github.com/adafruit/circuitpyth ... /tag/3.0.0

Be sure to back up all your code, reflashing the gemma will wipe your entire USB drive.

And of course, once you have Mu up and running the REPL tab will tell you exactly what the error is.

If you go further there are optimizations you can make to minimize the mpy file more. I suggest looking at the -O and -s options. And if needed you can load your code into the flash memory to get it out of RAM by freezing modules in a custom build of CP. It's a bit tricky, but there are guides for all this here and even a newb like me eventually figured it out.

aaronaverill
 
Posts: 66
Joined: Sun Jun 03, 2018 10:45 am

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 12:09 pm

Hey mate,

Thanks for the response, i will install MU and check it out, i'm working with visual studio code as my ide..
I am using mpy-cross that was installed with the vagrant box i downloaded, it had all the tools on it..

I also believe the libraries might use incompatible versions, thats my initial guess.

I did upgrade the gemma from what i remember, not sure if it was the trinket or gemma

I will follow your instructions and post back.

Regarding the code, i connected a neopixel ring + jewel and i have tons of functions each running a different type of "demo" light mode..
I am sure that it is fine because if i just strip out some code lines it starts running (without conversion to mpy)

If i do convert to mpy, without "import demos" the code executes, with "import demos" it gets stuck
Last edited by pixelicous on Fri Aug 10, 2018 12:57 pm, edited 1 time in total.

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 12:27 pm

Ok upgraded to 3.0, updated neopixel.mpy to 3.0 as well.
Didn't quite check what i was on before, but updated anyways.

I did install mu-editor but it told me that it cannot find an attached circuitpython device and it will load from documents folder.
Clicking on serial doesnt show anything..

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 1:36 pm

Ok, so i tried importing my demos.mpy file with "import demos"

Then i just called a function that is saved in that demos.mpy file but i get an error
File "/vagrant/demos.py", line 2, in flowerDemo
NameError: name 'pixels' is not defined


Not sure why it added the prefix vagrant or why the .py file
Pixels is basically the following line
Code: Select all | TOGGLE FULL SIZE
pixels = neopixel.NeoPixel(pixpin, num_pixels, brightness=0.1, auto_write=False)


Another question i have is.. how come the functions in the mpy cant find an object that i declared from outside
When those functions were in the same python file i didnt have this error on "pixels"
i tried importing the demos library file after, still no go..

update: i copied the demos.py file to the mpy-cross folder and ran the command so i dont get that /vagrant/ prefix, now i get ./ . but it doesnt effect the issue..
i still cannot find the object i created on the main file.. are functions in mpy file handled a bit differently?

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by aaronaverill on Fri Aug 10, 2018 2:07 pm

pixelicous wrote:Ok, so i tried importing my demos.mpy file with "import demos"

Then i just called a function that is saved in that demos.mpy file but i get an error
File "/vagrant/demos.py", line 2, in flowerDemo
NameError: name 'pixels' is not defined



So it sounds like the configuration is not an issue, and it's a problem with your code. Good news.

It's probably easier to assist if you include all your code, rather than snippets, and explain the file layout, etc.

pixelicous wrote:Not sure why it added the prefix vagrant or why the .py file

mpy-cross includes the file name for debugging purposes. It makes some assumptions based on the file you pass in and where it's located with respect to your current path. You can specify the debug module name with -s, eg:

-s demo.py

pixelicous wrote:Pixels is basically the following line
Code: Select all | TOGGLE FULL SIZE
pixels = neopixel.NeoPixel(pixpin, num_pixels, brightness=0.1, auto_write=False)


Another question i have is.. how come the functions in the mpy cant find an object that i declared from outside
When those functions were in the same python file i didnt have this error on "pixels"
i tried importing the demos library file after, still no go..

You may want to read a bit on the python import statement, but it sounds like you have a name scope issue. Googling stack exchange is a great source for learning python.

You're better off thinking of an imported module as a library, which should contain methods or classes that you can use from the code importing it. So if you want to access the "pixels" object inside an imported module, you'll want to do something like this in the demo.py file:


Code: Select all | TOGGLE FULL SIZE
def doit(pixels):
    pixels.dosomething...

inside your imported module, and then from your main, call it like...

Code: Select all | TOGGLE FULL SIZE

import demo

pixels = # create some pixels
doit(pixels) # do some stuff with pixels using the demo library

aaronaverill
 
Posts: 66
Joined: Sun Jun 03, 2018 10:45 am

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 3:02 pm

Hey aaron,

Yeah thats a module not a library, i thought about it after posting.

Anyhow, i know scoping and i understood my issue was with it.. However what is weird is that the exact same function with its definition block used inside the .py file didnt have a scoping problem..

I will share code as soon as im near my laptop again

Oh and i know stack exchange but initially i thought its an issue with frozen modules so i figured this would be a much friendlier place :)

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 3:35 pm

According to what i'm reading, I am setting a global variable by putting it in the main file.
http://python-textbok.readthedocs.io/en ... Scope.html

I tried "global pixels" as a first like inside the function, didn't help it..
Below you can find the files (I removed the non relevant lines and functions)

Main.py (scroll down there is more code)
Code: Select all | TOGGLE FULL SIZE
### NeoPixel Pendant/Trinket by Pixel 2018
 
import time
import board
import neopixel
import demos

try:
    import urandom as random  # for v1.0 API support
except ImportError:
    import random

#strip setup
pixpin = board.D0
num_pixels = 19
pixels = neopixel.NeoPixel(pixpin, num_pixels, brightness=0.1, auto_write=False)

#Demos to run
flower = 1

RED = (255, 0, 0)
YELLOW = (255, 150, 0)
ORANGE = (255, 40, 0)
GREEN = (0, 255, 0)
TEAL = (0, 255, 120)
CYAN = (0, 255, 255)
BLUE = (0, 0, 255)
PURPLE = (180, 0, 255)
MAGENTA = (255, 0, 20)
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)

colorsList = [RED,YELLOW,ORANGE,GREEN,TEAL,CYAN,BLUE,PURPLE,MAGENTA,WHITE]


while True:
    if flower:
        for y in range(2):
            for i in range(len(colorsList)):
                flowerDemo(0.2,colorsList[i],shiftList(colorsList,3)[i],shiftList(colorsList,1)[i])


demos.mpy
Code: Select all | TOGGLE FULL SIZE
def flowerDemo(wait,colorOne,colorTwo,colorThree):   
    pixels[0] = colorThree
    for i in range(1,7):
        pixels[i] = colorOne 
    for i in range(7,19):
        pixels[i] = colorTwo
    pixels.write()
    time.sleep(wait)


If i put that demos.mpy code in main.py, it works, without passing pixels to the function.. that's why i don't get it..
Last edited by pixelicous on Fri Aug 10, 2018 5:33 pm, edited 1 time in total.

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by aaronaverill on Fri Aug 10, 2018 5:33 pm

Yeah you have a scoping problem.

When flowerDemo is in main.py, it is accessing the pixels object you defined in the main.py global scope.

When flowerDemo is in demos.mpy, it is trying to access a pixels object defined in demos.mpy global scope. But you never defined one.

Globals are module scoped. Here is a longer explanation.

https://stackoverflow.com/a/15959638/1516312

You need to pass in pixels as your first parameter. Then you should be good to go.

aaronaverill
 
Posts: 66
Joined: Sun Jun 03, 2018 10:45 am

Re: Main.Py character limitation and importing functions

by pixelicous on Fri Aug 10, 2018 5:48 pm

Once i passed pixels into the function i needed to also pass the time object
So i figure i should just better off importing again there and setting some of the variables such as pixels

I got to another stack exchange post describing similar things but a bit differently, the post you found helped fill in the picture

thanks again!

pixelicous
 
Posts: 27
Joined: Sun May 06, 2018 1:16 pm

Re: Main.Py character limitation and importing functions

by aaronaverill on Fri Aug 10, 2018 6:57 pm

Another way to think of this is there are no globals in python, they are actually module variables.

So you define a main.pixels, and then your demos.mpy tries to look for demos.pixels, which of course doesn't exist. And if it did, it would be a different one anyway.

time is a module, so you should definitely be importing it in your demos.mpy - when you call "time.sleep()" you're not calling the sleep() method on a time object, you're calling the sleep method in the time module name scope. If you pass it in - well you're just passing in a module (which is like an object) - it's a bit of an odd approach.

if for example, you did:

Code: Select all | TOGGLE FULL SIZE
import sleep from time

you could then just call sleep() without any namespace prefix. You sometimes see examples like this. And that's the reason you don't see the import module name prefix.

If you're accustomed to object oriented programming you may want to investigate that to make your code more clear. It's fairly simple in python. The syntax is a bit unconventional but it's all there, including inheritance.

aaronaverill
 
Posts: 66
Joined: Sun Jun 03, 2018 10:45 am

Please be positive and constructive with your questions and comments.