Dynamically Import All Python Files

bentraje

Hi,

I'm trying to import dynamically import python files in a directory.

For example if the directory is:

utility
------init.py
------file1.py
------file2.py
------file3.py

The code to import them would be

from utility import file1
from utility import file2
from utility import file3

I want this line of code to be executed dynamically. So if I add a file4.py, I don't want to write from utility import import file4. That's why I want them to import dynamically.

However, my code below gives an error of ImportError: cannot import name 'filename' where it treats the filename as an actual string and not as a variable.

Is there a way around this?

Here is the current code:

import utility

for f in files:
    if f.endswith(".py") and not f.startswith('__init__'):
        filename = f.split('.')  
        from utility import filename[0]

P.S. There are some solutions in the stack overflow like this one:
https://stackoverflow.com/questions/1057431/how-to-load-all-modules-in-a-folder

But again, it requires to be executed every time I add a python file in the directory (i.e. no dynamic)

zipit

Hi,

for clarification: I think you do not actually mean importing modules dynamically (which would imply importing them on a "as needed basis" depending on the branching of your code. From what I do understand you want to implicitly import all modules located in a package (folder). I assume you are aware that python has the wildcard syntax for that.

from package import a, b, c, d
# is equivalent to (assuming there is only a, b, c, d)
from package import *

To make this somewhat useful you will have to define in your __init__.py what you want to be imported (as described in your linked post).

However to import from packages Python has to know where to look for them. If you have your folder utility in your c4d plugin folder, this won't work out of the box. There are multiple ways to achieve this and people will probably boo me for telling you this, but the easiest way is to just add to (or pollute as some people would say) the sys path.

_path = os.path.dirname(__file__)

if _path not in sys.path:
    sys.path.insert(0, _path)

#assuming foo is a folder/package in the dir of __file__
from foo import bar

One of the drawbacks is that you have to do this:

_path = os.path.dirname(__file__)

if _path not in sys.path:
    sys.path.insert(0, _path)

from foo import bar

if SOME_SORT_OF_DEBUG_CONDITION_IS_TRUE:
    reload(bar)

If you want to reload that module on a running Python VM. For example when you hit "reload python plugins" in c4d. Otherwise you have to restart c4d for each code change to apply.

Cheers
zipit

bentraje

@zipit
Thanks for the response. Yes, I ended up "polluting" the sys.path. I don't mind though. It works as expected.
Here is the working code for me without using the __init__ file and it imports dynamically.

import c4d
import os
import sys

rigDir = "C:/Users/Luke/Dropbox/Scripts/c4d"
if not rigDir in sys.path:
    sys.path.append(rigDir)

dir_path = "C:/Users/Luke/Dropbox/Scripts/c4d/utility"
files = os.listdir(dir_path)

import utility
from utility import hello_world

for f in files:
    if f.endswith(".py") and not f.startswith('__init__'):
        filename = f.split('.')
        #from utility import filename[0]
        exec('from utility import %s' % filename[0])
        exec('reload(%s)'% filename[0])

The only caveat so far is to convert all scripts to

if __name__=='__main__':
    main()

Because at this point, all scripts get executed when imported hahaha.
Should have listed to everyone telling me to use the if __name__=='__main__' from the start.

m_adam

Hi @bentraje,

While @zipit has almost said everything I would like to add few things:

if __name__=='__main__' is really needed since its ensure that your code is not called from anywhere, performance-wise its also very important, even if it's not a big issue in your plugin logic if your code is executed each time you import stuff, think to the user and his performance.
It's recommended to do local import and not global import (as you do) since if you add a module named xyz to the sys.path from your plugin A and from plugin B you try to import another module named xyz, it will load the one from plugin A (the first one register). And the other plugin will not work. For more information read Best Practise for import and especially the great localimport module from @NiklasR that is mentioned in the Best Practise for Import topic

Cheers,
Maxime.

bentraje

@m_adam

Thanks for the additional insight and the related links. Appreciate it a lot.