Develop Reference

I can not figure out why I can not start this simlple python script

My directory looks like this
When I start directly with PyCharm it works.
But when I try to start the script with a commandline I get this error messsage
> python .\PossibilitiesPlotter.py
Traceback (most recent call last):
File "C:\Users\username\PycharmProjects\SwapMatrixPlotter\possibilitiesplotter\PossibilitiesPlotter.py", line 7, in <module>
from plotterresources.PlotterProps import PlotterProps
ModuleNotFoundError: No module named 'plotterresources'
This is how the import looks from my main class PossibilitesPlotter.py
import sys
sys.path.append("plotterresources/PlotterProps.py")
from csv import reader
from pathlib import Path
from plotterresources.PlotterProps import PlotterProps
from possibilitiesplotter.PossibilitiesGraph import PossibilitiesGraph
from possibilitiesplotter.PossibilitiesModel import PossibilitiesModel
class PossibilitiesPlotter:

As a workaround, add the following line to PossibilitesPlotter.py:
sys.path.append("../plotterresources/PlotterProps.py")
This will add the directory one level above the commandline pwd to the PATH variable. So this is always relative to the location of the calling script/shell.
Thus in general:
NEVER append to the PATH/PYTHONPATH variable from within modules. Instead restructure your module. For more details, take a look at the documentation on Packaging Python Projects

ModuleNotFoundError: No module named 'sharedFunctions'

I have a Python project in which I have the following folder structure:
> root
> download_module
> __init__.py
> downloadProcess.py
> sharedFunctions.py
> someHelper.py
> useSharedFunction.py
The download_module/__init__.py has the following code:
from .sharedFunctions import stringArgumentToDate
from .downloadProcess import downloadProcessMethod
The sharedFunctions.py file contains the following function:
def stringArgumentToDate(arg):
dateformat = "%m/%d/%Y"
date = None
if arg.isnumeric():
date = datetime.fromtimestamp(int(arg))
if date == None:
date = datetime.strptime(arg, dateformat)
return date
Then on the useSharedFunction.py I try to import the shared function and use it like this.
from download_module import stringArgumentToDate
from download_module import downloadProcessMethod
def main():
arg = '03/14/2022'
dateArg = stringArgumentToDate(arg)
if __name__ == '__main__':
main()
When I try to run this by using python3 useSharedFunction.py I got the following error:
Traceback (most recent call last):
File "useSharedFunction.py", line 4, in <module>
from download_module import stringArgumentToDate
File "/Users/jacobo/Documents/project/download_module/__init__.py", line 2, in <module>
from .download_module import downloadAndProcessMethod
File "/Users/jacobo/Documents/project/download_module/downloadProcess.py", line 10, in <module>
from sharedFunctions import stringArgumentToDate, otherFunction
ModuleNotFoundError: No module named 'sharedFunctions'
I do believe the error is in downloadProcess since at the beggining of the file we got this import:
from sharedFunctions import stringArgumentToDate, otherFunction
from someHelper import Helper
Which refers to sibling files.
However I'm unsure what will be a proper fix to allow to run the downloadProcess.py main independently but also, being able to call it one of its method from a root or any other file out of the module.

Consider this structure:
┬ module
| ├ __init__.py
| ├ importing_submodule.py
| └ some_submodule.py
├ __main__.py
├ some_submodule.py
└ module_in_parent_dir.py
with content:
__main__.py
import module
/module/__init__.py
from . import importing_submodule
/module/importing_submodule.py
from some_submodule import SomeClass
/module/some_submodule.py
print("you imported from module")
class SomeClass:
pass
/some_submodule.py
print("you imported from root")
class SomeClass:
pass
/module_in_parent_dir.py
class SomeOtherClass:
pass
How sibling import works
(skip this section if you know already)
Now lets run __main__.py and it will say "you imported from root".
But if we change code a bit..
/module/importing_submodule.py
from module.some_submodule import SomeClass
It now says "You imported from module" as we wanted, probably with scary red line in IDE saying "Unresolved reference" if you didn't config working directory in IDE.
How this happen is simple: script root(Current working directory) is decided by main script(first script that's running), and python uses namespaces.
Python's import system uses 2 import method, and for convenience let's call it absolute import and relative import.
Absolute import: Import from dir listed in sys.path and current working directory
Relative import: Import relative to the very script that called import
And what decide the behavior is whether we use . at start of module name or not.
Since we imported by from some_submodule without preceeding dot, python take it as 'Absolute import'(the term we decided earlier).
And then when we also specified module name like from module.some_submodule python looks for module in path list or in current working directory.
Of course, this is never a good idea; script root can change via calls like os.chdir() then submodules inside module folder may get lost.
Therefore, the best practices for sibling import is using relative import inside module folder.
/module/importing_submodule.py
from .some_submodule import SomeClass
Making script that work in both way
To make submodule import it's siblings when running as main script, yet still work as submodule when imported by other script, then use try - except and look for ImportError.
For importing_submodule.py as an example:
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
Importing modules from parent directory is a bit more tricky.
Since submodule is now main script, relative import to parent level directory doesn't work.
So we need to add the parent directory to sys.path, when the script is running as main script.
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
# now since we don't have parent package, we just append the path.
from sys import path
import pathlib
path.append(pathlib.Path(__file__).parent.parent.as_posix())
print("Importing module_in_parent_dir from sys.path")
else:
print("Importing module_in_parent_dir from working directory")
# Now either case we have parent directory of `module_in_parent_dir`
# in working dir or path, we can import it
# might need to suppress false IDE warning this case.
# noinspection PyUnresolvedReferences
from module_in_parent_dir import SomeOtherClass
Output:
"C:\Program Files\Python310\python.exe" .../module/importing_module.py
you imported from module
Importing module_in_parent_dir from sys.path
Process finished with exit code 0
"C:\Program Files\Python310\python.exe" .../__main__.py
you imported from module
Importing module_in_parent_dir from working directory
Process finished with exit code 0

A strange "No module named 'XX' " problem

I know how to import a package or module, but I meet a quite strange problem.
If I run swmm5_extend_function/example.py, everything is fine. However, when I run example.py, errors occur:
Traceback (most recent call last):
File "example.py", line 2, in <module>
from swmm5_extend_function.Swmm5Extend import SWMM5ReadInp
File "C:\project\swmm5_extend_function\Swmm5Extend.py", line 1, in <module>
import swig.SWMM5ReadInpFile as swmm
ModuleNotFoundError: No module named 'swig'
Here is my project structure:
project/
-- example.py
-- ......
-- swmm5_extend_function/
-- __init__.py
-- example.py
-- Swmm5Extend.py
-- swig/
-- __init__.py
-- SWMM5ReadInpFile.py
-- ....
Here is code of each .py file:
swmm5_extend_function/Swmm5Extend.py
import swig.SWMM5ReadInpFile as swmm
class SWMM5ReadInp(object):
pass
swmm5_extend_function/example.py
from Swmm5Extend import SWMM5ReadInp
example.py
from swmm5_extend_function.Swmm5Extend import SWMM5ReadInp
I want to know why this strange error happens.

For a better explanation, I've created the following folder structure:
test/
-- __init__.py
-- greeter.py # Greets in German
-- subfolder/
-- __init__.py
-- greeter.py # Greets in Italian
-- test.py
-- deepfolder/
-- __init__.py
-- greeter.py # Greets in France
As you may notice, we have 3 files with the same name, each one greets in a different language using a function with the same name. The only function in a greeter.py file is:
def says():
print("Hello World!")
IMPORT FROM THE SAME FOLDER
If from test.py file we import greeter and run the says function, we'll have:
import greeter as greeter
greeter.says()
Output:
Buongiorno Mondo! # Italian output
IMPORT FROM A SUBFOLDER
But what if we want to import from a subfolder?
To import from a subfolder (i.e., deepfolder/), we simply add an empty __init__.py file into the folder, then we can specify the path in the import:
import deepfolder.greeter as greeter
greeter.says()
Output:
Bonjour le Monde! # France output
IMPORT FROM A PARENT FOLDER
At last, you may want to import from a parent folder.
You should try to have your main running file at the top of the folder tree, but things happens and you find yourself looking to import a module from a parent folder.
For doing this, you need to add the parent folder to the sys.path:
import sys
sys.path.append("/path/to/dir")
from test import greeter as greeter
greeter.says()
Output:
Guten Morgen Welt! # German output
Importing scripts and modules isn't really the most pythonic way to solve things, you may want to have a look on Python's documentation about packages.
TL;DR
In your project/example.py use
import swmm5_extend_function.swig.SWMM5ReadInpFile as swmm
instead of
import swig.SWMM5ReadInpFile as swmm

Easily importing all third-party packages and scripts as a common package?

I want to add an external package to my program which should include all recipes and third-party packages it uses. I don't want to force anyone (including myself) to install those packages, not to mention the version incopatibilities. I just
want to put them into their own subfolders and use them. They,
of course, come from various sources.
The folder structure should look like this:
| main.py
|---[external]
|---[networkx]
| | ...
|---[xlrd]
| | ...
| __init__.py
| recipe1.py
| recipe2.py
I want to reach the packages and recipes in the following ways in my program:
import external
import external.xlrd
from external import networkx as nx
import external.recipe1
from external import recipe2 as magic
from external import *
Any package however may contain absolute imports which can result ImportError assuming an external has an empty __init__.py:
>>> import external.networkx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".\external\networkx\__init__.py", line 46, in <module>
from networkx import release
ImportError: No module named 'networkx'
Is there an easy way to make such an external package?

I made a function which does this. All you need to do is to put this line to the __init__.py of the external folder:
__all__ = import_all(__path__)
This function can use generally to import all submodules, even if they are nested, just put this to all __init__.py involved.
If you know a cleaner solution to the question, please share it with us! I won't accept my own answer.
from os import listdir
from os.path import abspath, basename, exists, isdir, join, \
relpath, sep, splitext
from sys import path
def import_all(path__here):
"""Imports all subpackages and submodules, excluding .py
files starting with an underscore. The directories have to
have an __init__.py to get imported.
Add this line to the __init__.py:
__all__ = import_all(__path__)"""
result, packagepath = [], relpath(path__here[0])
for e in listdir(packagepath):
mod_path = join(packagepath, e)
mod_name = splitext(basename(e))[0]
file_to_import = (e.endswith(".py")
and not mod_name.startswith("_"))
dir_to_import = (isdir(mod_path)
and exists(join(mod_path, "__init__.py")))
if not file_to_import and not dir_to_import:
continue
im_str = ".".join(mod_path.split(sep)[:-1] + [mod_name])
try:
__import__(im_str)
except ImportError as err:
# In case of a subpackage countains absolute imports
# assuming it is in the root, we put it into the
# system path and try again.
if abspath(packagepath) not in path:
path.insert(0, abspath(packagepath))
__import__(im_str)
else:
raise err from None
result.append(mod_name)
return result

how to import the blog.py(i import the 'blog' folder)

my dir location,i am in a.py:
my_Project
|----blog
|-----__init__.py
|-----a.py
|-----blog.py
when i 'from blog import something' in a.py , it show error:
from blog import BaseRequestHandler
ImportError: cannot import name BaseRequestHandler
i think it import the blog folder,not the blog.py
so how to import the blog.py
updated
when i use 'blog.blog', it show this:
from blog.blog import BaseRequestHandler
ImportError: No module named blog
updated2
my sys.path is :
['D:\\zjm_code', 'D:\\Python25\\lib\\site-packages\\setuptools-0.6c11-py2.5.egg', 'D:\\Python25\\lib\\site-packages\\whoosh-0.3.18-py2.5.egg', 'C:\\WINDOWS\\system32\\python25.zip', 'D:\\Python25\\DLLs', 'D:\\Python25\\lib', 'D:\\Python25\\lib\\plat-win', 'D:\\Python25\\lib\\lib-tk', 'D:\\Python25', 'D:\\Python25\\lib\\site-packages', 'D:\\Python25\\lib\\site-packages\\PIL']
zjm_code
|-----a.py
|-----b.py
a.py is :
c="ccc"
b.py is :
from a import c
print c
and when i execute b.py ,i show this:
> "D:\Python25\pythonw.exe" "D:\zjm_code\b.py"
Traceback (most recent call last):
File "D:\zjm_code\b.py", line 2, in <module>
from a import c
ImportError: cannot import name c

When you are in a.py, import blog should import the local blog.py and nothing else. Quoting the docs:
modules are searched in the list of directories given by the variable sys.path which is initialized from the directory containing the input script
So my guess is that somehow, the name BaseRequestHandler is not defined in the file blog.py.

what happens when you:
import blog
Try outputting your sys.path, in order to make sure that you have the right dir to call the module from.