I am fairly new to Python (3.3.2) and I have minimal experience using
from .... import ....
beyond the use of time, turtle, and math. I am working on several Project Euler problems that require a function to determine the primality of a number using my simple isPrime(n) function. The isPrime(n) function is the only function in a file named isPrime.py.
I have a separate folder for each Project Euler problem I attempt to keep my code straight but I find it inconvenient having to copy my isPrime(n) function directly from isPrime.py and paste it into every new file within a folder devoted to each problem. I have some ideas about improving my isPrime(n) function and I don't want to have to open up several of the Problem folders just to change the lines in each file when I make some revisions.
So I thought that writing the following line at the top of each Problem's file would work:
from isPrime.py import isPrime
or even from isPrime import isPrime
However, I encounter the following error both ways:
ImportError: No module named 'isPrime'
Here is a basic description of my folder hierarchy:
ProjectEuler Folder
Primality Functions Folder
isPrime.py [contains the function isPrime(n)]
anotherFile.py
yetAnotherFile.py
Problem X Folder
problemX.py
something.py
somethingElse.py
Problem Y Folder
problemY.py
something2.py
somethingElse2.py
Problem Z Folder
problemZ.py
something3.py
somethingElse3.py
My question is:
What do I need to type in the opening lines of the files problemX.py,problemY.py, and problemZ.py to use the function isPrime(n) from the file isPrime.py in the folder Primality Functions?
I have searched at length here at stackoverflow and I see many questions are related to Python and difficulty with importing files/modules. However, I don't understand what a system path is, or what a relative import is, and I have no idea what the Python documentation found here is telling me. Please do not mark this question as duplicate of the many Python/Import questions already here. All I'm looking for is a simple 1-3 lines of code to place at the top of my files so I can import my functions from other folders and won't have to copy-and-paste my functions every time I attempt a new problem. I will be most appreciative of the most simply-worded answer. Thank you!
I had the same problem as you i.e., I had some utility modules that may be useful to solve multiple project euler problems and each project euler problem had its own directory.
Initially I put the utility folder into sys.path as #user3114046 suggested:
import sys; sys.path.insert(0, '../Primality Functions Folder')
from isPrime import isPrime
It felt dirty but it worked (note: I even used a relative path here!).
As an alternative, you could create a simple setup.py file in the same directory as isPrime.py:
from distutils.core import setup
NAME = 'isPrime'
setup(name=NAME, version='0.0.1', py_modules=[NAME])
and install your utility module:
$ python setup.py install
after that you could use it in any script:
from isPrime import isPrime
If you want to use several utility modules; you could put them into project_euler_utils directory and install it as Python package. Put setup.py along side project_euler_utils directory:
from distutils.core import setup
NAME = 'project_euler_utils'
setup(name=NAME, version='0.0.1', packages=[NAME])
Run pip install project_euler_utils, to install it. After that; you could use it from any script/module:
from project_euler_utils.isprime import isprime
Note: I've used lowercase names as pep-8 suggests (you need to rename your module and the function in it).
I tried using the fullest path I can think of with "C:\Users\Owner\Documents\Primality Functions Folder", it gave me a Syntax Error: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape.
Use raw string literals for Windows paths r'C:\Users\Owner...' (note: r'' prefix) Otherwise backslash is special inside literal strings, in particular u'\U0001F385' is a not 10 characters; it is a single Unicode codepoint U+1F385.
Since you want to keep the modules you are importing in different directories, you can tell python where to find them by adding those directories to sys.path:
from sys import path
path.append('fully specified path name of directory with module you want to import')
from mymodule import myfunction
This doesn't need to be done if isPrime.py is in the same folder as the program which imports it, but sometimes this is not practical.
It's not optimal for the module to have the same name as one of its functions, though, so I would consider changing it.
Related
I’ve been working a lot lately with python 3 and I found that I am unable to import a module from a separate folder. Is there a way to import it while it’s in a folder subdivision?
To give more context of this issue here is the “launcher” location and folder that I want to access:
Launcher.py
Folder
- program-to-import.py
That’s the layout. How do I import into launcher from my other module?
As mentioned by others, - in name is invalid, try importing after removing them if you have them in your file name. For now, let's call it program_to_import
from folder import program_to_import
And to call a function from program_to_import, you use this -
program_to_import.function_to_call()
Also, it's always a good idea to look at the documentation
You could also try by adding an __init__.py in your folder. The use of __init.py__ is as follows -
The init.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later (deeper) on the module search path. In the simplest case, init.py can just be an empty file, but it can also execute initialization code for the package or set the all variable, described later.
Python supports importing from child paths rather trivially.
in Launcher.py enter the following.
from Folder.program-to-import import *
I'm using a from . import module statement to do exactly that: import a local module to my script. The script and module reside in the same folder.
# module.py
def foo():
print('Foo!')
# script.py
from . import module
module.foo()
> ImportError: cannot import name 'module'
This should be pretty easy, and doing just import module does work, but as this answer suggests one should, I modified the statements to the former form.
The end goal is to have a package, from which I can use things, but also to have executable scripts inside the package that import other parts of that package. Apparently, after a few days worth of searching and a few questions I still don't quite understand the import and packaging machinery.
These might be the cause:
Import statements are different in 2.7 and 3.x, I'm using 3.6, the question was on 2.7
Relative imports are different inside packages (folder with __init__.py)
The working directory is different or the folders are not in sys.path
Having an __init__ file does not make a difference at least in a fresh project in PyCharm. Also, the working directory is set to the folder of the sources and it is in path.
Have I missed something? Or rather, what's the correct way of achieving the functionality described in the end goal? Any help is greatly appreciated!
Since writing this answer I have realised it is more convenient and better style in my humble opinion to install the package with pip install -e . and use absolute imports. So even within a package writing from package.sub.module import thing. This makes refactoring a lot easier and there's no need to ever manipulate module variables or sys.path.
When running a script directly, Python consideres the name (a special variable, __name__) of that script to be "__main__". In case of an import, the name is set to the name of the module. In the latter case relative imports are fine. But import actually looks at the combination of __name__ and another special variable, __package__, which is None for an executed script, but the path to a module for an imported module, e.g. parent.sub.
The searched variable is... drumroll...
__package__ + '.' + __name__
The secret ingredient is manipulating __package__:
# script.py
__package__ = 'package_name' # or parent.sub.package
from . import module
This lets Python know you are inside a package even though the script is executed directly. However, the top level folder needs to be in sys.path and the package name has to reflect that directory structure.
See this very comprehensive answer on the topic of relative imports.
I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures
Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.
Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.
This import works fine, but feels dirty in a few ways. Mainly that it uses a specific number in the slice* to get the parent path, and that it annoys the flake8 linter.
import os
import sys
sys.path.append(os.path.dirname(__file__)[:-5])
from codeHelpers import completion_message
It's in a file system that looks a bit like this:
parent_folder
__init__.py
codeHelpers.py
child_folder
this_file.py
(child_folder is actually called week1, hence the 5 in the slice)
This question is extremely similar to Python import from parent directory, but in that case the discussion focused on whether or not it was good to run tests from the end point. In my case, I have a series of directories that have code that uses helpers that live in the parent.
Context: each directory is a set of weekly exercises, so I'd like to keep them as simple as possible.
Is there a cleaner, more pythonic way to do this import?
#cco solved the number problem, but it's still upsetting the linter.
First since you haven't been specific about which lint error you are getting, I am going to assume it's because you have an import after your sys.path.append.
The cleanest way to do it is with relative or absolute imports.
Using absolute imports:
from parent_path.codeHelpers import completion_message
Using relative imports:
from ..codeHelpers import completion_message
For the simple example listed in the original question this should be all that's required. It's simple, it's pythonic, it's reliable, and it fixes the lint issue.
You may find yourself in a situation where the above does not work for you and sys.path manipulation is still required. A drawback is that your IDE will likely not be able to resolve imports to modules from the new path causing issues such as automatic code completion not working and flagging the imports as errors, even though the code will run properly.
If you find you still need to use sys.path and want to avoid lint errors for this type of situation create a new module and do the sys.path manipulation in it instead. Then make sure that you import your new module before any modules that require the modified sys.path.
For example:
local_imports.py
"""Add a path to sys.path for imports."""
import os
import sys
# Get the current directory
current_path = os.path.dirname(__file__)
# Get the parent directory
parent_path = os.path.dirname(current_path)
# Add the parent directory to sys.path
sys.path.append(parent_path)
Then in the target file:
import local_imports # now using modified sys.path
from codeHelpers import completion_message
The drawback to this is it requires you to include local_imports.py in each child_folder and if the folder structure changes, you would have to modify each one local_imports file.
Where this pattern is really useful is when you need to include external libraries in your package (for example in a libs folder) without requiring the user to install the libs themselves.
If you are using this pattern for a libs folder, you may want to make sure your included libraries are preferred over the installed libraries.
To do so, change
sys.path.append(custom_path)
to
sys.path.insert(1, custom_path)
This will make your custom path the second place the python interpreter will check (the first will still be '' which is the local directory).
You can import from a module a level up in a package by using ... In this_file.py:
from ..codeHelpers import completion_message
Had you wanted to go more levels up just keep adding dots...
While I'm here, just be aware that from ..codeHelpers is a relative import, and you should always use them when importing something in the same package. from codeHelpers is an absolute import, which are ambiguous in Python 2 (should it import from in the package or from the unfortunately named codeHelpers module you have installed on your system?), and in Python 3 actually forbidden as a way to import from within the same module (i.e. they are always absolute). You can read the ancient PEP 328 for an explanation of the difficulties.
It might be easier to use absolute import paths, like the following:
from parent_folder.code_helpers import completion_message
But this would require you to make sure that the PYTHONPATH environment variable is set such that it can see the highest root directory (parent_folder in this case, I think). For instance,
PYTHONPATH=. python parent_directory/child_directory/this_file.py
# here the '.' current directory would contain parent_directory
Make sure to add an __init__.py to the child_directory as well.
You can remove the assumption about the length of the final directory name by applying os.path.dirname twice.
e.g. instead of os.path.dirname(__file__)[:-5], use os.path.dirname(os.path.dirname(__file__))
Either way you have to hack around. If you main goal avoid flakes warnings
add a noqa comment
exec(open("../codeHelpers.py").read(), globals())
you can pass a filename with interpreter option -c (should not bother flakes8)
I am bundling python source code with py2exe. The directory structure is as follows:
some_Mod.py
some_dir/another_dir/some_Mod.py
Inside the latter some_dir/another_dir/some_Mod.py I am trying to import the other Python Module with
from ..some_Mod import *
Using the import causes no problems with the python interpreter, but if I run the same constellation in the bundled package, I get an Exception:
ImportError: No module named some_Mod
Can somebody explain why?
Remark: Renaming the Modules is actually no problem, but I was just wondering, why py2exe cannot deal with this constellation.
If you have __init__.py files in each of those sub-directories then all import statements should work correctly.
Assuming that's not the problem, here's an excellent guide to importing best practices:
http://blog.habnab.it/blog/2013/07/21/python-packages-and-you/
In summary, never use relative imports - always absolute (see the link above for why).
Second (and I'm not entirely sure why), always keep your py2exe setup.py script in the exact folder where your main script is. I've tried modifying py2exe's 'script' option to allow my script to be somewhere else... but your exact problem happened to me. So, try making sure it is right where the main script is.
Finally, you can always give py2exe a little help. I usually have to add the root directory to the system path so the import statements are valid. Note, I'm not modifying sys.path in any of my application's code - only the py2exe script I use to build the exe.
At the top of my py2exe setup script:
import sys
sys.path.append(PATH_WHERE_PACKAGES_ARE)
# add any packages that need explicit importing here located in root directory:
import package1 # apparently it wasn't found...
import package2 # apparently same thing
Generally I don't import packages though, adding the project root where they exist usually is enough.
I'm not sure that py2exe now how to handle the from ..some_Mod import * syntax, check this to ensure that the some_Mod.py module is correctly packaged : python -m py2exe.mf -d some_dir/another_dir/some_Mod.py as explained in the py2exe FAQ