I have a folder called Script and inside I have temp.py script. My temp script imports module from sub-folder called lib.
Lib folder has inside empty __init__.py and my parent_computer_test.py script.
in my temp.py script is the following code:
import lib.parent_computer_test as parent_computer_test
parent_computer_test.mainChunk()
parent_computer_test.splitChunks()
I managed to import module from sub-folder without any bigger problems.
This workflow/script works fine, BUT for a specific reason, my lib folder has to be somewhere else on my computer. There is a long story why, but it has to be that way.
Long story short. I want that my temp.py from the /Script folder imports modules from folder lib (or any other name) with parent_computer_test.py, but at the same time this folder is no sub-folder of /Script - so it is somewhere else in the computer. It can be C:/development/... or something.
So my question is how to import a module from a specific folder?
Append the path to lib folder to the SYS PATH Environment Variable. Then it can be imported from anywhere
import os, sys
lib_path = os.path.abspath(os.path.join('..', '..', '..', 'lib'))
sys.path.append(lib_path)
import mymodule
import imp
yourModule = imp.load_source('yourModuleName', '/path/to/yourModule.py')
foo = yourModule.YourFunction("You", "get", "the", "idea.")
I realize this is a special case, but in general, I would avoid stuff like this. Things can get ugly when you use absolute paths, especially if you move things around, and I would only use this for throwaway scripts or on systems that aren't going to change very much.
EDIT: Aswin's answer is a much better longterm solution.
You have to use sys.path.append("path") . But use this just one time. Then try
import "my_module". IT should be fine.
If you want to remove the path appended you can use,
sys.path.remove("path").
Related
I have my application folder with my main.py and a lib folder with some modules:
LeapMotionInterpreter --> DataExtractor.py
lib --> Leap.py
I've already looked at how to do this and they all say to use sys.path.insert or sys.path.append but nothing seems to work, python doesn't even seem to know there is a lib folder there even though printing the sys.path does show the new added path to lib.
Here is the code I have tried:
1.
import sys
sys.path.insert(0, 'C:/Users/winba/Desktop/TYP/LeapMotionInterpreter/lib')
print (sys.path)
outputs:
['C:/Users/winba/Desktop/TYP/LeapMotionInterpreter/lib', 'C:\\Users\\winba\\Desktop\\TYP\\LeapMotionInterpreter'
import sys
sys.path.append('C:/Users/winba/Desktop/TYP/LeapMotionInterpreter/lib')
print (sys.path)
This one just outputs the lib path at the end of the array, I have also tried writing the paths with single \ or double \ instead of / but all of these just give me a 'No module named 'Leap' error if I try to import it and doesn't even acknowledge the existence of the lib folder. Also, putting my module in the same file seems to work as python is able to find the Leap.py file instantly however this is going to be a fairly big project and I wanted to keep it neat using folders so any help would be appreciated. Thank you.
If I'm understanding correctly, the directory you're targeting is a sub-directory of the current directory. You can add the file to the Python path at runtime by giving it it's relative path:
# DataExtractor.py
import sys
sys.path.insert(1, 'lib')
import Leap
This should solve the problem
Desiring to improve my Python style, I ran a PEP-8 style checker on one of my script and it complained about something I don't know how to fix. The prologue of the script is something like:
#! /bin/env python3
import sys
import os
exe_name = os.path.basename(os.path.realpath(__file__))
bin_dir = os.path.dirname(os.path.realpath(__file__))
inst_dir = os.path.dirname(bin_dir)
sys.path.insert(0, inst_dir+'/path/to/packages')
import mypackage.mymodule
and the style checker complain on the import mymodule line, stating that it should be a top of file. But I obviously can't move it before setting the path where it will be found. Is there a good way to achieve this (mandating an environment variable or a shell wrapper are not what I find better than my current code) while respecting PEP-8 recommendations at the same time?
If you want to avoid path manipulation, you may be able to do so by using the under-known .pth feature.
sys.path should begin with the directory containing the main program either by name or by reference as ''. I assume that the file importing mymodule is not part of mypackage, so that the '' entry is not useful for importing mymodule.
sys.path should end with the site-packages directory for the executing binary. That is the normal place for added packages. If you do not want to move mypackage into site-packages, you can extend the latter 'vitually' by putting a mystuff.pth file in it. It should contain one line: the path to the directory containing mypackage. Call it myprojects. Then mypackage and any other package in myprojects can be imported as if they were in site-packages.
One advantage of .pth files is that you can put identical copies in multiple site-packages directories. For instance, I have multiple projects in F:/python. I have multiple versions of Python installed. So I have put python.pth containing that one line in the site-packages for each.
The best strategy would be to put the sys.path related code in separate file and import it in working code file.
So, I will split above code in two files. One named my_module.py and other named working_module.py
Now my_module will contain below lines
import sys
import os
exe_name = os.path.basename(os.path.realpath(__file__))
bin_dir = os.path.dirname(os.path.realpath(__file__))
inst_dir = os.path.dirname(bin_dir)
sys.path.insert(0, inst_dir+'/path/to/packages')
and working_module will contain
import my_module
import mypackage.mymodule
because we are importing my_module before mypackage, it will execute the path related code before and your package will be available in path.
You can use importlib module (python 3.5) or imp for python 2.7 to load mypackage.mymodule programatically. Both have the same purpose:
mechanisms used to implement the import statement
This question might help you more:
How to import a module given the full path?
https://docs.python.org/3/library/importlib.html#examples
https://docs.python.org/2/library/imp.html
I had a small problem with importing a script that was in a parent folder, but I managed to resolve it using:
import sys
sys.path.append("../")
My directory is like this:
Data
|->->code
|->->script1.py
|->->->->subfolder
|->->->script2.py
When I run script2 (which imports script1) from the subfolder directory, the script runs without problems. But if I try to run script2 from the code directory using:
:~ ./subfolder/script2.py
I get an error :
ImportError: No module named script1
I tried using relative imports but because my code is not structured in packages it doesn't work. Is there a way I can run script2 from both directories (the parent and the child) and still be able to import script1 everytime?
Thank you in advance,
Georgi Nikolov
EDIT: Ok, after I read through all the suggestions, I did a "simple" hack which is quite ugly in my opinion but works quite well:
import sys
parent_folder = sys.path[0].split("/subfolder")[0]
sys.path.append(parent_folder)
import script1
Now I can even call script2 from the root and it will manage to import script1
To import a module that is up a level, you can use this.
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
An Explanation
__file__ # The full path to your running file.
os.path.dirname # See below. (1)
os.path.join # See below. (2)
sys.path.append # See below. (3)
".." # Universal for "up a level".
os.path.dirname(path) -
Return the directory name of pathname path.
os.path.join(path, *paths) -
Join one or more path components intelligently.
sys.path -
A list of strings that specifies the search path for modules.
You can add a string with the append method.
You should use absolute paths:
import sys, os
HERE = os.path.abspath(os.path.dirname(__file__))
sys.path.append(os.path.join(HERE, ".."))
But better do not do this: Guido views running scripts within a package as an anti-pattern
You should put the standalone scripts in the root folder of the project. A script should not be used as a module and as the main script.
If you cannot move the script to the root, make another bootstrap script at the root which will import your service script running a dedicated function in it.
I am running into pathing issues with some scripts that I wrote to test parsers I've written. It would appear that Python (27 and 3) both do not act like IronPython when it comes to using the current working directory as part of sys.path. As a result my inner package pathings do not work.
This is my package layout.
MyPackage
__init__.py
_test
__init__.py
common.py
parsers
__init__.py
_test
my_test.py
I am attempting to call the scripts from within the MyPackage directory.
Command Line Statements:
python ./parsers/_test/my_test.py
ipy ./parsers/_test/my_test.py
The import statement located in my my_test.py file is.
from _test.common import TestClass
In the python scenario I get ONLY the MyPackage/parsers/_test directory appended to sys.path so as a result MyPackage cannot be found. In the IronPython scenario both the MyPackage/parsers/_test directory AND MyPackage/ is in sys.path. I could get the same bad reference if I called the test from within the _test directory (it would have no idea about MyPackage). Even if i consolidated the test files into the _test directory I would still have this pathing issue.
I should note, that I just tested and if i did something like
import sys
import os
sys.path.append(os.getcwd())
It will load correctly. But I would have to do this for every single file.
Is there anyway to fix this kind of pathing issue without doing the following.
A. Appending a relative pathing for sys.path in every file.
B. Appending PATH to include MyPackage (I do not want my package as part of the python "global" system pathing)
Thanks alot for any tips!
Two options spring to mind:
Use the environment variable PYTHONPATH and include .
Add it to your sys.path at the beginning of your program
import sys.path
sys.path.append('.')
If you need to import a relative path dynamically you can always do something like
import somemodule
import sys
dirname = os.path.dirname(os.path.abspath(somemodule.__file__))
sys.path.append(dirname)
I know that there are plenty of similar questions on stack overflow. But the common answer doesn't seem to be working for me.
I have a file structure like this
proj/
lib/
__init__.py
aa.py
bb.py
test/
__init__.py
aa_test.py
I figured that if I include the code in my test.py
import lib.aa
or
from lib import aa
I would be able to reference the modules in the lib/ directory. But that did not work.
So I tried to add to path, and it adds it correctly:
os.environ["PATH"] += ":%s" % os.path.abspath(os.path.join("..",""))
print os.environ["PATH"]
but even now when I try the import statements above... I keep getting the error
ImportError: No module named aa
or
ImportError: Importing from non-package <Something...>
Is there something obvious I am missing?
Is there a way to check if I have configured my __init__.py files correctly, or to see my package hierarchy?
You need to update your sys.path, which is where python looks for modules, as opposed to your system's path in the current environment, which is what os.environ["PATH"] is referring to.
Example:
import os, sys
sys.path.insert(0, os.path.abspath(".."))
import aa
After doing this, you can use your functions in aa like this: aa.myfunc()
There's some more information in the accepted answer for python: import a module from a directory
The lib directory needs to be in your python module search path, which isn't the same things as the search path used by your shell.
This will probably work for you:
import sys, os
sys.path.append(os.path.abspath(".."))
However, it is probably better to run your code from a context where the lib package is already on the path. Such as from the 'proj' directory.
Where is the code that you're trying to import lib.aa from? I'm guessing /proj/ is not your working directory and it would need to be as it's setup right now. Instead of PATH, you would want to add your directory to PYTHONPATH so it appears in the search path for an import. See http://docs.python.org/tutorial/modules.html#the-module-search-path
Also, please take a look at http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html It strongly recommends you put an extra level of directory in place so instead of lib.aa, you would refer to it as my_proj.lib.aa.
I had similar problems and here is my advice.
Instead of changing sys.path, better run your test.py from being in proj (i.e. project root) directory. This way project dir will automatically be in sys.path and you will be able to import lib package.
And use absolute imports.
System PATH variable is not used by python import statement. It uses PYTHONPATH, but best way to add new directory to import search path is to modify sys.path.
If this does not help, add to the question your value of sys.path and value returned by os.getcwd().
In Sublime Text 3, abspath it didn't work for me.
I use this instead in the top "__ init __.py" file
Hope it works for you.
from os.path import dirname
from sys import path
path.insert( 0 , dirname( __file__ ) ) ;
from test import aa_test