I'm having an issue with a failing import statement, it is called by a manage.py command. It works in the manage.py shell. It also recently worked, i've tried to retrace my steps to no avail. Any advice?
Your question does not have enough information to answer definitively, but I can at least offer some hints to debug the problem.
Understand the import statement
Read and understand the documentation for the import statement for your version of Python..
Check the Python path
One key step towards debugging any import problem is to ensure that your module is available on the python path. Add the following code to the code that's having the problem:
import sys
print "\n".join( sys.path )
Somewhere in that list must be the directory tree that contains your module. If it's not there, you'll either have to reference your module differently, or add the right directory to the python path. Keep in mind that Python is a dynamic language -- the python path can be changed as a program runs, and what matters is the state of the python path at the time of them first import of a module.
Remember to add an __init__.py file to your packages
It must be present, even if empty.
Search StackOverflow before asking a question
A simple search for [python] import or [django] import may turn up a similar question, with answers that fit your situation.
Related
I have a specific problem which might require a general solution. I am currently learning apache thrift. I used this guide.I followed all the steps and i am getting a import error as Cannot import module UserManager. So the question being How does python import lookup take place. Which directory is checked first. How does it move upwards?
How does sys.path.append('') work?
I found out the answer for this here. I followed the same steps. But i am still facing the same issue. Any ideas why? Anything more i should put up that could help debug you guys. ?
Help is appreciated.
On windows, Python looks up modules from the Lib folder in the default python path, for example from "C:\Python34\Lib\". You can add your Python libaries in a custom folder ("my-lib" or sth.) in there, but you need a file in order to tell Python that you can import from there. This file is called __init__.py , and is totally empty. That data structure should look like this:
my-lib
__init__.py
/myfolder
mymodule.py
(This is how every Python module works. For example urllib.request, it's at "%PYTHONPATH%\Lib\urllib\request.py")
You can import from the "mymodule.py" file by typing
import my-lib
and then using
mylib.mymodule.myfunction
or you can use
from my-lib import mymodule
And then just using the name of you function.
You can now use sys.path.append to append the path you pass into the function to the folders Python looks for the modules (Please note that thats not permanent). If the path of your modules should be static, you should consider putting these in the Lib folder. If that path is relative to your file you could look for the path of the file you execute from, and then append the sys.path relative to your file, but i reccomend using relative imports.
If you consider doing that, i recommend reading the docs, you can do that here: https://docs.python.org/3/reference/import.html#submodules
If I got you right, you're using Python 3.3 from Blender but try to include the 3.2 standard library. This is bound to give you a flurry of issues, you should not do that. Find another way. It's likely that Blender offers a way to use the 3.3 standard library (and that's 99% compatible with 3.2). Pure-Python third party library can, of course, be included by fiddling with sys.path.
The specific issue you're seeing now is likely caused by the version difference. As people have pointed out in the comments, Python 3.3 doesn't find the _tkinter extension module. Although it is present (as it works from Python 3.2), it is most likely in a .so file with an ABI tag that is incompatible with Blender's Python 3.3, hence it won't even look at it (much like a module.txt is not considered for import module). This is a good thing. Extension modules are highly version-specific, slight ABI mismatches (such as between 3.2 and 3.3, or two 3.3 compiled with different options) can cause pretty much any kind of error, from crashes to memory leaks to silent data corruption or even something completely different.
You can verify whether this is the case via import _tkinter; print(_tkinter.file) in the 3.2 shell. Alternatively, _tkinter may live in a different directory entirely. Adding that directory won't actually fix the real issue outlined above.
For any new readers coming along that are still having issues, try the following. This is cleaner than using sys.path.append if your app directory is structured with your .py files that contain functions for import underneath your script that imports those files. Let me illustrate.
Script that imports files: main.py
Function files named like: func1.py
main.py
/functionfolder
__init__.py
func1.py
func2.py
The import code in your main.py file should look as follows:
from functionfolder import func1
from functionfolder import func2
As Agilix correctly stated, you must have an __init__.py file in your "functionfolder" (see directory illustration above).
In addition, this solved my issue with Pylance not resolving the import, and showing me a nagging error constantly. After a rabbit-hole of sifting through GitHub issues, and trying too many comparatively complicated proposed solutions, this ever-so-simple solution worked for me.
You may try with declaring sys.path.append('/path/to/lib/python') before including any IMPORT statements.
I just created a __init__.py file inside my new folder, so the directory is initialised, and it worked (:
A few hours ago I was careless enough to name my short script as code.py. Apparently, there is such a package which is used e.g. by ptvsd or pdb. This led to my code.py to be imported instead and caused a bunch of nested unhandled exceptions with missing imports upon trying to debug my code. What was making it more frustrating is that traceback didn't show any sign of importing my code.py file, so I spent quite a while to find the source of the problem.
I'd like to avoid such situations in the future, so my question is: what's the best practice to ensure that the modules you use aren't importing your code by mistake due to such a name collision?
This is a common gotcha, and actually there's no failsafe way to avoid it. At least you can make sure your modules all live in packages (at least one package if that's a small project with no reusable code) so that you'd use them as from mypackage import code instead of import code (also make sure you use either absolute imports etc), and that you always run your code from the directory containing the package(s), not from within the package directory itself (python inserts the current working directory in first position of sys.path).
This won't prevent ALL possible name masking issues but it should minimize them. Now from experience, once you've experienced this kind of issues at least once, you usually spot the symptoms very quickly - the most common and quite obvious being that some totally unrelated stlib or third-part module starts crashing with ImportErrors or AttributeErrors (with "module X has no attribute Y" messages). At this point, if you just added a new module to your own code, chances are it's the new module that breaks everything, so you can just rename it (make sure you clean up .pyo/.pyc files if any) and see if it solves the issue. Else check the traceback to find out which imports fails, most of the time you'll find you have a module or package by the same name in your current working directory.
You can't avoid completely, that somebody is able to import your module by mistake.
You can structure your code better in subpackages going from "well known" to "less known" names. E.g. if you are developing code for a certain company then you might want to structure like:
company.country.location.department.function
If your code is then getting more accepted and used by others, you can bring the code in the upper namespace, so that it is made available in company.country.location.department.function
and company.country.location.department
You can modify sys.path at the beginning of your main module, before you start importing other modules:
import sys
sys.path.append(sys.path.pop(0))
so that the main module's starting directory is placed at the last of the module search paths rather than at the front, in order for other modules of the same name to take precedence.
EDIT: To all the downvoters, this answer actually works.
For example, running code.py with the following content:
import pdb
pdb.run('print("Hello world")')
would raise:
AttributeError: module 'pdb' has no attribute 'run'
because code.py has no run defined, while running code.py with the following content instead:
import sys
sys.path.append(sys.path.pop(0))
import pdb
pdb.run('print("Hello world")')
would execute pdb.run properly:
> <string>(1)<module>()
(Pdb)
I recently upgraded from python2.7 to python3 and think it may have screwed up some configurations. Now when I try to run a module, I get import errors. Let's say I have a directory structure like this:
/directory
/directory/__init__.py
/directory/run.py
/directory/app/db.py
/directory/app/views.py
/directory/app/__init__.py
with the following imports...
/directory/run.py says 'import app'
/directory/app/db.py says 'import views'
When I execute run.py, I get an error saying the module views cannot be found. However, if I go into /directory/app and execute db.py, then the import runs correctly. I've also found that if I change the /directory/app/db.py to say "from app import views" then it works correctly when executing run.py. However, this used to all work!
It seems like the import statements are not taking into account the folder it's being executed in. It seems like this wants me to base all of my imports out of the root folder, which seems incorrect and would take me time to change everything.
Any ideas as to what happened? This has been driving me crazy.
In Python3, implicit relative imports have been removed, all imports need to be either absolute, or use explicit relative imports.
This is not going to change, you need to either replace them with from app import views or from . import views.
Python 2.x and Python 3.x differ in so many ways it is usually extremely helpful to use 2to3 or another similar tool to "port" (convert) the code.
The issue you are running into likely has to do with the fact that Python 2 uses relative imports but Python 3 uses absolute imports (I may have that backwards). It is possible to change the import statement to make the import work, though for the torrent of compatibility issues that will surely follow, I highly recommend using 2to3 and then making any final adjustments manually.
Good luck!
I realize this may be considered a duplicate question to some of the other questions out there, but I've spent over an hour now reading through various pages and documentation and still don't understand what's going on here.
I'm trying to work with python files in multiple directories; I have essentially this:
myproject/
__init__.py
some_file.py
some_data.dat
tests/
__init__.py
test_some_file.py
test_some_file.py is run from the command line, as the name implies, intended to run the code contained in some_file.py, and needs to import it. However, I can't seem to do so.
I've tried:
from myproject import some_file
and also
from .. import some_file
I did manage to make it run using sys.path, but that doens't seem to be the correct way to do things based on what I've read.
Secondly, when I did make it run, using sys.path, I got an error that it couldn't find some_data.dat which is used by some_file.py.
This is a perennial question from Python programmers. The issue is that Python doesn't play nicely with scripts that are inside of packages. The situation has improved a bit over the last few releases, but it still doesn't do the right thing a lot of the time.
I think the best answer is to restrict where you run your test_some_file.py from, and use the Python interpreter's -m parameter. That is, change into the directory above myproject, and then run python -m myproject.tests.test_some_file. This is the only way that will work without messing around with sys.path.
This will allow either of your import lines to work correctly. PEP 8 currently recommends using absolute imports always, so the first version is probably better than the relative version using ...
For cases like yours, I add the directory of some_file.py to sys.path (temporarily).
Code:
import sys, os
dirname = os.path.dirname( # going up by 1 directory
os.path.dirname( # going up by 2 directories
sys.argv[0]))
sys.path.append(dirname)
import some_test
I have two Python packages where one needs to be imported by the other. The directory structure is like follows:
workspace/
management/
__init__.py
handle_management.py
other_management.py
utils/
__init__.py
utils_dict.py
I'm trying to import functionality from the utils project in the handle_management.py file:
import utils.utils_dict
Error I'm getting when trying to run handle_management.py:
ImportError: No module named utils.utils_dict
I've read a lot about how to resolve this problem and I can't seem to find a solution that works.
I started with Import a module from a relative path - I tried the applicable solutions but none worked.
Is the only solution to make workspace/ available via site_packages? If so, what is the best way to do this?
EDIT:
I've tried to add the /home/rico/workspace/ to the PYTHONPATH - no luck.
EDIT 2:
I was able to successfully use klobucar's solution but I don't think it will work as a general solution since this utility is going to be used by several other developers. I know I can use some Python generalizations to determine the relative path for each user. I just feel like there is a more elegant solution.
Ultimately this script will run via cron to execute unit testing on several Python projects. This is also going to be available to each developer to ensure integration testing during their development.
I need to be able to make this more general.
EDIT 3:
I'm sorry, but I don't really like any of these solutions for what I'm trying to accomplish. I appreciate the input - and I'm using them as a temporary fix. As a complete fix I'm going to look into adding a script available in the site_packages directory that will add to the PYTHONPATH. This is something that is needed across several machines and several developers.
Once I build my complete solution I'll post back here with what I did and mark it as a solution.
EDIT 4:
I feel as though I didn't do a good job expressing my needs with this question. The answers below addressed this question well - but not my actual needs. As a result I have restructured my question in another post. I considered editing this one, but then the answers below (which would be very helpful for others) wouldn't be meaningful to the change and would seem out of place and irrelevant.
For the revised version of this question please see Unable to import Python package universally
You have 2 solutions:
Either put workspace in your PYTHONPATH:
import sys
sys.path.append('/path/to/workspace')
from utils import utils_dict
(Note that if you're running a script inside workspace, that is importing handle_management, most probably workspace is already in your PYTHONPATH, and you wouldn't need to do that, but it seems it's not the case HERE).
Or, make "workspace" a package by adding an empty (or not) __init__.py file in the workspace directory. Then:
from ..utils import utils_dict
I would prefer the second, because you would have a problem if there's another module called "utils" in you PYTHONPATH
Apart from that, you are importing wrong here: import utils.utils_dict.py. You don't need to include the extension of the file ".py". You are not importing the file, you are importing the module (or package if it's a folder), so you don't want the path to that file, you need its name.
What you need to do is add workspace to your import path. I would make a wrapper that does this for you in workspace or just put workspace in you PYTHONPATH as an environment variable.
import sys
# Add the workspace folder path to the sys.path list
sys.path.append('/path/to/workspace/')
from workspace.utils import utils_dict
Put "workspace/" in your PYTHONPATH to make the packages underneath available to you when it searches.
This can be done from your shell profile (if you are on *nix, or environment variables on windows.
For instance, on OSX you might add to your ~/.profile (if workspace is in your home directory):
export PYTHONPATH=$HOME/workspace:$PYTHONPATH
Another option is to use virtualenv to make your project area its own contained environment.