I am working on a project involving the popular aiortc/aioquic github package. Located at src/aioquic/buffer.py there is an import statement:
from ._buffer import Buffer, BufferReadError, BufferWriteError # noqa
This line is meant to import a C/stub module _buffer.c / _buffer.pyi . When used from the root directory to run examples found in the examples directory this import line will work, but when trying to run in a Docker container from the same directory this import fails and when importing from the /src directory the import fails. I have tried reading several other questions on python relative imports but none seem to make sense to me.
To reproduce the issue:
open python in the src directory
from aioquic import about (works)
from aioquic import buffer (fail)
from aioquic import buffer
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/aioquic/src/aioquic/buffer.py", line 1, in <module>
from ._buffer import Buffer, BufferReadError, BufferWriteError # noqa
ModuleNotFoundError: No module named 'aioquic._buffer'
I don't understand why the examples http3_client and http3_server work but when trying to import it directly it fails. I thought I understood roughly how to use python imports but apparently not. What would cause the same line to work when running from one directoy but fail when running from another directory? Is it something magical with the PYTHONPATH? I'm hoping someone smarter than I am can explain why this is failing.
[EDIT and most importantly how to fix this so the import will work nomatter where the user runs python from. I have tried several different combinations but none seem to work.]
Thanks in advance.
I figured out what was wrong.
The stub/C module had to be built into a .pyd file before it could imported. This was done by
pip install -e .
on the parent directory
Is it something magical with the PYTHONPATH?
Yes.
Typically . dot (current working directory) will appear
in that exported env var,
so where you started matters.
Take care to ensure you're using the same env var setting,
and the same pwd,
during both the interactive and the Docker runs.
Related
I have a question that I assume has a simple answer, but for some reason I am struggling to find it on my own. I have created and activated a virtual environment with virtualenv, and I am trying to install all the necessary packages in order to create a requirements.txt file.
I have, for example, a Python file that begins like this:
import xml.etree.ElementTree as ET
from lib.project import Projector
from lib import writer
import os
import datetime
from datetime import timedelta
from datetime import datetime
import pprint
When I try to run this file from the virtual machine, I receive the following error:
Traceback (most recent call last):
File "readMap.py", line 2, in <module>
from lib.project import Projector
ModuleNotFoundError: No module named 'lib.project'
My problem is that I'm not sure why the virtual environment can't find project.py. My directory structure is:
regiaoSul
lib
__init__.py
arrival_conversion.py
coord_conversion.py
message_conversion.py
project.py
route_conversion.py
stop_conversion.py
wkt_parser.py
writer.py
readMap.py
json_generator.py
The import on line 2 implies lib is a module rather than "a simple repository".
I will try running the script with the flag -m. Something like this -
python -m script_name
make sure to drop the .py extension when you run with -m flag.
Another advice: you don't need to install python files to the virtual environment, they are not some external libraries. They only need to be present (with the same order of packaging) when you run your script.
Thanks to everyone who responded. I believe the issue was some sort of dependency problem. In readMap.py I had imported writer from lib, and in writer.py I had imported Projector from project. I moved the function that required Projector from writer.py to readMap.py and it worked.
I still don't fully understand why this was a problem. Until recently I had been running my scripts in PyCharm and they all worked with the structure I had. It was only when I tried to run them from the command line in my virtual machine that they didn't work.
If anybody would like to explain the distinction to me and what the exact problem was with my imports, feel free to.
I sometimes face the same issue. A solution is to add the path to sys.path by:
import sys
sys.path.insert(0, "/path/to/your/package_or_module")
I have a folder with a file "main.py" and a file "_test.pyd" (note .pyd). The file "main.py" looks like this:
import _test
I get the following error:
Traceback (most recent call last):
File "main.py", line 1, in <module>
import _test
ImportError: DLL load failed while importing _test: The specified module could not be found.
Why is this error coming up? Cheers.
Note: I was given this code by others, and it works for the original authors, so I'm not sure what's wrong with me/my machine.
Update: Running os.path.isfile('_test.pyd') returns True, so I don't think it's a problem with the path
You should append the path of the folder which contains the imported module before import.
Code:
import os
import sys
sys.path.append(os.path.realpath(os.path.dirname(__file__)))
import _test # noqa: E402
EDIT:
Other ideas:
Adding __init__.py file to the related director.
Checking the PyInit_foo() function in .pyd file.
If the Python finds the .pyd file, it will attempt to call PyInit_foo() to initialize it
Update Following posts from people experiencing similar issues, I tried downgrading my Python version (from 3.8.4rc1 to 3.5.4) and the import now works correctly. No clue why. I guess the .pyd file was written in that version of Python (I'm not the author of the file), but still I'm clueless as to what the exact origin of the problem is.
I've been through this error and what I found after a lot of investigation:-
issue was in Opencv==4.5.1 build from source with cuda and flag cuda_with_fast_math=on
I just rebuild OpenCV and disable
cuda_with_fast_math
and it works for me.
I'm having problem when I trying to import a custom module I have which is very simple but I'm always getting:
Traceback (most recent call last):
File "demo_module1.py", line 12, in <module>
import mymodule
ModuleNotFoundError: No module named 'mymodule'
I have tried to set environment variables:
set PYTHONHOME=C:\Software\python-3.7.4
set PYTHONPATH=%PYTHONPATH%;C:\pyproys\test
Everything is located here: 'C:\pyproys\test'
The only way it works is if I add it directly in the code "But I don't want to do it in every single script I have so don't want to maintain it in that way".
import sys
sys.path.append('C:\pyproys\\test')
print(sys.path)
Here is the script I'm trying to run:
demo_module1.py
import mymodule
mymodule.greeting("Jonathan")
'mymodule.py' is in the same folder as 'demo_module1.py'
I'm expecting the code to run fine by just executing:
python demo_module1.py
Can someone please point me out what I'm doing wrong?
Try to find the directory /lib/site-packages in your Python folder in C drive and paste your module in that folder and then restart the system. Hope it'll solve your issue.
I am trying to import Pybedtools in Spyder.
from pybedtools import BedTool
This is the error I am getting:
Traceback (most recent call last):
File "<ipython-input-13-7a8ea5d1dea8>", line 1, in <module>
from pybedtools import BedTool
File "/Users/michaelsmith/anaconda2/lib/python2.7/site-packages/pybedtools/__init__.py", line 9, in <module>
from . import scripts
ImportError: cannot import name scripts
I just downloaded Anaconda and there doesn't seem to be a reason as to why this happens. What is the typical protocol for resolving bugs like this?
UPDATE:
So within my pybedtools folder there is a scripts folder (which is presumably the module we're trying to import). I changed both the command within __init__.py to:
from . import scripts2
and changed the name of the folder to scripts2 as well. However, I still get the error as such:
ImportError: cannot import name scripts2
So I must be doing something wrong here, which module should I be renaming exactly? Sorry if this is a silly question, I am quite new to python.
This is caused because Anaconda has a module named scripts and therefore your import is "shadowed" by that module. You can double check that when you call import scripts in a new notebook it works even if you have never defined such a module. A very good explanation of import traps can be found here:
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html
A workaround would be to rename the script module of pybedtools to something else and also change all the imports to the new name.
I have a python script which uses gentle to transcribe some audio.
At the moment I have to copy my file into the gentle repo and then I can import gentle and use gentle later in the code as I would like.
However I would prefer not to have to copy my file into the gentle directory. I have tried changing the import to import gentle.gentle but either way I get the following error when running my script from the parent directory:
Traceback (most recent call last):
File "process_text.py", line 6, in <module>
import gentle.gentle # or just import gentle
File "/usr/local/lib/python2.7/dist-packages/gentle/__init__.py", line 2, in <module>
from resources import Resources
File "/usr/local/lib/python2.7/dist-packages/gentle/resources.py", line 4, in <module>
from util.paths import get_resource, ENV_VAR
ImportError: No module named util.paths
Is there an easy way to use the module without having to copy my script into its directory? Thank you!
There are many fixes to this issue, see the tutorial page on the docs for more info https://docs.python.org/3/tutorial/modules.html.
Basically, if gentle is pip-installable you may want to try to install it using:
pip install --user gentle
Or you may want to create an environment with either conda or virtualenv.
Alternatively, you could just set your PYTHONPATH environment variable to include the path you need, e.g.
PYTHONPATH="/path/to/lib:$PYTHONPATH" python /path/to/script.py