I’m in the process of writing unit tests for my software.
I wrote some helpers for my tests. For convenience I would like to use them in a jupyter notebook.
When I try to import them inside of the notebook though, I get an error.
from tests import helpers
->
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-12-d8ba72c24738> in <module>
----> 1 from tests.helpers import some_helper
ModuleNotFoundError: No module named 'tests.helpers'
Digging a little, I found out that importing tests actually imports this folder as a module:
'/my_project_path/.venv/lib/python3.8/site-packages/IPython/extensions/tests'
The folder contains test_autoreload.py and test_storemagic.py, which are tests for extensions I use.
Here’s my question, how do I properly manage this conflict? I would like to keep those extension installed, and I would like to keep the name tests for my folder, as it is the convention when working with pytest.
I installed those extensions with pip. Did I miss an option to ignore the tests when installing or something?
Thanks! :)
Not really a solution but some hints here in case they help you:
I was having the same issue today and getting a bit crazy. Not in jupyter but int the ipyton shell. As jupyter uses ipython I guess it will be the same issue.
IPython has a module called tests
within the IPython/extensions folder, and this folder is added is added to the search path when using ipython instead of python.
This are my paths from a python shell (using sys.path):
['',
'/usr/local/lib/python37.zip',
'/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/usr/local/lib/python3.7/site-packages']
And the ones from an ipython shell:
['/usr/local/bin',
'/usr/local/lib/python37.zip',
'/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'',
'/usr/local/lib/python3.7/site-packages',
'/usr/local/lib/python3.7/site-packages/IPython/extensions',
'/home/dmontaner/.ipython']
So, if your are in iptyhon or in a notebook and you import tests or from tests import something, the interpreter will search in:
'/usr/local/lib/python3.7/site-packages/IPython/extensions'
So far all is as expected. But I also had as you my own tests module in the same directory where my ipython session was running. What should happen then is that my own tests module should be imported first. But it was not and I had the same error as you.
If I would remove the 'IPython/extensions' from the path doing sys.path.remove(.../IPython/extensions') then I was able to import my own module.
I did uninstall ipyton and ipython_genutils and, using python -m pytest in the command line (on a .py file, not a notebook) I was still having a slightly similar issue this time about dash trying to load the IPython. The weird thing is that in my project I am not using dash but flask and dash is not a dependency of pytest anyway I think.
So, I uninstalled ipyton, ipython_genutils, dash and plotly and then I could run pytest importing from my own tests module.
I reinstalled the 4 libraries again and the problem was solved! And now I can import from my tests module even from within a jupyter notebook.
I guess the message is that there is some buggy setup or dependency among all those libraries (and may be some others). Try to reinstall latest versions or recreate your virtual environment in case it helps.
Sorry that I could not figure out what was the exact problem... but I hope this helps you.
The long-term fix for these kind of naming conflicts is to put all your code, including your test code, into a project package, such that you imports look like
import my_project.tests.helpers instead of import tests.helpers.
By choosing a unique name for your project, you define your own namespace and avoid naming conflicts.
Related
I am coding in PyCharm and to set up the environment I use poetry, python version 3.10
Everything was working ok, but starting from one moment of time I got problem with module imports.
But the problem does not relate to the import of the whole module but to the import of class or function from module.
You can see as an example that BaseSettings from pydantic is underlined with red wavy line and Pycharm says that BaseSettings is unresolved reference.
I tried
to invalidate cache in Pycharm
install, uninstall the module (poetry add/remove)
change the environment and return back
marked folder as a source root (assume it is irrelevant but anyway)
but unfortunately nothing helped
aiohttp.client.py
aiohttp.init.py
I found the out what was the reason for this strange behaviour. It was the problem of PyCharm IDE. PyCharm stopped recognising init.py as python files that produced import errors. Actually I do not know why this happened since I did not changed any settings or updated PyCharm.
The solution was to manually add pattern "init.py" to python files mask in Settings -> Editor ->File types -> Python (please see screenshot in the attachment)
The Summary
I have a python import that works when run from the VS Code terminal, but that VS Code's editor is giving warnings about. Also, "Go to Definition" doesn't work.
The Problem
I have created a docker container from the image tensorflow/tensorflow:1.15.2-py3, then attach to it using VS Code's "Remote- Containers" extension. Then I've created the following file in the container.
main.py:
import tensorflow.compat.v1 as tf
print(tf.__version__)
This runs fine in the VS Code terminal, but the Editor and the Problems pane both give me an unresolved import 'tensorflow.compat' warning. Also "Go to Definition" doesn't work on tf.__version__.
I'm using several extensions but I believe the relevant ones are the Microsoft Python extension (installed in the container), as well as the Remote - Containers extension, and now the Pylance extension (installed in the container).
The Things I've Tried
I've tried this with the default pylint, and then also after installing pylance with similar results. I've also seen some docs about similar issues, but they were related to setting the correct source folder location for modules that were part of a project. In contrast, my code within my project seems to work fine with imports/go-to-definition. It's external libraries that don't seem to work.
Also, for the sake of this minimal example, I've attached to the container as root, so I am guessing it's not an issue of elevated permissions.
I've also tried disabling all the extensions except the following, but got the same results:
Remote - Containers (local)
Remote - WSL (local)
Python (on container)
Jupyter (on container, required by Python for some reason)
All the extensions above are on the latest versions.
I've also fiddled around with setting python.autocomplete.extraPaths, but I'm not sure what the right path is. It also seems like the wrong thing to have to add libraries to the path that are installed in the global python installation, especially since I'm not using a virtual environment (it being in a docker container and all).
The Question
How do I fix VS Code so that it recognizes this import and I can use "Go to Definition" to explore these tensorflow functions/classes/etc?
tldr;
TensorFlow defines some of its modules in a way that pylint & pylance aren't able to recognize. These errors don't necessarily indicate an incorrect setup.
To Fix:
pylint: The pylint warnings are safely ignored.
Intellisense: The best way I know of at the moment to fix Intellisense is to replace the imports with the modules they are aliasing (found by importing alias in a repl as x then running help(x)). Because the target of the alias in my case is an internal name, you probably don't want to check in these changes to source control. Not ideal.
Details
Regarding the linting: It seems that tensorflow defines its modules in a way that the tools can't understand. Also, it appears that the package is an alias of some kind to another package. For example:
import tensorflow.compat.v1 as tf
tf.estimator.RunConfig()
The above code gives the pylint warning and breaks intellisense. But if you manually import the above in a REPL and run help(tf), it shows you the below package, which you can use instead:
import tensorflow_core._api.v1.compat.v1 as tf
tf.estimator.RunConfig()
This second example does not cause the pylint warning. Also the Intellisense features (Go to definition, Ctrl+Click, etc) work with this second example.
However, based on the _api, it looks like that second package name is an internal namespace, so I'm guessing it is probably best to only use this internal name for local debugging.
Confirmation/Tickets
pylint: I've found a ticket about pylint having issues with a couple tensorflow imports that looks related.
Intellisense: I've opened a ticket with pylance.
So for me I was trying to
import pandas as pd
but I got the error
"pd" is not accessedPylance (module) pd
SO what I did was reload the extension Python IntelliSense(Pylance) and that solved my issue.
I had the same problem but with all kinds of packages.
My solution was to go to the VSCode settings and search for "python.analysis.extraPaths", and add the path to your site-packages.
In my case, I added C:\Code\Python39\Lib\site-packages, and now it's working fine.
What, usually, solves the pylance issues for me is pointing my Python interpreter to the virtualenv one.
Open the command palette Ctrl + Shift + P
Type: Python: Select Interpreter
It will show a list of all the python Interpreters it actually detects:
Select Enter interpreter path
Type in the path to your local venv/bin folder or click find to navigate using the file explorer.
Your path should look something like:
venv/bin/python3.9
i changed "import tensorflow as tf" to "from tensorflow import compat as tf"
it ll even work for tf.gfile.Gfile()
I had a few imports I was using for my project. Some of these included packages I downloaded using pip like
import MySQLdb
While others were modules from within my project like
sys.path.append(os.path.join(os.path.dirname(__file__), "../")) .
from util.myFile import get_mysql_connection, execute_query,
These were all working and I was able to run my application on PyCharm. I closed PyCharm and opened it back up to find most of these imports "could not be found". Any help regarding this will be much appreciated. Thanks
Update:
I'm able to run the file from terminal (iTerm) without errors. However, the file still shows red error marks by the imports. Seems like PyCharm isn't recognizing some sort of path
Check the traceback for what version of python is being called.
There could be a mismatch between the version you've imported modules with and the version that runs by default.
Double check what is being used as your project interpreter. This can change the available packages to your current project.
Found in PyCharm by going to Settings -> Project:<'Your-Project'> -> Project Interpreter
Also, did you install MySQLdb in this way? MySQLdb - ModuleNotFoundError
I'm trying to set up a roguelike Python project, but I can't seem to be able to import libtcod module into my project. This helloworld crashes, and the IDE keeps telling me that there is no module named libtcodpy.
import libtcodpy
def main():
print('Hello World!')
if __name__ == '__main__':
main()
What is the proper way to import modules into Python projects? I'm used to Java, so I was expecting something along the lines of Maven to manage the dependencies. There indeed seems to be something like that in PyCharm as well, this package manager for the venv, which from what I gather serves to isolate the project-specific stuff from the OS- or python-global stuff:
but libtcod simply isn't present in the rather exhaustive list of modules that appears after clicking on the "+" button, just some other module that has something to do with libtcod library (I guess?). Moreover, all the tutorials I found on setting libtcod up advise one to manually copy over files somewhere or run some command that I suppose does the importing somehow and other such solutions, all of which i tried and none of which worked. I don't want to pollute my project structure by using such hodgepodge ways of handling dependencies if I can at all avoid it.
Q: How do I get libtcod to work in my PyCharm project in the most clean and convention-abiding way possible?
Take a look at this github project called tcod: https://github.com/libtcod/python-tcod/blob/master/README.rst#installation
It's a python port of libtcod.
To install using pip, use the following command:
python -m pip install tcod
If you get the error "ImportError: DLL load failed: The specified module could not be found." when trying to import tcod/tdl then you may need the latest Microsoft Visual C runtime.
Blockquote
I'm getting an "unresolved reference" error for imports, when the imports exist. What might cause this?
The details:
I'm developing in Pycharm, Python 3.5 32 bit virtual environment.
I'm getting import errors with the Quantopian Zipline library.
I've imported Zipline in Pycharm Project Interpreter.
I'm getting import errors using the example code in the Zipline library...
from zipline.api import order, record, symbol
What's going on here?
If you look at the stack frame, you see that it is structured as follows:
buyapple.py imports zipline
zipline imports logbook
logbook imports gevent
gevent fails to load
So the module is there, is found, but in the course of importing zipline, zipline itself imports another module, and that module imports yet another module, and that module fails.
Something is wrong with the gevent installation - to be more precise, the compiled binary is missing functions that the Python part of the module calls. One simple step you can do is try reinstalling it (via pip install -U gevent or equivalent) and see whether that works.