query what libraries are on the host of the python notebook - python

In a jupyter notebook, I can fairly easily check with python code if some libraries are installed on the current kernel of the python notebook.
However there is also the "host" kernel which has its own python env (ie the python process that was launched when jupyter notebook was called). Depending on what libraries/extensions were installed on the host, it may not be possible to do specific things on the jupyter notebook client itself.
Is there a way to query what libraries/modules/extensions are installed on the host, from the "client" notebook ? thanks

Did you test !pip list or !pip freeze?
To list locally installed packages and their version # within a pipenv environment, cd into a pipenv project and enter the following command:
pipenv lock -r

I guess you can use !pip list to show the modules/libraries installed on the current env. But I don't think you can view extensions.

Please use !pip list --local or !pip freeze --local if you're using a virtual environment.

This could get you started:
import sys
import os
for i in sys.path:
try:
print("->", i, ":")
print(repr(os.listdir(i or ".")))
except Exception:
print("can't list", i)
pass
NOTE that my code isn't a complete solution, but it is a start. I will attempt to improve it shortly, but even as it is it may be immediately useful for you to develop your own solution.
What it doesn't do:
it does not determine whether it found a python file or a directory containing a valid package or something that isn't either (i.e. is not 'import'-able)
It skips over zipped packages (such as this item that could be in sys.path: /home/pyodide/lib/python39.zip)
it does not attempt to find built-in packages (those that are compiled into Python itself). Most of those would be already loaded, so you will find them in sys.modules, though
If you need to test for individual packages, rather than get a whole list: you may be better off using 'import' inside a try/except statement:
try:
import something
except ImportError:
pass
# do what you want if the package isn't available

If I understand correctly, you can run:
help("modules")
Or you can use pydoc for your work:
import pydoc
!pydoc modules

I believe !pip list should work- if not, then pydoc most likely would, see above.

Related

So i installed numpy . But when i call it in a program an error occurs. Any method to solve it permanently in windows 10

Here is the error
import numpy
Exception has occurred: ModuleNotFoundError
No module named 'numpy'
File "C:\path\to\file\32.py", line 1, in <module>
import numpy
Let me know how did you install the NumPy package; using pip or something else?
If you have multiple python versions, i.e. 2.x and 3.x at the same time, please make sure your interpreter for the 32.py file is the version that you installed NumPy on.
To possibly fix your problem, you should first try installing it and see if there are any errors. You should also check the version of Python you are running on Windows 10, because when you update Python it sometimes switches names between py and python
As you can see, the version of Python has changed between py and python so you should try changing that first.
If this does not work, you should try finding the directory for NumPy and adding it to the system PATH in your script. The installer usually shows you the location by doing the following:
import sys
sys.path.append("<insert numpy location here>")
import NumPy
This should manually force it into finding the package. If none of this works, please tell us and we should be able to find a different solution.
Happy Coding!
If you're using a code editor like PyCharm, you could install it by clicking on
file then settings then the project interpreter setting and install new module! You can search for the module and install.
Make sure that the python version that you want to use is a Windows Environmental Variable. You can test this by running this line in your command line.
> python --version
If you get some other python version that is not the one that you wish to use you can set the python version you want by finding where exactly your Python folder is located and go into settings and set the path as a new variable (I can leave a tutorial for you). If that is too much of a hassle, the Python installers can set the python that you will install as an environmental variable for you. (You could uninstall and reinstall and make sure that you allow it to make it an environmental variable.
After that you should be able to import whatever external packages you want using pip
for example:
pip install numpy

Jupyter-notebook returns "module not found" for every single import, despite them being downloaded

For example, import trackpy returns the module not found error.I have already confirmed that trackpy has been downloaded somewhere on my computer, because attempting to install it again via conda install -c soft-matter trackpy will eventually return something to the effect of "all files already installed". This seems to occur for every "external import" (numpy, scipy, matplotlib), i.e. one that was downloaded somewhere from the internet. This does not happen for "internal imports" (sys, os). I believe this is just a matter of jupyter not looking for the files in the correct place, but I don't know how to fix something like this.
Edit: Relevant info: I ran
import sys
sys.executable
which returns 'c:\\users\\reese\\miniconda3\\python.exe'. In the pkgs folder for miniconda3, there are none of the imports that I want. However in 'c:\\users\\reese\\Anaconda\\pkgs' are all the imports, trackpy and all else. Is there an easy way to make jupyter check here for imports? I already tried straight up copying the entire pkgs folder and pasting it in miniconda3's pkgs folder, but it did not work.
Two solutions I would propose.
Okay Solution:
Yes, you can add the path to your other packages with sys.path:
import sys
sys.path.insert(0,'PATH_TO_YOUR_OTHER_PACKAGES')
import Packages_of_another_path
By insert it at index zero, you ensure that your other packages get first priority in case there in another package with the same name.
Better Solution:(Recommendable)
Always use environments. E.g.
conda create —name your_env python=3.6 pip
conda activate your_env
conda install packages1 packages2
pip install package3
In this environment you can keep all you things together.
Everything you wish to use your packages, activate your environment and start hacking ;)

Need help determining correct filepaths for pip, kernels etc

I'm having a 'module not found' issue, and my problem research has pointed me toward possible filepath name issues. I need some help verifying that my filepaths are correctly aligned.
When I first set up Jupyter Notebook and Python on my computer, and I didn't understand virtualenv at all, I made a lot of errors which were later discovered when coming across the 'module not found' issue.
I thought I had gotten this sorted at a few months back, as I haven't had any problems since and have used pip multiple times since for loading new libraries.
THE ERROR:
import tweepy
ModuleNotFoundError: No module named 'tweepy'
CHECKING IN PYTHON NOTEBOOK:
import sys
print(sys.executable)
/Users/Moos/anaconda3/envs/python3/bin/python
print(sys.path)
['',
'/Users/Moos/anaconda3/envs/python3/lib/python36.zip',
'/Users/Moos/anaconda3/envs/python3/lib/python3.6',
'/Users/Moos/anaconda3/envs/python3/lib/python3.6/lib-dynload',
'/Users/Moos/anaconda3/envs/python3/lib/python3.6/site-packages',
'/Users/Moos/anaconda3/envs/python3/lib/python3.6/site-
packages/aeosa',
'/Users/Moos/anaconda3/envs/python3/lib/python3.6/site-
packages/IPython/extensions','/Users/Moos/.ipython'
]
IN TERMINAL:
which pip
/Users/Moos/anaconda3/bin/pip
which pip3
/Users/Moos/anaconda3/bin/pip3
jupyter kernelspec list
Available kernels:
python2 /usr/local/share/jupyter/kernels/python2
python3 /usr/local/share/jupyter/kernels/python3
Is there an issue with the kernelspec list? Should they be referencing the anaconda3 pathway?
Repeat: I do not have this problem with any other library.
Maybe a trivial suggestion but have you checked if the "tweepy" is actually available in your current environment?
To check this you can type the following in anaconda prompt: "conda list tweepy", otherwise you might use the anaconda navigator to perform the same check visually.

ImportError: cannot import name 'moduleTNC' - python

I'm having a problem in python when I try to import linear_model from sklearn library: from sklearn import linear_model. I have just installed it via pip simply in this way: pip install sklearn. I know that to avoid this error suffices uninstall and reinstall sklearn, but it didn't work. I also installed it via conda but opening the idle (is that correct?) it gives the same error.
How to avoid it?
NB: If I use jupyter from conda it works well as it should.
I had this same problem and resolved it with:
conda remove scipy scikit-learn -y
conda install scipy scikit-learn -y
I saw it here, where many other people said it solved their problems too.
With regards to the following error:
ImportError: cannot import name 'moduleTNC'
It can be solved by renaming moduletnc.cp36-win_amd64.pyd with moduleTNC.cp36-win_amd64.pyd in:
AppData\Roaming\Python\Python36\site-packages\scipy\optimize
I can't mark as possible duplicate, so I'm just pasting here. If that's a wrong behaviour, I'm sorry:
Import module works in Jupyter notebook but not in IDLE
The reason is that your pip/conda installed library paths are not accessible by python IDLE. You have to add those library paths to your environment variable(PATH). To do this open my computer > properties > advanced system settings > system.
Under environment variables look for PATH and at the end add the location of installed libraries. Refer this for more details on how to add locations in path variable. Once you do these you will be able to import the libraries. In order to know what locations python searches for libraries you can use
import sys
print sys.path
This will give you a list of locations where python searches for libraries. Once you edit the PATH variable those locations will be reflected here.
Refer this also in order to know how to add python library path.
Note: The tutorial is a reference on how to edit PATH variable. I encourage you to find the location of installed libraries and follow the steps to edit the same.

How to install a missing python package from inside the script that needs it?

Assuming that you already have pip or easy_install installed on your python distribution, I would like to know how can I installed a required package in the user directory from within the script itself.
From what I know pip is also a python module so the solution should look like:
try:
import zumba
except ImportError:
import pip
# ... do "pip install --user zumba" or throw exception <-- how?
import zumba
What I am missing is doing "pip install --user zumba" from inside python, I don't want to do it using os.system() as this may create other problems.
I assume it is possible...
Updated for newer pip version (>= 10.0):
try:
import zumba
except ImportError:
from pip._internal import main as pip
pip(['install', '--user', 'zumba'])
import zumba
Thanks to #Joop I was able to come-up with the proper answer.
try:
import zumba
except ImportError:
import pip
pip.main(['install', '--user', 'zumba'])
import zumba
One important remark is that this will work without requiring root access as it will install the module in user directory.
Not sure if it will work for binary modules or ones that would require compilation, but it clearly works well for pure-python modules.
Now you can write self contained scripts and not worry about dependencies.
As of pip version >= 10.0.0, the above solutions will not work because of internal package restructuring. The new way to use pip inside a script is now as follows:
try: import abc
except ImportError:
from pip._internal import main as pip
pip(['install', '--user', 'abc'])
import abc
I wanted to note that the current accepted answer could result in a possible app name collision. Importing from the app namespace doesn't give you the full picture of what's installed on the system.
A better way would be:
import pip
packages = [package.project_name for package in pip.get_installed_distributions()]
if 'package' not in packages:
pip.main(['install', 'package'])
Do not use pip.main or pip._internal.main.
Quoting directly from the official documentation (boldface emphasis and editing comments mine, italics theirs):
As noted previously, pip is a command line program. While it is... available from your Python code via import pip, you must not use pip’s internal APIs in this way. There are a number of reasons for this:
The pip code assumes that [it] is in sole control of the global state of the program. pip manages things like... without considering the possibility that user code might be affected.
pip’s code is not thread safe. If you were to run pip in a thread, there is no guarantee that either your code or pip’s would work as you expect.
pip assumes that once it has finished its work, the process will terminate... calling pip twice in the same process is likely to have issues.
This does not mean that the pip developers are opposed in principle to the idea that pip could be used as a library - it’s just that this isn’t how it was written, and it would be a lot of work to redesign the internals for use as a library [with a] stable API... And we simply don’t currently have the resources....
...[E]verything inside of pip is considered an implementation detail. Even the fact that the import name is pip is subject to change without notice. While we do try not to break things as much as possible, all the internal APIs can change at any time, for any reason....
...[I]nstalling packages into sys.path in a running Python process is something that should only be done with care. The import system caches certain data, and installing new packages while a program is running may not always behave as expected....
Having said all of the above[:] The most reliable approach, and the one that is fully supported, is to run pip in a subprocess. This is easily done using the standard subprocess module:
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'my_package'])
It goes on to describe other more appropriate tools and approaches for related problems.

Categories

Resources