Question:
What's the best way to set a Jupyterlab's notebook to have the same sys.path as Python when it's run from the Mac terminal?
Background:
I can use a python module (Beautiful Soup) successfully by starting Python from the Mac terminal (within a virtual environment named VENV). It installs, imports and works fine. But the Jupyterlabs notebook using the same VENV environment, won't import the very same module. I confirmed they are both using the same Python instance using the "Which Python" command, but terminal and notebook have a different sys.path, which appears to be restricting what the notebook can see.
What's the best way to handle this path inconsistency?
I found a workaround to make it work.
Thanks to Peter Bakke.
https://www.peterbakke.com/data-analysis/set-system-path-python-jupyter-notebooks/
I also installed BeautifulSoup4, rather than bs4.
Related
I installed python virtual env. I use vs code. I imported scrapy in my code and vs code doesn't recognize the module. Actually, it works well when I run it. scrapy crawl tester
Just, vs code shows red underlines that mean "Unable to import scrapy" So this is just vs code issue, not venv or scrapy package install issue.
This code works well and actuallay, scrapy is imported without any issue. This is just vs code issue. Thank you.
You might be able to solve your problem by using CTRL+Shift+P to add "Python: Select Interpreter" to your project.
If you created a virtual environment and activated it as well
type which python on macOS/Linux,
type where python on Windows,
inside the activated terminal session.
(env) userpc#pc:~$ which python
/home/userpc/Desktop/foldername/env/bin/python
In VSCode press Ctrl+Shift+P, under >Select Interpreter paste the location you receive using the former command.
Once done, restart VSCode.
VSCode will also ask you to install pylint in your environment and if it doesn't you can do the same by activating you environment and typing
pip install pylint
inside the activated terminal session.
Ran into the same problem - selected the correct Python interpreter in VS Code, pip installed all the desired Python libraries but import is still underlined in VS Code.
What you need to do
What worked for me is to make sure that the linter that you are using in this VS Code instance (I was using pylint) is from the bin folder of the virtual environment, not somewhere else.
How you can do it
I'll use the absolute path to the desired virtual environment /User/ProjectFolder/env as an example.
To check that you meet the conditions stated in What you need to do, toggle the settings.json file in VS Code by pressing ⌘ + ,:
Make sure that the correct Python interpreter is selected. i.e. "python.pythonPath":/User/ProjectFolder/env/bin/python3.
Make sure that the linter (e.g. pylint) is located in that bin folder, not anywhere else. i.e "python.linting.pylintPath":/User/ProjectFolder/env/bin/pylint, NOT something like "python.linting.pylintPath":/usr/local/bin/pylint.
This means that you have to install your desired linter in the virtual environment.
Hope this helps.
I tried #cleon-w's answer which worked for me. (Thanks Cleon)
I was using pyenv on my Mac M1 (Big Sur) with vscode. vscode could not resolve the
paths to python that the pyenv provides, so I looked within the pyenv scripts to find the absolute paths
to the underlying python and the pylint files.
As #Cleon W says I ensured pylint and python are in the same bin directory.
Then edit VS Code settings.json to point to them directly (bypass .pyenv) and the imports could be found.
settings.json (vscode)
"python.defaultInterpreterPath": "/Users/USERNAME/.pyenv/shims/python",
"python.pythonPath": "/Users/USERNAME/.pyenv/versions/3.9.1/bin/python",
I have similar problem with Django. What solved my problem was:
create .vscode/settings.json file inside root dir for project.
add this json {"python.defaultInterpreterPath": "path to bin directory in root dir for project"}
Ctrl-Shif-P -> Python: Select Interpreter -> select one that said: Use Python from python.defaultInterpreterPath. If it's not in the list, hit refresh (circled refresh button on the top of settings box).
For my current job it would be extremely helpful to be able to configure a virtualenv with the appropiate libraries versions, and be able to run either a python project, or cells in jupyter. This is because some people at my job work with jupyter, and some with Python, and sometimes both, and this way I would have a centralized program that could run both types, which I have not found outsaid of the paid version of PyCharm, which my company does not provide.
I just learned a few days ago about Windows Subsystem for Linux, WSL, and that it can be launched from withing Visual Studio Code, so I feel like this is my best bet to achieve that dual nature of programming from just ONE program, instead of running several like in the past.
As of right now, I have a repository cloned with WSL for a git project with different ".py" files, I open it with VSC, then open the terminal inside VSC, and I can both edit the Python code, and run it on the terminal, using bash commands as I would if I were in Ubuntu (I am doing all this from windows but can switch to Ubuntu if it would mean to be able to do this type of setup).
When I run with "Run Python file in terminal", it uses the virtualenv I have previously created.
The problem is, with Jupyter, it does not detect I have the libraries installed (like Pandas for example)
Description of my process with Jupyter so far: With the WSL console, I launch a jupyter notebook &. I then connect to that server, usin the VSC option for "Specify Local or Remote Jupyter server for connections", use the "Existing" option, copy the URL, then I go to the ".ipynb" file and start running code.
If, in a Jupyter cell, I do
import os
os.environ['VIRTUAL_ENV']
I can see my virtual enviroment. If right after that, I run import pandas I get ModuleNotFoundError: No module named 'pandas'
If I do !pip freeze I can see all the libraries and right versions that I have installed in that enviroment.
I feel like I am almost there but something is missing. My guess is that import might be going to some default installed Python, and not the one from the enviroment, for some reason I am missing.
Solved by doing what this answer suggests. The rest of answers may be of help, too:
https://stackoverflow.com/a/51036073/6028947
I have downloaded the rpy2 package using MacPorts, but every time I try to import it using my IDE (Enthought Canopy version 1.5.4), I get the following error:
ImportError: No module named rpy2
However, when I run Python in my terminal and run import rpy2 there, it imports without any issue.
I did some googling on the issue, and it appears that Python's environmental variables in the terminal are not the same as the Python's environmental variables in the IDE, particularly the PYTHONPATH. I checked my PYTHONPATH in my terminal by running import sys; sys.path and got the following file paths:
/usr/local/lib/python2.7/site-packages
/Users/bob
/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/Users/bob/Library/Python/2.7/lib/python/site-packages
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/rpy2-2.8.6-py2.7-macosx-10.6-intel.egg
The very last file path points to the location of the module that won't be imported in my IDE.
I then ran the same commands on a script in my IDE and got the following file paths:
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python27.zip
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/plat-darwin
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/plat-mac
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/plat-mac/lib-scriptpackages
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/lib-tk
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/lib-old
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/lib-dynload
/Users/bob/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages
/Users/bob/Library/Python/2.7/lib/python/site-packages
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages
/Applications/Canopy.app/appdata/canopy-1.5.4.3105.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/IPython/extensions
/Users/bob/.ipython
The location of rpy2 module is not present in the above block of file paths. So, it seemed logical to me to simply add its file path to the PYTHONPATH environmental variable.
However, an answer to this post (Using MacPorts to install modules via a certain path) mentions that it is potentially dangerous for the following command to be carried out:
sys.path.insert(-1, '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/rpy2-2.8.6-py2.7-macosx-10.6-intel.egg')
Can anybody support/gainsay this claim? Any additional explanation would be most helpful.
UPDATE
So, it turns out, it actually is BAD for the script. I decided to try running some examples from the tutorials, and the kernel just crashes every time, and must be restarted. How can I remedy this?
SECOND UPDATE
After following the advice set out in How to: Macports select python , I decided to run the following command in the terminal in order for MacPorts to select the version of Python that was being used by my Canopy IDE:
sudo port select --set python python27
This seems to have solved the issue, as my kernel does not crash anymore when I run basic commands from the rpy2 module. But I remain skeptical as to whether this is a long term solution or not, as I am still confused behind the discrepancies of installing a module via MacPorts versus the IDE's package manager.
I am using pycharm with a virtual environment set in preferences. That environment has installed python 2.7.13. I have installed anaconda in the virtual environment, which contains jupyter. I had the professional version but have allowed the subscription to lapse.
According to https://www.jetbrains.com/help/pycharm/2016.3/using-ipython-jupyter-notebook-with-pycharm.html I can create a notebook by opening the project window and using command-N, When I do this, I CAN select jupiter notebook but there is no way to enter the filename that they require.
Trying it another way, I use File->New->Python File and, for the name use test.ipynb. Pycharm accepts it but puts on a '.py' suffix, treating it as a normal python code file: test.ipynb.py.
Any ideas on how I get this to work?
I think u should try File->New->Jupyter Notebook, this works for me.
Having some weird troubles installing python modules on my work computer (read: no admin/root rights), I'm using 2.7.5. I downloaded and unpacked the tarball and ran 'setup.py', but it had no effect: When I open the python shell, it can't find the module (this specific one is fuzzywuzzy). However, if I right click -> edit with IDLE the setup.py, and then run the shell from that file, it loads and works perfectly fine. Or, if I then open a new file from that shell, use the module and run it, it works fine. -__-
I've tried using:
import sys
sys.path.append('path here')
to append the location where the module is installed, but this doesn't help, nor does the path stay in the sys.path list when I close/reopen the shell.
This is actually driving me insane. Can anyone help? I'm relatively new to programming and python.
The best and easy way provided by python to install/uninstall packages is to use PIP.
use this
python -m pip install packagename==version
same way to uninstall
python -m pip uninstall packagename==version
if you are using windows you need to set path variable first usually python file will be in path C:\Python27 to set path variable
PATH=%PATH%;C:\Python27;