I've cloned a codebase from Heroku onto a new comp, when I try to run it none of the Python libraries that I've installed are present. After I run which pip I see that my path is /usr/local/bin/pip.
(1) How do I change the path so all the libraries install into my virtual env and (2) how can I install everything from my requirements.txt instead of individually install libraries.
(venv)admins-MacBook-Air:lhv-talenttracker surajkapoor$ which pip
/usr/local/bin/pip
Try looking at your venv/bin/activate file and see if the VIRTUAL_ENV matches your current path. If it doesn't match, change it to match your path and activate again.
$ cat activate |grep VIRTUAL_ENV=
VIRTUAL_ENV="/does/this/path/match?"
Related
I've tried everything I've seen on SO to get this to work, but so far everything fails. Using macOS Big Sur 11.6, bash in Terminal (not zsh).
I'm trying to create a setup file and execute with sh setup.sh that will setup the env, install python, and then activate it. Nothing fancy. Doing it manually works fine, but once I put it in a shell script, it won't work. I'm running this script from inside an empty project folder.
Current script:
conda create -n MASTER python=3.8.5 -y
conda activate MASTER
Yeah, it's that simple to start with. I commented out the other pip installs until this works properly.
I tried running: bash -i setup.sh but it still does not activate. I get no errors but I'm still stuck in (base).
I tried using source: source /opt/anaconda3/etc/profile.d/conda.sh at beginning of script and/or before activate, still doesn't work. No errors again, but stuck in (base).
I tried using: eval $(conda shell.bash hook) at the start of script and before I try to activate the env, but it fails. This time I get the error:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
But if I run conda init bash (in Terminal or in the script itself), it outputs:
no change /opt/anaconda3/condabin/conda
no change /opt/anaconda3/bin/conda
no change /opt/anaconda3/bin/conda-env
no change /opt/anaconda3/bin/activate
no change /opt/anaconda3/bin/deactivate
no change /opt/anaconda3/etc/profile.d/conda.sh
no change /opt/anaconda3/etc/fish/conf.d/conda.fish
no change /opt/anaconda3/shell/condabin/Conda.psm1
no change /opt/anaconda3/shell/condabin/conda-hook.ps1
no change /opt/anaconda3/lib/python3.8/site-packages/xontrib/conda.xsh
no change /opt/anaconda3/etc/profile.d/conda.csh
no change /Users/liquidRock/.bash_profile
No action taken.
I tried doing /opt/anaconda3/bin/conda activate MASTER which also prompts me to do conda init bash.
Even tried adding #!/bin/bash to the top of the file just in case, but no dice.
Thanks to #fravadona for the simplest of solutions.
Simply executing the script with source instead of sh. 🤦🏻‍♂️
Final setup.sh script (with my preliminary pip installs):
# env & python
conda create -n MASTER python=3.8.5 -y
conda activate MASTER
# pip installs
pip install cmake
pip install --upgrade pip setuptools wheel
pip install opencv-python==4.2.0.32
pip install argparse
pip install datetime
pip install colorama
pip install python-dotenv
pip install python-dotenv[cli]
Executed thusly:
$ source setup.sh
Anaconda creates the env, installs python and dependencies, activates the env, then pip installs the additional dependencies.
Still not sure why it won't work by adding other things to the shell script, but this is still a great, simple solution. And yes, I am a novice with this stuff.
I'm learning Python and have Anaconda installed, and I'm trying to familiarize myself with the process of getting an Eye-Color Detection project working.
I'm running into the following error after going through readme:
Eye-Color-Detection git:(master) âś— sudo pip install -r requirements.txt
WARNING: The directory '/Users/{user}/Library/Caches/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
When trying to update:
(tweet) âžś Eye-Color-Detection git:(master) âś— conda update --all
[Errno 13] Permission denied: '/Users/{user}/opt/anaconda3/envs/tweet/lib/python3.8/site-packages/wrapt/__init__.py' -> '/Users/{user}/opt/anaconda3/envs/tweet/lib/python3.8/site-packages/wrapt/__init__.py.c~'
Q: How might I go about doing this correctly within the same Conda environment?
Most of the time, a sudo pip install is almost never what you really want. While in some cases, it may "appear" to work and solve you're immediate problem. More often than not you've just broken your system python without knowing it.
In the context of that repo, I'd ignore the repo's README and do this.
$ git clone https://github.com/ghimiredhikura/Eye-Color-Detection
$ cd Eye-Color-Detection
Create a virtualenv environment, change yourenvname as you like.
$ conda create -n yourenvname python=3.x
$ conda activate yourenvname
Install the dependencies and run the code
$ pip install -r requirements.txt
$ python3 eye-color.py --input_path=sample/2.jpg --input_type=ima
As fixing you conda environment may be difficult to debug depending on what else you've sudo'd in attempting to resolve your issue. If you happen to be familiar with "regular" virtualenv's created using python's builtin virtual environment tooling, then you could also try this to get you going.
$ python3 -m venv .venv --copies
$ source .venv/bin/active
$ pip install -r requirements.txt
$ python3 eye-color.py --input_path=sample/2.jpg --input_type=image
What you need to do is you have to change the directory premission to writable.
You can do it using this command,
$ sudo chmod 7777 /Users/{user}/library/caches/
to change permissions recursively,
$ sudo chmod 7777 -R /Users/{user}/library/caches/
or you can own that directory by using this command,
$ sudo chown OWNER:GROUP /Users/{user}/library/caches/
where OWNER is the username for your computer which you can find in the terminal by using this command.
$ whoami
GROUP is optional.
I have two machine that one does not have internet access. I want to install modules with anaconda and copy them to offline computer from the other computer that has internet access.
I tried looking for dependencies and install tar. files manually one by one and sent them to the offline machine but it is very time-consuming.
What is the easiest way? Does miniconda helpful ??
P.S: I forgot to mention that I am using anaconda in both machines. So I guess I need to create an env., install packages then export it for offline computer. Are there any other way to install number of packages to offline comp. from a copy <dir> in the online computer ??
Edit: I tried conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file://C:\Users\myName\Desktop\OfflineInstall\pkgs2 but offline machine still tried to connect internet. I also used --no-deps
Edit2: For those who stuck on the same problem, I solved using conda install --file C:\Users\myName\Desktop\OfflineInstall\packagelist.txt --channel file:///C:\Users\myName\Desktop\OfflineInstall\pkgs2 --override-channels The tricky way is the file:/// prefix. You need to put ///. Also remember to put --override-channels flag to prevent connection to default channels.
It sounds like Conda-pack is what you are looking for.
Installing:
$ conda install conda-pack
On the source machine:
# Pack environment my_env into my_env.tar.gz
$ conda pack -n my_env
On the target machine:
# Unpack environment into directory `my_env`
$ mkdir -p my_env
$ tar -xzf my_env.tar.gz -C my_env
# Use python without activating or fixing the prefixes. Most python
# libraries will work fine, but things that require prefix cleanups
# will fail.
$ ./my_env/bin/python
# Activate the environment. This adds `my_env/bin` to your path
$ source my_env/bin/activate
# Run python from in the environment
(my_env) $ python
# Cleanup prefixes from in the active environment.
# Note that this command can also be run without activating the environment
# as long as some version of python is already installed on the machine.
(my_env) $ conda-unpack
The caveat being that conda-pack will take the whole environment.
Had this problem the other day, very simple implementation.
First make a .txt file which contains all your python libraries. Now you can just pass this .txt file to whatever machine you want the solution to be installed under and issue the following command :
pip install -r packages.txt
Where "packages" is the name of your .txt file. Hope this helps!
Edit using Conda :
while read requirement; do conda install --yes $requirement; done < requirements.txt
I would like to use virtualenvwrapper with Python 3.6, however, I am working on a Mac which defaults to using Python 2.7 and I am having issues. Here's what I have done so far. Using the following commands, I have found the locations of where each Python version is saved:
>> which python
>> /usr/bin/python
>> which python3
>> /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
I have successfully installed virtualenv and virtualenvwrapper using:
>> pip3 install virtualenv
>> pip3 install virtualenvwrapper
I then search for the location of virtualenv and virtualenv wrapper to confirm their locations:
>> pip3 show virtualenv
>> Location: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
>> pip3 show virtualenvwrapper
>> Location: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
If I go to this site-packages directory there is the following:
virtualenvwrapper
|--- __pycache__
|--- hook_loader.py
|--- project.py
|--- user_scripts.py
virtualenvwrapper-4.8.2-py2.7-nspkg.pth
virtualenvwrapper-4.8.2.dist-info
According to virtualenvwrapper's documentation, I should add the following to my shell startup file to ensure initialization, changing virtualenvwrapper's path to the one set on my machine:
export WORKON_HOME=~/Envs
$ mkdir -p $WORKON_HOME
$ source /usr/local/bin/virtualenvwrapper.sh
I then searched for where virtualenvwrapper.sh is actually located (for some reason, it is not located where the virtualenvwrapper module is installed):
>> which virtualenvwrapper.sh
>> /Library/Frameworks/Python.framework/Versions/3.6/bin/virtualenvwrapper.sh
I changed my .bashrc file using:
>> nano ~./bashrc
Copied in the following:
# script for virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /Library/Frameworks/Python.framework/Versions/3.6/bin/virtualenvwrapper.sh
I then opened up a new terminal and ran:
>> source `which virtualenvwrapper.sh`
Which gave me:
/usr/bin/python3: No such file or directory
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3 and that PATH is
set properly.
What I don't understand is why virtualenvwrapper needs to be located at /usr/bin/python3 which is not a directory. Since it is in bin, it is also impossible for me to make it a directory. Additionally, the virtualenvwrapper.hook_loader is not where virtualenvwrapper.sh is which I think may be causing an issue.
I also tried creating a symbolic link to /usr/local/bin because some people stated that it solved their issue but this didn't solve mine:
sudo ln /Library/Frameworks/Python.framework/Versions/3.6/bin/virtualenvwrapper.sh /usr/local/bin/virtualenvwrapper.sh
Another issue altogether is whether or not I need to install it using pip3 if I want to use virtualenvwrapper for Python3. For example, this post states that you only need to pass a flag, however, you should also be wary of installing it on your base Python installation.
Add
export VIRTUALENVWRAPPER_PYTHON=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
to .bashrc and execute the command in the terminal. That sets the path to Python interpreter for virtualenvwrapper.
I am attempting to install Portia, a python app from Github: https://github.com/scrapinghub/portia
I use the following steps at the command line:
set up new virtualenv 'portia' in Mac terminal
git clone https://github.com/scrapinghub/portia.git
follow readme instructions:
cd slyd
pip install -r requirements.txt
run Portia
cd slyd
twistd -n slyd
But every time I attempt the last step to run the program, I get the following error:
ImportError: No module named scrapy
Any idea why this error is occurring? All previous steps seem to install correctly. Is it an error earlier in my install process?
Thanks!
I don't have the rep to upvote Alagappan's answer but he's correct. Also, if you're as inexperienced as I am, you may need further clarity on this.
You have to create, activate and navigate into the virtualenv before installing anything (including cloning portia from github). Here's the whole thing working from start to finish:
1: cd to wherever you’d like to store your project...
and Install virtualenv:
$ pip install virtualenv
2: Create the virtual environment. (I called mine “portia” but this can be anything.):
$ virtualenv portia
3: Activate the virtual environment you created (change the path to reflect the name you used here if not “portia”.):
$ source portia/bin/activate
At this point your terminal should have display the virtualenv name in parenthesis before the standard directory path prompt:
 (name-of-virtualenv) [your-machine]:[current-directory]: [user]$
...and if you list the files within your pwd you’ll see the name of you virtualenv there.
4: cd into your virtualenv (“portia” for me):
$ cd portia
5: Now you can clone portia from github into your virtualenv...
$ git clone https://github.com/scrapinghub/portia
6: cd into the cloned portia/slyd...
$ cd portia/slyd
7/8: pip install twisted and Scrapy...
$ pip install twisted
$ pip install Scrapy
You’re virtualenv should still be activated and you should still be in [virtualenv-name]/portia/slyd
9: Install the requirements.txt:
$ pip install -r requirements.txt
10: Run slyd:
$ twistd -n slyd
--- No more scrapy error! ---
Another Installation Method For Portia: Using Vagrant
Here is the method that made me install Portia with ease. Works with Mac, Windows and Linux. With a few commands and clicks, you'll get a fully functional web scraper.
Things Needed:
VirtualBox
Vagrant
Clone the repo for Portia or download the zip file.
Additional Steps To Take:
Install VirtualBox.
Install Vagrant
Open your terminal and navigate to where you cloned the Portia repo or where you've extracted it (in case of a zip file).
Then make a command vagrant up - This will download and setup a VirtualBox Guest VM for you + will install all the necessary requirements for Portia and will install Portia from start to finished.
After the above process, you may now open your browser and navigate to
http://the-virtualbox-ip:8000/static/main.html
And you're setup.
It's quite simple, you just need to install the python module scrapy in the same way that the Twitter API requires setuptools
pip install scrapy
I suppose the issue you are facing is because of the virtualenv. Once you setup a new virtual environment you need to run the activate script in order to start using it. In your case you'll have to run the following command:
$ source portia/bin/activate
On successful activation, your prompt will look like:
(portia) $
Can you check if you activated your virtual environment before you installed the packages using pip? I believe doing so will fix your issue.