When I create a virtualenv, it installs setuptools and pip. Is it possible to add new packages to this list?
Example use cases:
Following this solution to use ipython in virtualenv (from this question) requires installing ipython in every virtualenv (unless I allow system-site-packages).
Or if I'm doing a only flask/pygame/framework development, I'd want it in every virtualenv.
I took a different approach from what is chosen as the correct answer.
I chose I directory, like ~/.virtualenv/deps and installed packages in there by doing
pip install -U --target ~/.virtualenv/deps ...
Next in ~/.virtualenv/postmkvirtualenv I put the following:
# find directory
SITEDIR=$(virtualenvwrapper_get_site_packages_dir)
PYVER=$(virtualenvwrapper_get_python_version)
# create new .pth file with our path depending of python version
if [[ $PYVER == 3* ]];
then
echo "$HOME/.virtualenvs/deps3/" > "$SITEDIR/extra.pth";
else
echo "$HOME/.virtualenvs/deps/" > "$SITEDIR/extra.pth";
fi
Post that basically says the same thing.
You can write a python script, say personalize_venv.py that extends the EnvBuilder class and override its post_setup() method for installing any default packages that you need.
You can get the basic example from https://docs.python.org/3/library/venv.html#an-example-of-extending-envbuilder.
This doesn't need a hook. Directly run the script with command line argument dirs pointing to your venv directory/directories. The hook is the post_setup() method itself of EnvBuilder class.
Related
I use it on my windows machine by downloading its binary. I also use it in Heroku from its herokus build pack. I don't know what operating system replit use. But I try every possible commed like.
!pip install ta-lib
!pip install talib-binary
It's not working with replit. I thought it work like google co-lab but its not the same.
can anyone use TA-LIB with replit. if so. How you install it?
Getting TA-Lib work on Replit
(by installing it from sources)
Create a new replit with Nix toolset with a Python template.
In main.py write:
import talib
print (talib.__ta_version__)
This will be our test case. If ta-lib is installed the python main.py (executed in Shell) will return something like:
$ python main.py
b'0.6.0-dev (Jan 1 1980 00:00:00)'
We need to prepare a tools for building TA-Lib sources. There is a replit.nix file in your project's root folder (in my case it was ~/BrownDutifulLinux). Every time you execute a command like cmake the Nix reports that:
cmake: command not installed. Multiple versions of this command were found in Nix.
Select one to run (or press Ctrl-C to cancel):
cmake.out
cmakeCurses.out
cmakeWithGui.out
cmakeMinimal.out
cmake_2_8.out
If you select cmake.out it will add a record about it into the replit.nix file. And next time you call cmake, it will know which cmake version to launch. Perhaps you may manually edit replit.nix file... But if you're going to add such commands in a my way, note that you must execute them in Shell in your project root folder as replit.nix file is located in it. Otherwise Nix won't remember your choice.
After all my replit.nix file (you may see its content with cat replit.nix) content was:
{ pkgs }: {
deps = [
pkgs.libtool
pkgs.automake
pkgs.autoconf
pkgs.cmake
pkgs.python38Full
];
env = {
PYTHON_LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath [
# Needed for pandas / numpy
pkgs.stdenv.cc.cc.lib
pkgs.zlib
# Needed for pygame
pkgs.glib
# Needed for matplotlib
pkgs.xorg.libX11
];
PYTHONBIN = "${pkgs.python38Full}/bin/python3.8";
LANG = "en_US.UTF-8";
};
}
Which means I executed libtool, autoconf, automake and cmake in Shell. I always choose a generic suggestion from Nix, without a specific version. Note: some commands may report errors as we executing them in a wrong way just to add to a replit.nix.
3.
Once build tools are set up we need to get and build TA-Lib C library sources. To do that execute in Shell:
git clone https://github.com/TA-Lib/ta-lib.git
then
cd ta-lib/
libtoolize
autoreconf --install
./configure
If configure script is completed without any problems, build the library with:
make -j4
It will end up with some compilation errors, but they are related to some additional tools which are used to add new TA-Lib indicators and build at the end, but not the library itself. The library will be successfully compiled and you should be able to see it with:
$ ls ./src/.libs/
libta_lib.a libta_lib.lai libta_lib.so.0
libta_lib.la libta_lib.so libta_lib.so.0.0.0
Now we have our C library built, but we can't install it to a system default folders. So we have to use the library as is from the folders where it was build. All we need is just one more additional preparation:
mkdir ./include/ta-lib
cp ./include/*.h ./include/ta-lib/
This will copy a library headers to a subfolder, as they are designed to be used from a such subfolder (which they don't have due to impossibility to perform the installation step).
4.
Now we have TA-Lib C library built and prepared to be used locally from its build folders. All we need after that - is to compile the Python wrapper for it. But Python wrapper will look for a library only in system default folders, so we need to instruct it where our library is.
To do this, execute pwd and remember the absolute path to your project's root folder. In my case it was:
/home/runner/FormalPleasedOffice
Then adjust the paths (there are two) in a following command to lead to your project path:
TA_INCLUDE_PATH=/home/runner/FormalPleasedOffice/ta-lib/include/ TA_LIBRARY_PATH=/home/runner/FormalPleasedOffice/ta-lib/src/.libs/ pip install ta-lib
This is one line command, not a two commands.If the paths would be shorter it would look like:
TA_INCLUDE_PATH=/path1/ TA_LIBRARY_PATH=/path2/ pip install ta-lib.
After execution of this command the wrapper will be installed with two additional paths where it will look for a library and its header files.
That's actually all.
An alternative way would be to clone the wrapper sources, edit its setup.py and install wrapper manually. Just for the record this would be:
cd ~/Your_project
git clone https://github.com/mrjbq7/ta-lib.git ta-lib-wrapper
cd ta-lib-wrapper
Here edit the setup.py. Find the lines include_dirs = [ and library_dirs = [ and append your paths to these lists. Then you just need to:
python setup.py build
pip install .
Note the dot at the end.
5.
Go to the project's folder and try our python script:
$python main.py
b'0.6.0-dev (Jan 1 1980 00:00:00)'
Bingo!
The #truf answer is correct.
after you add the
pkgs.libtool
pkgs.automake
pkgs.autoconf
pkgs.cmake
in the replit.nix dippendancies.
git clone https://github.com/TA-Lib/ta-lib.git
cd ta-lib/
libtoolize
autoreconf --install
./configure
make -j4
mkdir ./include/ta-lib
cp ./include/*.h ./include/ta-lib/
TA_INCLUDE_PATH=/home/runner/FormalPleasedOffice/ta-lib/include/ TA_LIBRARY_PATH=/home/runner/FormalPleasedOffice/ta-lib/src/.libs/ pip install ta-lib
Note : FormalPleasedOffice should be your project name
Done.
Here is the youtube video :
https://www.youtube.com/watch?v=u20y-nUMo5I
As the title says, I want to either run multiple conda environements from 1 flask app such that certain pages use 1 version of packages and the others use a different version of packages.
Alternatively, I could do something where I run 2 apps concurrently and then would need to be able to properly redirect from one to another.
I scoured the internet and didn't find anything. Any ideas/documentation on where to get started?
EDIT I was told this was a bad idea and to elaborate on the problem rather than my attempted solution
The problem is that I have certain packages that I am trying to interact with 2 different ML models that were done in different versions of scikit. I can't recreate the model because it was given to me by a coworker. Additionally I am doing some name matching using fuzzywuzzy which is causing issues with other packages I need.
You can do what you are asking by installing both versions to different locations (so they don't overwrite each other), and then renaming the package as this seems to be your only option.
Take the following example, I am going to setup 2 virtual environments, in the first I'll install scitkit-learn 0.22.2 and in the second I'll install 0.20.4, then move the name of the package so python can differentiate them and print the version ($ denotes something to enter on the command line):
$ python3 -m venv sk1
$ source sk1/bin/activate
$ pip3 install scikit-learn==0.22.2 # install to venv 1
$ deactivate # leave
$ python3 -m venv sk2
$ source sk2/bin/activate
$ pip3 install scikit-learn==0.20.4 # install to venv 2
$ deactivate
# move the package names
$ mv ./sk1/lib/python3.7/site-packages/sklearn ./sk1/lib/python3.7/site-packages/sklearn0222
$ mv ./sk2/lib/python3.7/site-packages/sklearn ./sk2/libpython3.7/site-packages/sklearn0204
# add both of them to your PYTHONPATH
$ export PYTHONPATH=$PYTHONPATH:$(pwd)/sk1/lib/python3.7/site-packages/sklearn0222
$ export PYTHONPATH=$PYTHONPATH:$(pwd)/sk2/lib/python3.7/site-packages/sklearn0204
Now let's go into the python interpreter, import them:
$ python3
>>> import sklearn0222 as sk0222
>>> import sklearn0204 as sk0204
>>> sk0222.__version__
'0.22.2'
>>> sk0204.__version__
'0.20.4'
This will use the packages version specific code to run, but you have to be SUPER CAREFUL when referencing each and you cannot use both packages within the same module. so in mymodule1.py you can import sklearn0222 and use its submodules and in mymodule2.py you can import sklearn0204 and use its submodules, but if you try to use both in the same module in your program the second will not be recognized.
Again, this is a bad idea but this is a way to get what you are looking for.
How can I set the installation path for pip using get-pip.py to /usr/local/bin/? I can't find any mention in the setup guide or in the command line options.
To clarify I don't mean the path where pip packages are installed, but the path where pip itself is installed (it shuold be in /usr/local/bin/pip).
Edit
I do agree with many of the comments/answers that virtualenv would be a better idea in general. However it simply isn't the best one for me at the moment since it would be too disruptive; many of our user's scripts rely on a python2.7 being magically available; this is not convenient either and should change, but we have been using python since before virtualenv was a thing.
Pip itself is a Python package, and the actual pip command just runs a small Python script which then imports and runs the pip package.
You can edit locations.py to change installation directories, however, as stated above, I highly recommend that you do not do this.
Pip Command
Pip accepts a flag, '--install-option="--install-scripts"', which can be used to change the installation directory:
pip install somepackage --install-option="--install-scripts=/usr/local/bin"
Source method
On line 124 in pip/locations.py, we see the following:
site_packages = sysconfig.get_python_lib()
user_site = site.USER_SITE
You can technically edit these to change the default install path, however, using a virtual environment would be highly preferable. This is then used to find the egglink path, which then finds the dist path (code appended below, from pip/__init__.py).
def egg_link_path(dist):
"""
Return the path for the .egg-link file if it exists, otherwise, None.
There's 3 scenarios:
1) not in a virtualenv
try to find in site.USER_SITE, then site_packages
2) in a no-global virtualenv
try to find in site_packages
3) in a yes-global virtualenv
try to find in site_packages, then site.USER_SITE
(don't look in global location)
For #1 and #3, there could be odd cases, where there's an egg-link in 2
locations.
This method will just return the first one found.
"""
sites = []
if running_under_virtualenv():
if virtualenv_no_global():
sites.append(site_packages)
else:
sites.append(site_packages)
if user_site:
sites.append(user_site)
else:
if user_site:
sites.append(user_site)
sites.append(site_packages)
for site in sites:
egglink = os.path.join(site, dist.project_name) + '.egg-link'
if os.path.isfile(egglink):
return egglink
def dist_location(dist):
"""
Get the site-packages location of this distribution. Generally
this is dist.location, except in the case of develop-installed
packages, where dist.location is the source code location, and we
want to know where the egg-link file is.
"""
egg_link = egg_link_path(dist)
if egg_link:
return egg_link
return dist.location
However, once again, using a virtualenv is much more traceable, and any Pip updates will override these changes, unlike your own virtualenv.
It seems the easiest workaround I found to do this is:
install easy_install; this will go in /usr/local/bin/ as expected; the steps for doing this are listed here; I personally ended up running wget https://bootstrap.pypa.io/ez_setup.py -O - | python
install pip with /usr/local/bin/easy_install pip; this will make pip go in /usr/local/bin
I am using simple entry points to make a custom script, with this in setup.py:
entry_points = {
'my_scripts': ['combine_stuff = mypackage.mymod.test:foo']
}
where mypackage/mymod/test.py contains:
import argh
from argh import arg
#arg("myarg", help="Test arg.")
def foo(myarg):
print "Got: ", myarg
When I install my package using this (in same directory as setup.py)
pip install --user -e .
The entry points do not get processed at all it seems. Why is that?
If I install with distribute easy_install, like:
easy_install --user -U .
then the entry points get processed and it creates:
$ cat mypackage.egg-info/entry_points.txt
[my_scripts]
combine_stuff = mypackage.mymod.test:foo
but no actual script called combine_stuff gets placed anywhere in my bin dirs (like ~/.local/bin/). It just doesn't seem to get made. What is going wrong here? How can I get it to make an executable script, and ideally work with pip too?
The answer was to use console_scripts instead of my_scripts. It was unclear that the scripts name was anything other than internal label for the programmer.
What file do I edit, and how? I created a virtual environment.
The most elegant solution to this problem is here.
Original answer remains, but this is a messy solution:
If you want to change the PYTHONPATH used in a virtualenv, you can add the following line to your virtualenv's bin/activate file:
export PYTHONPATH="/the/path/you/want"
This way, the new PYTHONPATH will be set each time you use this virtualenv.
EDIT: (to answer #RamRachum's comment)
To have it restored to its original value on deactivate, you could add
export OLD_PYTHONPATH="$PYTHONPATH"
before the previously mentioned line, and add the following line to your bin/postdeactivate script.
export PYTHONPATH="$OLD_PYTHONPATH"
The comment by #s29 should be an answer:
One way to add a directory to the virtual environment is to install virtualenvwrapper (which is useful for many things) and then do
mkvirtualenv myenv
workon myenv
add2virtualenv . #for current directory
add2virtualenv ~/my/path
If you want to remove these path edit the file myenvhomedir/lib/python2.7/site-packages/_virtualenv_path_extensions.pth
Documentation on virtualenvwrapper can be found at http://virtualenvwrapper.readthedocs.org/en/latest/
Specific documentation on this feature can be found at
http://virtualenvwrapper.readthedocs.org/en/latest/command_ref.html?highlight=add2virtualenv
You can create a .pth file that contains the directory to search for, and place it in the {venv-root}/lib/{python-version}/site-packages directory. E.g.:
cd $(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
echo /some/library/path > some-library.pth
The effect is the same as adding /some/library/path to sys.path, and remain local to the virtualenv setup.
Initialize your virtualenv
cd venv
source bin/activate
Just set or change your python path by entering command following:
export PYTHONPATH='/home/django/srmvenv/lib/python3.4'
for checking python path enter in python:
python
\>\> import sys
\>\> sys.path
I modified my activate script to source the file .virtualenvrc, if it exists in the current directory, and to save/restore PYTHONPATH on activate/deactivate.
You can find the patched activate script here.. It's a drop-in replacement for the activate script created by virtualenv 1.11.6.
Then I added something like this to my .virtualenvrc:
export PYTHONPATH="${PYTHONPATH:+$PYTHONPATH:}/some/library/path"
It's already answered here -> Is my virtual environment (python) causing my PYTHONPATH to break?
UNIX/LINUX
Add "export PYTHONPATH=/usr/local/lib/python2.0" this to ~/.bashrc file and source it by typing "source ~/.bashrc" OR ". ~/.bashrc".
WINDOWS XP
1) Go to the Control panel
2) Double click System
3) Go to the Advanced tab
4) Click on Environment Variables
In the System Variables window, check if you have a variable named PYTHONPATH. If you have one already, check that it points to the right directories. If you don't have one already, click the New button and create it.
PYTHON CODE
Alternatively, you can also do below your code:-
import sys
sys.path.append("/home/me/mypy")