My startup command I have for my discord.py bot installs requirements from a requirements.txt file and puts them into the specified folder (the requirements folder) but the issue I'm having is that the bot doesn't look into that folder for dependencies and just looks at the root folder instead
Is there a way I could make the bot look at a specific folder for dependencies instead of the root one? Because if I were to use the root folder, it'd make it look messy and harder to navigate
Here is the command I use:
pip install -U --target /home/container/requirements -r requirements.txt; fi; /usr/local/bin/python /home/container/bot.py
If I understand correctly, You need to use a virtual environment.
You can do that with may ways venv poetry pipenv.
I recommend you use pipenv all you have to do is install your requirements.txt to the environment and then run your code inside of it.
pipenv install -r requirements.txt
I'm trying to implement Deploy to Heroku functionality for my Python application:
https://github.com/jet-admin/jet-bridge/tree/heroku
It woks OK if just use requirements.txt to install dependencies, but it require me to modify my requirements.txt to include some extra packages that i normally don't need (psycopg2, mysqlclient).
Is it possible not to include all requirements in requirements.txt, but install it with some extra command? I've tried adding postdeploy script which will perform pip install command, but after deploy succed my application says that psycopg2 is not installed (thought i installed it in postdeploy command).
The Heroku Python buildpack has a hook where you can execute extra commands after the initial slug compilation.
To use it, you can add a bin/post_compile file, putting inside the shell commands for installing the extra packages.
You can even make it depend on an environment variable, like:
# assuming you have files mysql-requirements.txt and postgres-requirements.txt
if [ "$JET_BRIDGE_DB" == "mysql" ]; then
echo "Installing Python dependencies for MySQL support"
pip install -r mysql-requirements.txt
else
echo "Assuming Postgres database, installing Python dependencies"
pip install -r postgres-requirements.txt
fi
Read more:
Buildpacks: https://devcenter.heroku.com/articles/buildpacks
Heroku Python Buildpack: https://github.com/heroku/heroku-buildpack-python
Slug compilation: https://devcenter.heroku.com/articles/slug-compiler
How can I run target make install only if requirements.txt is changed ?
I don't want to upgrade packages each time when I do make install
I found some workaround by creating fake file _requirements.txt.pyc but is ugly and dirty. It will refuse install pip requirements second time because requirements.txt has no changes
$ make install-pip-requirements
make: Nothing to be done for 'install-pip-requirements'.
But my goal is to do:
# first time,
$ make install # create virtual environment, install requirements
# second time
$ make install # detected and skipping creating virtual env,
# detect that requirements.txt have no changes
# and skipping installing again all python packages
make: Nothing to be done for 'install'.
Python package looks like:
.
├── Makefile
├── README.rst
├── lambda_handler.py
└── requirements.txt
I am using file, Makefile, for some automation in python:
/opt/virtual_env:
# create virtual env if folder not exists
python -m venv /opt/virtual_env
virtual: /opt/virtual_env
# if requirements.txt is modified than execute pip install
_requirements.txt.pyc: requirements.txt
/opt/virtual_env/bin/pip install -r --upgrade requirements.txt
echo > _requirements.txt.pyc
requirements: SOME MAGIG OR SOME make flags
pip install -r requirements.txt
install-pip-requirements: _requirements.txt.pyc
install: virtual requirements
I am sure that
Must be a better way
to do this;)
Not sure it will answer your question at this point. The better way is to use a fully fledged Python PIP project template.
We use cookiecutter to create a particular pip package with this cookiecutter template.
It has a Makefile, which does not constantly re-install all the dependencies and it makes use of Python tox, which allows running a project tests in different python envs automatically. You still can develop in dev virtualenv, but we update it only when new package is added, everything else is handle by tox.
But, what you show so far is trying to write a Python build from scratch, which was done with numerous project templates. If you really want to understand what is going on there, you can analyze these templates.
As followup: Because you expect it to work with a makefile, I'd suggest removing the --upgrade flag from the pip command. I suspect your requirements do not include versions that are needed for the project to work. We made an experience, that not putting versions there might badly brake things. Thus our requirements.txt looks like:
configure==0.5
falcon==0.3.0
futures==3.0.5
gevent==1.1.1
greenlet==0.4.9
gunicorn==19.4.5
hiredis==0.2.0
python-mimeparse==1.5.2
PyYAML==3.11
redis==2.10.5
six==1.10.0
eventlet==0.18.4
Using the requirements without --upgrade causes pip simply verify what is in virtualenv and what not. Everything that satisfies the required version will be skipped (no download). You can also reference git versions in requirements like that:
-e git+http://some-url-here/path-to/repository.git#branch-name-OR-commit-id#egg=package-name-how-to-appear-in-pip-freeze
#Andrei.Danciuc, make just needs two files to compare; you can use any of the output files from running pip install.
For example, I usually use a "vendored" folder, so I can alias the path to the "vendored" folder instead of using a dummy file.
# Only run install if requirements.txt is newer than vendored folder
vendored-folder := vendored
.PHONY: install
install: $(vendored-folder)
$(vendored-folder): requirements.txt
rm -rf $(vendored-folder)
pip install -r requirements.txt -t $(vendored-folder)
If you don't use a vendored folder, this code below should work for both virtualenv and global setups.
# Only run install if requirements.txt is newer than SITE_PACKAGES location
.PHONY: install
SITE_PACKAGES := $(shell pip show pip | grep '^Location' | cut -f2 -d':')
install: $(SITE_PACKAGES)
$(SITE_PACKAGES): requirements.txt
pip install -r requirements.txt
I am trying to install a python module from a git repository, however it seems that the only way to successfully do this is to add -e to the command which leaves behind local files. When I remove the -e it says it installs, but I am left without any files and it fails to import. Here is my command:
pip install -e git+http://gitlab.valk.net/operations/cilib.git#egg=cilib
Further note, when the -e is left off, there is only an egg-info file in site-packages but not files. Is anyone aware of how you do this?
Thanks
Am on Debian 5, I've been trying to install cx_oracle module for python without any success. First, I installed oracle-xe-client and its dependency (followed tutorial in the following link here).
Then, I used the scripts in /usr/lib/oracle/xe/app/oracle/product/10.2.0/client/bin to populate environment variables such as PATH, ORACLE_HOME and NLS_LANG.
Once, this was completed, I tried to run:
sudo easy_install cx_oracle
But I keep getting the following error:
Searching for cx-oracle
Reading http://pypi.python.org/simple/cx_oracle/
Reading http://cx-oracle.sourceforge.net
Reading http://starship.python.net/crew/atuining
Best match: cx-Oracle 5.0.4
Downloading http://prdownloads.sourceforge.net/cx-oracle/cx_Oracle-5.0.4.tar.gz?download
Processing cx_Oracle-5.0.4.tar.gz
Running cx_Oracle-5.0.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xsylvG/cx_Oracle-5.0.4/egg-dist-tmp-8KoqIx
error: cannot locate an Oracle software installation
Any idea what I missed here?
The alternate way, that doesn't require RPMs. You need to be root.
Dependencies
Install the following packages:
apt-get install python-dev build-essential libaio1
Download Instant Client for Linux x86-64
Download the following files from Oracle's download site:
Extract the zip files
Unzip the downloaded zip files to some directory, I'm using:
/opt/ora/
Add environment variables
Create a file in /etc/profile.d/oracle.sh that includes
export ORACLE_HOME=/opt/ora/instantclient_11_2
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME
Create a file in /etc/ld.so.conf.d/oracle.conf that includes
/opt/ora/instantclient_11_2
Execute the following command
sudo ldconfig
Note: you may need to reboot to apply settings
Create a symlink
cd $ORACLE_HOME
ln -s libclntsh.so.11.1 libclntsh.so
Install cx_Oracle python package
You may install using pip
pip install cx_Oracle
Or install manually
Download the cx_Oracle source zip that corresponds with your Python and Oracle version. Then expand the archive, and run from the extracted directory:
python setup.py build
python setup.py install
I recommend that you grab the rpm files and install them with alien. That way, you can later on run apt-get purge no-longer-needed.
In my case, the only env variable I needed is LD_LIBRARY_PATH, so I did:
echo export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client/lib >> ~/.bashrc
source ~/.bashrc
I suppose in your case that path variable will be /usr/lib/oracle/xe/app/oracle/product/10.2.0/client/lib.
The following worked for me, both on mac and Linux. This one command should download needed additional files, without need need to set environment variables.
python -m pip install cx_Oracle --pre
Note, the --pre option is for development and pre-release of the Oracle driver. As of this posting, it was grabbing cx_Oracle-6.0rc1.tar.gz, which was needed. (I'm using python 3.6)
Thx Burhan Khalid, I overlooked your "You need to be root" quote, but found the way when you are not the root here.
At point 7 you need to use:
sudo env ORACLE_HOME=$ORACLE_HOME python setup.py install
Or
sudo env ORACLE_HOME=/path/to/instantclient python setup.py install
Thanks Burhan Khalid. Your advice to make a a soft link make my installation finally work.
To recap:
You need both the basic version and the SDK version of instant client
You need to set both LD_LIBRARY_PATH and ORACLE_HOME
You need to create a soft link (ln -s libclntsh.so.12.1 libclntsh.so in my case)
None of this is documented anywhere, which is quite unbelievable and quite frustrating. I spent over 3 hours yesterday with failed builds because I didn't know to create a soft link.
I think it may be the sudo has no access to get ORACLE_HOME.You can do like this.
sudo visudo
modify the text add
Defaults env_keep += "ORACLE_HOME"
then
sudo python setup.py build install
Alternatively you can install the cx_Oracle module without the PIP using the following steps
Download the source from here https://pypi.python.org/pypi/cx_Oracle
[cx_Oracle-6.1.tar.gz ]
Extract the tar using the following commands (Linux)
gunzip cx_Oracle-6.1.tar.gz
tar -xf cx_Oracle-6.1.tar
cd cx_Oracle-6.1
Build the module
python setup.py build
Install the module
python setup.py install
This just worked for me on Ubuntu 16:
Download ('instantclient-basic-linux.x64-12.2.0.1.0.zip' and 'instantclient-sdk-linux.x64-12.2.0.1.0.zip') from Oracle web site and then do following script (you can do piece by piece and I did as a ROOT):
apt-get install -y python-dev build-essential libaio1
mkdir -p /opt/ora/
cd /opt/ora/
## Now put 2 ZIP files:
# ('instantclient-basic-linux.x64-12.2.0.1.0.zip' and 'instantclient-sdk-linux.x64-12.2.0.1.0.zip')
# into /opt/ora/ and unzip them -> both will be unzipped into 1 directory: /opt/ora/instantclient_12_2
rm -rf /etc/profile.d/oracle.sh
echo "export ORACLE_HOME=/opt/ora/instantclient_12_2" >> /etc/profile.d/oracle.sh
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME" >> /etc/profile.d/oracle.sh
chmod 777 /etc/profile.d/oracle.sh
source /etc/profile.d/oracle.sh
env | grep -i ora # This will check current ENVIRONMENT settings for Oracle
rm -rf /etc/ld.so.conf.d/oracle.conf
echo "/opt/ora/instantclient_12_2" >> /etc/ld.so.conf.d/oracle.conf
ldconfig
cd $ORACLE_HOME
ls -lrth libclntsh* # This will show which version of 'libclntsh' you have... --> needed for following line:
ln -s libclntsh.so.12.1 libclntsh.so
pip install cx_Oracle # Maybe not needed but I did it anyway (only pip install cx_Oracle without above steps did not work for me...)
Your python scripts are now ready to use 'cx_Oracle'... Enjoy!
This worked for me
python -m pip install cx_Oracle --upgrade
For details refer to the oracle quick start guide
https://cx-oracle.readthedocs.io/en/latest/installation.html#quick-start-cx-oracle-installation
If you are trying to install in MAC , just unzip the Oracle client which you downloaded and place it into the folder where you written python scripts.
it will start working.
There is too much problem of setting up environmental variables.
It worked for me.
Hope this helps.
Thanks
Try to reinstall it with the following code:
!pip install --proxy http://username:windowspwd#10.200.72.2:8080 --upgrade --force-reinstall cx_Oracle
If you require to install a specific version of cx_Oracle, like 7.3 which was the last version with support for Python 2, you can do the following:
python -m pip install cx_Oracle==7.3