I don't understand this...
I want to install this https://gist.github.com/sixtenbe/1178136.
It is a peak detection script for python.
Everywhere I look I am told to use pip with the .git extension.
All I see is how to download the .zip, but from there I am lost.
How can I install this?
Thanks.
You can get the individual files in the Gist (or download the Gist as an ZIP and extract) and put them in your source code folder.
Then you will be able to import them as modules in your own scripts:
import analytic_wfm as AW
AW.ACV_A6( ... )
import peakdetect as PK
PK.peakdetect_parabola( ... )
Let's give it another look.
By "installing a package" we might mean that the package should be available via import.
For that the package directory should reside either in the current directory or in one of the other directories in the import search path.
One such directory is the "user-specific site-packages directory, USER_SITE":
python -c "import site; print(site.getusersitepackages())"
Git URL
First we might need a Git URL. Going to https://gist.github.com/sixtenbe/1178136 we can click on the Embed pop-up and switch it to Clone via HTTPS:
in order to obtain the GIT URL: https://gist.github.com/1178136.git.
git and bash
Having the Git URL and the Unix shell (bash) we can install the package manually into the USER_SITE.
Let's go into the USER_SITE first:
cd $(python -c "import site; print(site.getusersitepackages())")
pwd
Now that we are in the USER_SITE, let's download the Gist:
git clone https://gist.github.com/1178136.git analytic_wfm
Finally, let's verify that the package is now available:
cd && python -c "import analytic_wfm.analytic_wfm; print(analytic_wfm.analytic_wfm.__all__)"
If numpy is installed, it prints
['ACV_A1', 'ACV_A2', 'ACV_A3', 'ACV_A4', 'ACV_A5', 'ACV_A6', 'ACV_A7', 'ACV_A8']
pip
Let's try to install a Gist package with pip.
For pip install we should prefix the Git URL with git+:
pip install --user git+https://gist.github.com/1178136.git
This gives us the error:
ERROR: git+https://gist.github.com/1178136.git does not appear to be a
Python project: neither 'setup.py' nor 'pyproject.toml' found.
Looks like the package we've picked is missing the necessary pip configuration!
Let's try another one:
pip install --user git+https://gist.github.com/bf91613a021a536c7ce16cdba9168604.git
Installs NP:
Successfully built llog
Installing collected packages: llog
Successfully installed llog-1.0
Particularly because it has the setup.py.
Note also that Gist does not support subfolders, and pip seems to depend on them in handling the packages argument, but the code in setup.py can workaround this by creating the package subfolder on the fly and copying the Python files there!
Hence if you want to import that Gist, https://gist.github.com/sixtenbe/1178136, with the rest of the requirements.txt dependencies, - you can fork it and add setup.py to the effect.
pypi
Given that the analytic-wfm can also be found at the Python Package Index, https://pypi.org/project/analytic-wfm/, you can install it with
pip install analytic-wfm
Related
I would like to use the following sdk in my python project -> https://github.com/LBank-exchange/lbank-api-sdk-v2. It has sdk's for 3 languages (I just want the python one). I tried to install it using the command:
pip install git+https://github.com/LBank-exchange/lbank-api-sdk-v2.git#egg=lbank
which gave the error
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.
Looks like the developer didn't bother to package it properly. It it was me using it, I would fork it on GH, add the setup.py and use the fork. Maybe a good exercise for you?
Meanwhile, to just get it to work, in your project "root":
git clone https://github.com/LBank-exchange/lbank-api-sdk-v2.git
ln -s lbank-api-sdk-v2/python-sdk-api/LBank ./LBank
Then in your code just import LBank. This will leave the cloned repo untouched (so you can git pull to update it later) and just link the module directory to the root. Alternatively you can just include the api directory in sys.path for imports to work.
Think there is nothing to install, if you want to be able to "import" and use it like other packages you install through pip install you can just add the folder to your sys-path:
import sys
sys.path.append("path")
I'm new to python, and I was wondering if you could help me run a python script. I'm trying to run a script called PunchBox from Github: https://github.com/psav/punchbox. So far, I have Python 3.9.5 and Git Bash.
In the GitHub page, it says:
To install, clone the repo, cd into it and then execute the following:
virtualenv -p python2 .pb2
source .pb2/bin/activate
pip install -U pip
pip install .
What does this mean exactly? Where do I run this code?
So far, I tried downloading the zip file from GitHub, installing Python 3.5.9, using cmd, finding the directory with cd, and running that code; but got an error:
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name punchbox was given, but was not able to be found.
error in punchbox setup command: Error parsing C:\Users\Mi\Downloads\punchbox-master\punchbox-master\setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name punchbox was given, but was not able to be found.
There's also a requirements.txt that lists additional scripts needed:
pre-commit
click
mido
pbr
PyYAML
svgwrite
Do these install automatically upon running the script for the first time?
I'm a little confused why I'm getting an error. Do you know what I'm doing wrong?
Thank you so much!
Giovanni
I assume you are new to programming. You have to write these lines in a terminal.
On Windows, it is Command Prompt or PowerShell Applications (latter preferred). On macOS, it is terminal
Copy all these lines at once, and paste them to your preferred terminal. The terminal will automatically run these one after the another.
FYI: Venv is a python package to create a virtual environment. The preceding commands set up the environment. Now install the required dependencies using this command instead of the last command (pip install .)
pip install -r requirements.txt
Based on your comment, it looks like you don't have virtualenv installed in your system. You may install it using the command pip install virtualenv.
Now, as you are using a Windows machine, you may open a Command Prompt or Windows PowerShell window and navigate to the directory where your cloned project resides.
Now, execute the following commands.
virtualenv -p python2 .pb2
.pb2\Scripts\activate.bat
pip install -U pip
pip install -r requirements.txt
Once you are done working in your virtual environment (which is named .pb2), you may close it by executing deactivate command.
#Giovanni T.
See, as far as you have installed Python and also downloaded the GitHub Repository as a zip file.
pip install -r requirements.txt
Just run this command.
Please make sure that the directory is pointing to the folder where this requirements.txt file is stored.
When would the -e, or --editable option be useful with pip install?
For some projects the last line in requirements.txt is -e .. What does it do exactly?
As the man page says it:
-e,--editable <path/url>
Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.
So you would use this when trying to install a package locally, most often in the case when you are developing it on your system. It will just link the package to the original location, basically meaning any changes to the original package would reflect directly in your environment.
Some nuggets around the same here and here.
An example run can be:
pip install -e .
or
pip install -e ~/ultimate-utils/ultimate-utils-proj-src/
note the second is the full path to where the setup.py would be at.
Concrete example of using --editable in development
If you play with this test package as in:
cd ~
git clone https://github.com/cirosantilli/vcdvcd
cd vcdvcd
git checkout 5dd4205c37ed0244ecaf443d8106fadb2f9cfbb8
python -m pip install --editable . --user
it outputs:
Obtaining file:///home/ciro/bak/git/vcdvcd
Installing collected packages: vcdvcd
Attempting uninstall: vcdvcd
Found existing installation: vcdvcd 1.0.6
Can't uninstall 'vcdvcd'. No files were found to uninstall.
Running setup.py develop for vcdvcd
Successfully installed vcdvcd-1.0.6
The Can't uninstall 'vcdvcd' is normal: it tried to uninstall any existing vcdvcd to then replace them with the "symlink-like mechanism" that is produced in the following steps, but failed because there were no previous installations.
Then it generates a file:
~/.local/lib/python3.8/site-packages/vcdvcd.egg-link
which contains:
/home/ciro/vcdvcd
.
and acts as a "symlink" to the Python interpreter.
So now, if I make any changes to the git source code under /home/ciro/vcdvcd, it reflects automatically on importers who can from any directory do:
python -c 'import vcdvcd'
Note however that at my pip version at least, binary files installed with --editable, such as the vcdcat script provided by that package via scripts= on setup.py, do not get symlinked, just copied to:
~/.local/bin/vcdcat
just like for regular installs, and therefore updates to the git repository won't directly affect them.
By comparison, a regular non --editable install from the git source:
python -m pip uninstall vcdvcd
python -m pip install --user .
produces a copy of the installed files under:
~/.local/lib/python3.8/site-packages/vcdvcd
Uninstall of an editable package as done above requires a new enough pip as mentioned at: How to uninstall editable packages with pip (installed with -e)
Tested in Python 3.8, pip 20.0.2, Ubuntu 20.04.
Recommendation: develop directly in-tree whenever possible
The editable setup is useful when you are testing your patch to a package through another project.
If however you can fully test your change in-tree, just do that instead of generating an editable install which is more complex.
E.g., the vcdvcd package above is setup in a way that you can just cd into the source and do ./vcdcat without pip installing the package itself (in general, you might need to install dependencies from requirements.txt though), and the import vcdvcd that that executable does (or possibly your own custom test) just finds the package correctly in the same directory it lives in.
From Working in "development" mode:
Although not required, it’s common to locally install your project in
“editable” or “develop” mode while you’re working on it. This allows
your project to be both installed and editable in project form.
Assuming you’re in the root of your project directory, then run:
pip install -e .
Although somewhat cryptic, -e is short for
--editable, and . refers to the current working directory, so together, it means to install the current directory (i.e. your
project) in editable mode.
Some additional insights into the internals of setuptools and distutils from “Development Mode”:
Under normal circumstances, the distutils assume that you are going to
build a distribution of your project, not use it in its “raw” or
“unbuilt” form. If you were to use the distutils that way, you would
have to rebuild and reinstall your project every time you made a
change to it during development.
Another problem that sometimes comes up with the distutils is that you
may need to do development on two related projects at the same time.
You may need to put both projects’ packages in the same directory to
run them, but need to keep them separate for revision control
purposes. How can you do this?
Setuptools allows you to deploy your projects for use in a common
directory or staging area, but without copying any files. Thus, you
can edit each project’s code in its checkout directory, and only need
to run build commands when you change a project’s C extensions or
similarly compiled files. You can even deploy a project into another
project’s checkout directory, if that’s your preferred way of working
(as opposed to using a common independent staging area or the
site-packages directory).
To do this, use the setup.py develop command. It works very similarly
to setup.py install, except that it doesn’t actually install anything.
Instead, it creates a special .egg-link file in the deployment
directory, that links to your project’s source code. And, if your
deployment directory is Python’s site-packages directory, it will also
update the easy-install.pth file to include your project’s source
code, thereby making it available on sys.path for all programs using
that Python installation.
It is important to note that pip uninstall can not uninstall a module that has been installed with pip install -e. So if you go down this route, be prepared for things to get very messy if you ever need to uninstall. A partial solution is to (1) reinstall, keeping a record of files created, as in sudo python3 -m setup.py install --record installed_files.txt, and then (2) manually delete all the files listed, as in e.g. sudo rm -r /usr/local/lib/python3.7/dist-packages/tdc7201-0.1a2-py3.7.egg/ (for release 0.1a2 of module tdc7201). This does not 100% clean everything up however; even after you've done it, importing the (removed!) local library may succeed, and attempting to install the same version from a remote server may fail to do anything (because it thinks your (deleted!) local version is already up to date).
As suggested in previous answers, there is no symlinks that are getting created.
How does '-e' option work? -> It just updates the file "PYTHONDIR/site-packages/easy-install.pth" with the project path specified in the 'command pip install -e'.
So each time python search for a package it will check this directory as well => any changes to the files in this directory is instantly reflected.
I'm trying to install a library pyleargist. It requires another lib libfftw3 to be manually installed which I've installed. Since I don't have the root privilege, I have to install libfftw3 under my home directory: ~/usr/include and ~/usr/lib. Then I follow this post: https://superuser.com/questions/242190/how-to-install-matplotlib-on-os-x, added:
export LDFLAGS="-L~/usr/lib"
export CFLAGS="-I~/usr/include
So that pip knows it have to consult /usr to get the include (.h files) and lib (.a, *.so files). However, while running pip install --user pyleargist, it complains about:
gcc-4.4.real: src/leargist.c: No such file or directory
gcc-4.4.real: no input files
error: command 'gcc' failed with exit status 1
I guess what happened is that the path is incorrect so that it can't find the *.c files (I think pip should have downloaded the file somewhere but not sure where it is).
So my questions are the following: 1) in this particular case, how can I install pyleargist with include and lib path under ~/usr? 2) more generally, how can one provide additional path for pip so that it knows where to get the additional include files or libs if not found in the default path?
p.s I am on an ubuntu machine without sudo privilege.
ref:
https://pypi.python.org/pypi/pyleargist/1.0.1
http://www.fftw.org/
pip has a --global-option flag
You can use it to pass additional flags to build_ext.
For instance, to add a -I flag:
pip install --global-option=build_ext --global-option="-I/home/users/abc/include/" pyOpenSSL
This was a helpful thread. Just to add on to this, you can also use pip without root if you pass the --user flag at the end:
pip install --global-option="-I/home/users/abc/include/" mpi4py --user
For example, if you're using python-v2.7, the above command installs the python package to /home/username/.local/lib/python2.7/site-packages
if you dont have root you can get a virtual enviroment no root is needed to get one and your path will be in home
curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
tar xvfz virtualenv-1.10.1.tar.gz
cd virtualenv-1.10.1.tar.gz
python virtualenv.py myVE
then your path is set in your home:
cd myVE/bin
./python
>>> import sys
>>> sys.path
['', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python33.zip', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/plat-linux', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/lib-dynload', '/usr/lib64/python3.3', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-linux', '/home/foobar/temp/virtualenv-1.10.1/myVE/lib/python3.3/site-packages']
>>>
I have a git repository with many folders, one of them being a python module installable with pip, like this:
repo.git/
repo.git/folder1/
repo.git/folder2/
repo.git/mymodule/
repo.git/mymodule/__init__.py
repo.git/mymodule/setup.py
repo.git/mymodule/...
Right now I have to do the following to install:
git clone http://server/repo.git
cd repo
pip install mymodule
cd ..
rm -rf repo
Is it possible to install the module directly with pip without explicitly cloning ?
I tried:
pip install git+https://server/repo.git/mymodule/
pip install git+https://server/repo.git:mymodule/
But I get:
IOError: [Errno 2] No such file or directory: '/tmp/pip-88tlLm-build/setup.py'
There is a pull request regarding this feature, and it seems to have been merged to develop branch a month ago. The syntax is the following:
pip install -e git+https://git.repo/some_repo.git#egg=version_subpkg&subdirectory=repo # install a python package from a repo subdirectory
We probably have to wait for a while until it gets merged to master and is distributed.
UPDATE: This is now available and documented at https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support as follows:
For projects where setup.py is not in the root of project,
"subdirectory" component is used. Value of "subdirectory" component
should be a path starting from root of the project to where setup.py
is located.
So if your repository layout is:
- pkg_dir/
- setup.py # setup.py for package ``pkg``
- some_module.py
- other_dir/
- some_file
- some_other_file
You'll need to use
pip install -e vcs+protocol://repo_url/#egg=pkg&subdirectory=pkg_dir
Note: On Windows, you must place the URL in double quotes, or you'll get an error "'subdirectory' is not recognized as an internal or external command". E.g., use:
pip install -e "vcs+protocol://repo_url#egg=pkg&subdirectory=pkg_dir"
It's been already stated in one of the comments under the correct answer, but just to highlight this issue: when executing this from Linux command line, you must escape the &-character since ampersand is telling the command line to run a command in background:
git+https://git.repo/some_repo.git#egg=version_subpkg\&subdirectory=repo
Notice the backslash before the ampersand. The escaping behaviour might depend on the Linux distro; I'm not an expert.
If you ignore this, you might run into a cryptic error like the following:
bash: (...) command not found