Develop Reference

How do I run this python script?

I'm new to python, and I was wondering if you could help me run a python script. I'm trying to run a script called PunchBox from Github: https://github.com/psav/punchbox. So far, I have Python 3.9.5 and Git Bash.
In the GitHub page, it says:
To install, clone the repo, cd into it and then execute the following:
virtualenv -p python2 .pb2
source .pb2/bin/activate
pip install -U pip
pip install .
What does this mean exactly? Where do I run this code?
So far, I tried downloading the zip file from GitHub, installing Python 3.5.9, using cmd, finding the directory with cd, and running that code; but got an error:
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name punchbox was given, but was not able to be found.
error in punchbox setup command: Error parsing C:\Users\Mi\Downloads\punchbox-master\punchbox-master\setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name punchbox was given, but was not able to be found.
There's also a requirements.txt that lists additional scripts needed:
pre-commit
click
mido
pbr
PyYAML
svgwrite
Do these install automatically upon running the script for the first time?
I'm a little confused why I'm getting an error. Do you know what I'm doing wrong?
Thank you so much!
Giovanni

I assume you are new to programming. You have to write these lines in a terminal.
On Windows, it is Command Prompt or PowerShell Applications (latter preferred). On macOS, it is terminal
Copy all these lines at once, and paste them to your preferred terminal. The terminal will automatically run these one after the another.
FYI: Venv is a python package to create a virtual environment. The preceding commands set up the environment. Now install the required dependencies using this command instead of the last command (pip install .)
pip install -r requirements.txt

Based on your comment, it looks like you don't have virtualenv installed in your system. You may install it using the command pip install virtualenv.
Now, as you are using a Windows machine, you may open a Command Prompt or Windows PowerShell window and navigate to the directory where your cloned project resides.
Now, execute the following commands.
virtualenv -p python2 .pb2
.pb2\Scripts\activate.bat
pip install -U pip
pip install -r requirements.txt
Once you are done working in your virtual environment (which is named .pb2), you may close it by executing deactivate command.

#Giovanni T.
See, as far as you have installed Python and also downloaded the GitHub Repository as a zip file.
pip install -r requirements.txt
Just run this command.
Please make sure that the directory is pointing to the folder where this requirements.txt file is stored.

How do I install a package from GitHub gist?

I don't understand this...
I want to install this https://gist.github.com/sixtenbe/1178136.
It is a peak detection script for python.
Everywhere I look I am told to use pip with the .git extension.
All I see is how to download the .zip, but from there I am lost.
How can I install this?
Thanks.

You can get the individual files in the Gist (or download the Gist as an ZIP and extract) and put them in your source code folder.
Then you will be able to import them as modules in your own scripts:
import analytic_wfm as AW
AW.ACV_A6( ... )
import peakdetect as PK
PK.peakdetect_parabola( ... )

Let's give it another look.
By "installing a package" we might mean that the package should be available via import.
For that the package directory should reside either in the current directory or in one of the other directories in the import search path.
One such directory is the "user-specific site-packages directory, USER_SITE":
python -c "import site; print(site.getusersitepackages())"
Git URL
First we might need a Git URL. Going to https://gist.github.com/sixtenbe/1178136 we can click on the Embed pop-up and switch it to Clone via HTTPS:
in order to obtain the GIT URL: https://gist.github.com/1178136.git.
git and bash
Having the Git URL and the Unix shell (bash) we can install the package manually into the USER_SITE.
Let's go into the USER_SITE first:
cd $(python -c "import site; print(site.getusersitepackages())")
pwd
Now that we are in the USER_SITE, let's download the Gist:
git clone https://gist.github.com/1178136.git analytic_wfm
Finally, let's verify that the package is now available:
cd && python -c "import analytic_wfm.analytic_wfm; print(analytic_wfm.analytic_wfm.__all__)"
If numpy is installed, it prints
['ACV_A1', 'ACV_A2', 'ACV_A3', 'ACV_A4', 'ACV_A5', 'ACV_A6', 'ACV_A7', 'ACV_A8']
pip
Let's try to install a Gist package with pip.
For pip install we should prefix the Git URL with git+:
pip install --user git+https://gist.github.com/1178136.git
This gives us the error:
ERROR: git+https://gist.github.com/1178136.git does not appear to be a
Python project: neither 'setup.py' nor 'pyproject.toml' found.
Looks like the package we've picked is missing the necessary pip configuration!
Let's try another one:
pip install --user git+https://gist.github.com/bf91613a021a536c7ce16cdba9168604.git
Installs NP:
Successfully built llog
Installing collected packages: llog
Successfully installed llog-1.0
Particularly because it has the setup.py.
Note also that Gist does not support subfolders, and pip seems to depend on them in handling the packages argument, but the code in setup.py can workaround this by creating the package subfolder on the fly and copying the Python files there!
Hence if you want to import that Gist, https://gist.github.com/sixtenbe/1178136, with the rest of the requirements.txt dependencies, - you can fork it and add setup.py to the effect.
pypi
Given that the analytic-wfm can also be found at the Python Package Index, https://pypi.org/project/analytic-wfm/, you can install it with
pip install analytic-wfm

Fail when building xgboost using 64-bit mingw g++

I would like to compile xgboost for Anaconda Python 3.x in 64-bit Windows 10, so I followed the steps here - https://xgboost.readthedocs.org/en/latest/build.html#building-on-windows and here - Install xgboost under python with 64-bit msys failing. Using the 64-bit mingw g++, I got the following error message when "make -j4" using mingw64.mk:
g++: error: dmlc-core/libdmlc.a: No such file or directory
g++: error: rabit/lib/librabit_empty.a: No such file or directory
Makefile:120: recipe for target 'lib/libxgboost.dll' failed
mingw32-make: *** [lib/libxgboost.dll] Error 1
Can you please let me know what's going on and how I can resolve this?
Thanks a lot.

You have to first go to the dmlc-core and rabit sub directories and execute make there. Then come to the xgboost library and execute build.sh.

Maybe you were not in the right folder when running install. Try the following.
Pull up Anaconda prompt, run:
cd xgboost\python-package
python setup.py install
If this still fails, try the process in this link from scratch and it should work. Quote from the link:
First follow the official guide with the following procedure (in Git
Bash on Windows):
git clone --recursive https://github.com/dmlc/xgboost
git submodule init
git submodule update
then install TDM-GCC here and do the following in Git Bash:
alias make='mingw32-make'
cp make/mingw64.mk config.mk; make -j4
Last, do the following using anaconda prompt or Git Bash:
cd xgboost\python-package
python setup.py install

You are missing those libraries in your directory. You have to clone xgboost recursively to include them as well. Just follow the clone process as mentioned in the installation guide and you should be fine!

Pybrain installed but runs only from terminal

Here's the problem: after a lot of struggle I managed to instal pybrain, but it runs only from terminal when I use the command export 'PYTHONPATH=$PYTHONPATH:'. When I try to import modules and write in Python (through anaconda) I get this error 'no module named pybrain'. It's as if Anaconda couldn't see that pybrain is right there. Could it be a problem with the directory?
I don't understand what it means:
"Change into the directory Pybrain directory after either checking out got or downloading and extracting the archive. There you run (possibly as a superuser on Unix systems):
$ python setup.py install"
which I found on 'http://pybrain.org/docs/quickstart/installation.html'

Use the following command to install it properly to Anaconda:
pip install -i https://pypi.binstar.org/pypi/simple pybrain
That should work. The process to find out why is lengthy, and I'll follow up soon.
Proof:

One of the issues could be that it's not permitting you to install this module.
Try this in your PyBrain directory:
sudo python setup.py install
EDIT:
To navigate within terminal to the directory you use the cd - command. Let's say PyBrain is located in your desktop:
$ (this is where you'll be initially on your terminal)
EDIT 2:
To be in your root directory to be able to navigate to any file use the ../ command until you are there (it's appear as a house on your terminal window):
$ cd ../
$ cd ../
Then navigate to the directory:
$ cd desktop/pyBrain
Or in your case:
$ cd anaconda/lib/python3.4/site-packages/pybrain-master
Further documentation about installation can be found here: https://github.com/pybrain/pybrain/wiki/installation
EDIT 3:
Removing Anaconda:
rm -rf ~/anaconda
The rm command will remove the file, and the rf will force it to be deleted (otherwise it might not delete the directory).

How do I correctly install dulwich to get hg-git working on Windows?

I'm trying to use the hg-git Mercurial extension on Windows (Windows 7 64-bit, to be specific). I have Mercurial and Git installed. I have Python 2.5 (32-bit) installed.
I followed the instructions on http://hg-git.github.com/ to install the extension. The initial easy_install failed because it was unable to compile dulwich without Visual Studio 2003.
I installed dulwich manually by:
git clone git://git.samba.org/jelmer/dulwich.git
cd dulwich
c:\Python25\python setup.py --pure install
Now when I run easy_install hg-git, it succeeds (since the dulwich dependency is satisfied).
In my C:\Users\username\Mercurial.ini, I have:
[extensions]
hgext.bookmarks =
hggit =
When I type 'hg' at a command prompt, I see:
"*** failed to import extension hggit: No module named hggit"
Looking under my c:\Python25 folder, the only reference to hggit I see is Lib\site-packages\hg_git-0.2.1-py2.5.egg. Is this supposed to be extracted somewhere, or should it work as-is?
Since that failed, I attempted the "more involved" instructions from the hg-git page that suggested cloning git://github.com/schacon/hg-git.git and referencing the path in my Mercurial configuration. I cloned the repo, and changed my extensions file to look like:
[extensions]
hgext.bookmarks =
hggit = c:\code\hg-git\hggit
Now when I run hg, I see: *** failed to import extension hggit from c:\code\hg-git\hggit: No module named dulwich.errors.
Ok, so that tells me that it is finding hggit now, because I can see in hg-git\hggit\git_handler.py that it calls
from dulwich.errors import HangupException
That makes me think dulwich is not installed correctly, or not in the path.
Update:
From Python command line:
import dulwich
yields Import Error: No module named dulwich
However, under C:\Python25\Lib\site-packages, I do have a dulwich-0.5.0-py2.5.egg folder which appears to be populated. This was created by the steps mentioned above. Is there an additional step I need to take to make it part of the Python "path"?
From Python command line (as suggested in one of the answers):
import pkg_resources
pkg_resources.require('dulwich')
yields [dulwich 0.5.0 (c:\python25\lib\site-packages\dulwich-0.5.0-py2.5.egg)]
So what does that tell me? Importing dulwich fails, but apparently pkg_resources can find it. What can I do with that information?

That makes me think dulwich is not
installed correctly, or not in the
path.
You're absolutely right. Mercurial binary distributions for Windows are 'frozen' - they use the Python code and interpreter bundled with them and therefore independent of packages installed in system PYTHONPATH. When you specify path to hggit extension in Mercurial.ini, hg tries to import it using direct path, but dulwich library is not imported explicitly by hg and doesn't bundled with its library, so the import fails.
It is possible to add both Dulwich and HgGit into library.zip that is installed along with hg.exe, but for me the best way is to install everything from source including Mercurial and execute commands using .bat files installed into \Python\Scripts. In this case you will need to:
Install Mercurial from source. This builds "pure" version, because Windows users usually don't have Visual Studio or alternative compiler for compiling C speedups.
Install Dulwich - I'd use latest trunk snapshot for both Git and Dulwich.
python setup.py --pure install
Install latest HgGit snapshot
python setup.py install
Edit Mercurial.ini to enable hggit =
Launch Mercurial using your \Python\Scripts\hg.bat

I found a simpler solution at http://candidcode.com/2010/01/12/a-guide-to-converting-from-mercurial-hg-to-git-on-a-windows-client/
And then I found a yet simpler solution myself:
To use the hg-git Mercurial extension on Windows:
install the official Mercurial binaries
put dulwich folder
from dulwich sources and hggit folder from hg-git sources to the
root of library.zip in Mercurial installation folder
add the following to %USERPROFILE%\Mercurial.ini:
[extensions]
hgext.bookmarks=
hggit=
To have SSH support you need plink.exe from PuTTY family. After that you should add the following to Mercurial.ini:
[ui]
username = John Doe <foo#example.com>
ssh=d:/home/lib/dll/plink.exe -i "d:/home2/ssh-private-key.ppk"
When connecting to a SSH server for the first time, you should start putty.exe and try to connect using it. It will add the server key fingerprint to the registry. Otherwise plink will ask you to accept the fingerprint but it doesn't accept any input.
You can use puttygen.exe to generate private keys. Either use keys without a passphrase or use Pageant.exe ssh authentication agent.

If you can install TortoiseHg, it includes dulwich and other requirements.

Try following configuration (change to your path), which works for me:
[extensions]
; hg-git extention
hgext.bookmarks =
hggit = C:\Python26\Lib\site-packages\hg_git-0.2.1-py2.6.egg\hggit
In my case when I have empty value for hggit =, I get the same error as you do in this case. But I can import dulwich without problem in python shell, so you should check your easy-install.pth (as pointed out by David) if it contains dulwich-0.5.0-py2.5.egg.
I did install pure version of dulwich as well.

I got this error as well even after downloading the latest Tortoisehg and making sure the hggit plugin was installed as well as my .ini & hgrc files had the right entry to enable hggit.
Turns out my problem was that I had both mercurial and tortoisehg in my path. So when I ran any hg commands, it was using the hg.exe in mercurial folder as opposed to the hg.exe in the torsoisehg directory.
This makes sense but my mercurial installation did not have the plug ins. My fix was to remove mercurial from my path so hg commands go through the tortoisehg directory since it has hg completely bundled. Note however, the recommended option might be to upgrade mercurual to a version that has the plugins that one needs but this is what worked for me. I tried replacing the library.zip in mercurial with the one in tortoisehg and this worked but it led to other errors as one would imagine.
#techtonik's answer led me down this road for which I am grateful.
Recap: verify which hg exe is running your hg commands by checking your path because that hg.exe does not find the plugins for whatever reason.

Had this problem today when installing the latest TortoiseHg.
Get the latest python 2.7 (I used 2.7.16) setup on your system, you probably have this already.
Get a command window and go to the c:\Python27 folder
To run pip use Scripts\pip or easy_install use Scripts\easy_install
Try pip -V to make sure you get the 2.7 version and not some 3.X version
Wrong:
c:\Python27>pip -V
pip 20.2.4 from c:\python38\lib\site-packages\pip (python 3.8)
Right:
c:\Python27>Scripts\pip -V
pip 20.2.4 from c:\python27\lib\site-packages\pip (python 2.7)
If dulwich or hg-git are installed already
Scripts\pip uninstall dulwich
Scripts\pip uninstall hg-git
Install hg-git
Scripts\easy_install install hg-git
You should now have two folders
C:\Python27\Lib\site-packages\dulwich-0.19.16-py2.7-win-amd64.egg\dulwich
C:\Python27\Lib\site-packages\hg_git-0.9.0-py2.7.egg\hggit
It will only work if Dulwich is version 0.19.16 (less than 0.20.x)
Copy these folders (dulwich and hggit) into the zip-file
C:\Program Files\TortoiseHg\lib\library.zip
folder in zip file

Until you get import dulwich to work, hggit won't work. Check that the dulwich egg file is in your easy-install.pth file under site-packages.
For further debugging you can try ask pkg_resources about it:
import pkg_resources
pkg_resources.require("dulwich")

I ran into this problem too with dulwich.errors. Instead of installing everything from scratch. I just copied dulwich from my default site-packages to the mercurial site-packages. worked with no problems.

Based on techtonik's explanation of the reason for the failing import of dulwich.errors, I found a solution which appears simpler to me than the already proposed ones:
On the Mercurial download page there is offered
Mercurial <x.y.z> for Python 2.7 on Windows x86 (source install)
and
Mercurial <x.y.z> for Python 2.7 on Windows x64 (source install)
as .EXE files which install Mercurial as a Python module into an existing Python 2.7 (x86 or x64) installation.
If hg-git and dulwich have been installed to this Python installation as well, the import errors should vanish.
If you are not sure what to do now, I hope this step-by-step explanation helps:
Prerequisites
Python 2.7 is installed
<python 2.7 install dir> and <python 2.7 install dir>\Scripts are in the PATH environment variable
hg-git (and dulwich) have been installed into this Python installation via the instructions from here
Steps
Deinstall any existing Mercurial installation
Download and install one of the above-quoted EXE files, depending on whether your Python 2.7 installation is 32bit or 64bit (If you get a "Python 2.7 cannot be found in registry" error, you probably need the other one.)
Now hg clone <some git repo> should work on the console.

After a long research I finally managed to setup hggit properly.
What I have done:
I have installed the newest version of TortoiseHg (in my case Version 5.9.1)
This official version (automatically) contains Python-2.7.18 (According to Help/About)
Download dulwich: cd C:\dulwich, git clone https://github.com/dulwich/dulwich.git .
TortoiseHg uses Python 2.7, dulwich doesn't support this (old) Python version anymore. We have to go back to an older version of dulwich (<0.20)
git reset --hard c6993451a745d313f61e5d080b9f9d8611a8d7f4
The subfolder C:\dulwich\dulwich has to be copied inside the zip file C:\Program Files\TortoiseHg\lib\library.zip so that there is a path C:\Program Files\TortoiseHg\lib\library.zip\dulwich
This can be done by the following commands:
a) Extract the currenty content of TortoiseHg's library
md "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
tar -xf "C:\Program Files\TortoiseHg\lib\library.zip" -C "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
b) Copy dulwich files into directory
xcopy C:\dulwich\dulwich "C:\Program Files\TortoiseHg\lib\ExtractedLibrary\dulwich\"
c) Save old library.zip and create new library.zip
ren "C:\Program Files\TortoiseHg\lib\library.zip" library_old.zip
cd "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
tar -acf "C:\Program Files\TortoiseHg\lib\library.zip" *.*
d) Cleanup
cd ..
rd /q /s "C:\Program Files\TortoiseHg\lib\ExtractedLibrary"
rd /q /s C:\dulwich
Download hg-git: cd C:\hg-git, hg clone https://foss.heptapod.net/mercurial/hg-git .
The current version of hg-git also does not support Python 2.7 anymore. We have to go back to the last working version 0.9.0 (=revision 1320 in TortoiseHg):
hg update 6d7d6f174df8
Create a new empty repository which will be "connected" to a remote git repository
cd C:\MyLocalHgRepoWhichIsConnectedToGit, hg init
Create a textfile C:\MyLocalHgRepoWhichIsConnectedToGit\.hg\hgrc with the following content:
[paths]
default = C:\ConnectedGitRepoForPushAndPull
[extensions]
hgext.bookmarks =
hggit = C:\hg-git\hggit
Remark: Alternatively you can also copy the folder C:\hg-git\hggit inside library.zip and leave the path in the hgrc file empty. Indeed this is my favourite way. Do it the same way as I have explained under step 4 with dulwich.
That's it. Now the Mercurial Repo is linked to the Git Repo
and you can execute hg pull or hg push inside the Hg Repo.
If you don't want to read all my explanations you can simply use this batch file and execute it:
hggit.bat
#echo off
rem Installation von dulwich+hggit für TortoiseHg
rem Michael Hutter, 22.09.2021
rem https://stackoverflow.com/a/69279657/9134997
if "%1"=="install" goto install
if "%1"=="clonegitrepo" goto clonegitrepo
echo Syntax:
echo %0 install (installiert dulwich und hggit in TortoiseHg)
echo %0 clonegitrepo C:\HgRepoToCreate C:\GitRepoToClone (Klonen eines Git-Repos in ein Hg-Repo)
pause
goto ende
:install
set LibraryPath=C:\Program Files\TortoiseHg\lib
set TempPathDulwich=C:\Temp\dulwich
set TempPathHggit=C:\Temp\hg-git
if not exist "%LibraryPath%\library.zip" (
echo Die Datei %LibraryPath%\library.zip existiert nicht!
pause
goto Ende
)
rem Installation von dulwich
md %TempPathDulwich%
cd %TempPathDulwich%
git clone https://github.com/dulwich/dulwich.git .
git reset --hard c6993451a745d313f61e5d080b9f9d8611a8d7f4
pause
rem Installation von hg-git
md %TempPathHggit%
cd %TempPathHggit%
hg clone https://foss.heptapod.net/mercurial/hg-git .
hg update 6d7d6f174df8
pause
rem dulwich und hggit in library.zip aufnehmen
md "%LibraryPath%\ExtractedLibrary"
tar -xf "%LibraryPath%\library.zip" -C "%LibraryPath%\ExtractedLibrary"
xcopy %TempPathDulwich%\dulwich "%LibraryPath%\ExtractedLibrary\dulwich\"
xcopy %TempPathHggit%\hggit "%LibraryPath%\ExtractedLibrary\hggit\"
ren "%LibraryPath%\library.zip" library_old.zip
if exist "%LibraryPath%\library.zip" del "%LibraryPath%\library.zip"
cd "%LibraryPath%\ExtractedLibrary"
tar -acf "%LibraryPath%\library.zip" *.*
rem Aufräumen
cd ..
rd /q /s "%LibraryPath%\ExtractedLibrary"
rd /q /s %TempPathDulwich%
rd /q /s %TempPathHggit%
pause
goto ende
:clonegitrepo
rem Klonen eines Git-Repos in ein lokales Hg-Repo
echo Erstelle neues HgRepo %2
md %2
cd %2
if not exist %2 (
echo "Fehler: %2 existiert nicht!"
pause
goto ende
)
hg init
echo [paths] > .hg\hgrc
echo default = %3 >> .hg\hgrc
echo. >> .hg\hgrc
echo [extensions] >> .hg\hgrc
echo hgext.bookmarks = >> .hg\hgrc
echo hggit = >> .hg\hgrc
hg pull
hg update
pause
:ende

sudo apt-get install python-dev # Dam you dependency!
sudo easy_install dulwich
Success!