git cannot execute python-script as hook - python

I have created a little pre-commit hook in python. This hook works like a charm under Linux, but in Windows it keeps telling me:
error: cannot spawn .git/hooks/pre-commit: No such file or directory
I know there have been similar questions here about the same issue and the conclusion seams to be the shebang. My script has this on the very first line:
#!F:\PortableApps\PortablePython3.2\App\python.exe
It's also interesting to note that executing the script simply by writing .git/hooks/pre-commit works wonderful, but as soon as I try to commit, git spits out the above message.
Another interesting thing is, when I convert the encoding from ANSI to UTF-8 (using Notepad++), I get the following error when trying to execute the script:
.git/hooks/pre-commit: Cannot execute binary file
I'm using the following tools:
PortablePython 3.2.1.1
msysgit 1.7.6 (Portable)

I used the proxy-approach to make the python script work under windows (with msysgit). The complete script (with description on how I did it) might be found here: https://gist.github.com/1839424
Here is the important part about making it work under Windows
If you're working with Windows (and "msysgit"), it's a little more complicated. Since "msysgit" seems to have a problem handling the SHEBANG, you'll have to use a little trick to make the script executable (further information on this problem can be found here).
In order to make the script work, you'll want to remove the SHEBANG from the Python script ("pre-commit.py") and use a wrapper bash-script to call the interpreter. This script should look something like this:
#!/bin/sh
python .git/hooks/pre-commit.py
Store this script as a file called "pre-commit" (no file-ending). This assumes that you have Python in your PATH. If you don't, you can also specify the full path to your interpreter-executable.
This script will be called by "git commit" and call the python-script to check for the huge files. The path after the SHEBANG should not be changed, as "msysgit" will remap it automatically. You must specify a path relative to the repo-root for the Python script to be executed (because thats from where the script is called).
Afterwards you'll want to copy both the wrapper-file ("pre-commit") and the Python-script ("pre-commit.py") to your repos ".git/hooks"-directory, personalize the Python-script ("max_file_size" and "git_binary_path") and mark the "pre-commit"-file executable.

Related

Difference Between Running Python File as Executable vs. Running from Command Line?

Background
Below, I detail two different ways of running Python files - I am confused about the difference between them.
Running the Python file as an executable.
To run a Python file as an executable, I must first set a shebang in my file (# /usr/bin/env python3), then run $ chmod +x filename.py at the command line, then run $ ./filename.py at the command line to execute the file.
Running the Python file through the python3 command line command.
To run a Python file through the python3 command, I open my command line and run $ python3 filename.py.
My Question
I understand that, when running the Python file as an executable, the shebang directs the computer to launch the (in this case) python3 interpreter which will interpret the subsequent code in the file and therefore run the file. When running the file through the python3 command, I understand that this is just another way of directing the computer to launch python3 to interpret the code in the file. To me, these two techniques therefore seem identical.
Am I missing something? What's the difference, if any, between these two ways of running a Python file?
In practice, they're identical.
The shebang is just a convention that tells the OS what to do with the script file that you've chmod-ed to be executable -- i.e., executable with what. Without it, the OS just treats the file as a text file and will try to execute it as a shell script.
From the point of view of the system the shebang line is necessary when the file is run as an executable. After checking the permission bits the OS sends the file to the program loader which determines how to run the program by parsing the first line as an interpreter directive. Based on this the loader executes python3 based on the specified path to the executable. If instead /usr/bin/env was used in shebang line then the env command finds the executable based on PATH ( env can also be used to make other modifications to the environment using NAME=VALUE pairs). The loader then passes to the program the path that was used when the user tried to run the script as an argument.
In the second case the OS just loads python3 and passes the script as an argument, it doesn't care at all about the file and its permissions.
From the point of view of the user (which IMO is more important) the shebang line is just another level of abstraction that hides the details of implementation (in this case what program is used to run the script). This means the line can be changed (perhaps by modifying the environment or using a different executable) without the user having to change the way they invoke the script. Also, if the user puts the script in a location that is in PATH then they can invoke the script from anywhere without first navigating to the directory or remembering the location.
On the other hand invoking python3 directly allows the user to pass additional flags to python3 for example -i for interactive runs and -m to use additional modules such as pdb for debugging.
Edit: Based on #alaniwi's comment below explained the role of env in finding the path of the python3 executable in more detail.
Nope, you have pretty much captured it.
A practical consequence is that the shebang relieves you from having to remember whether it's python3 frobnicate or python frobnicate or sh frobnicate or bash frobnicate or awk frobnicate or perl frobnicate or...
This also makes it easy down the line to change your mind. Many tools of mine have started life as simple shell scripts, then been rewritten in Python or something else; but the calling interface doesn't change.
Before Unix, there was an unbridgable gap between system utilities (which you invoke simply by name) and user scripts (which before the introduction of the shebang always had to be called with an explicit interpreter).You still see remnants of this division in lesser systems. An important consequence was that users were able to easily and transparently wrap or replace standard commands with their own versions. This in some sense democratized the system, and empowered users to try out and evaluate improvement ideas for the system on their own. (Figuring out why your brilliant theory wasn't so great in practice is also an excellent way to learn and improve.) I don't think the importance of this versatility and flexibility can be overstated; it's one of those things which converted us from mere users to enthusiasts.

Trying to run a code I wrote myself on my Terminal, the result: Permission denied

I wrote some code on my laptop and now I am trying to run this code. But it give me the output : Permission denied
Usually that should only happen if a different user would try to access it. But I am the user who wrote it and is trying to run it now. I feel like this will be an easy issue to resolve but any help would be nice! Thanks.
I wrote and saved the code in IDLE python. Once I finished I tried running it in my Terminal (I have the macOS Mojave Version 10.14.5) but then I ran into the issue.
Evandros-MBP:~ evandro$ ./Desktop/FunCodes/pwd.py
This is how I tried to call up the code. I don't see any issues here.
Instead of calling the script with
./Desktop/FunCodes/pwd.py
try using
python ./Desktop/FunCodes/pwd.py
or also
python3 ./Desktop/FunCodes/pwd.py
if you want to execute it with python3 instead of python2.
Explanation
When you write ./some-file in the terminal (without writing python before the file path), the terminal executes it as a bash file (no matter if the file has a .sh extension or not). Of course, since your file is a python file, this will fail. But even before it fails, since your python file has no permission to be executed, permission will be denied and bash won't even start.
On creation, the file only has read and write permissions. Add a shebang pointing to the Python interpreter on the script's first line, for example #!/usr/bin/env python. Also, your script is not marked as executable, in order to do that you need to use:
chmod +x pwd.py
Otherwise you will only be able to run it with Python, with
python pwd.py

How do I change directory in python so it remains after running the script?

I'm trying to change the terminal directory through a python script. I've seen this post and others like it so I know about os.chdir, but it's not working the way I'd like. os.chdir appears to change the directory, but only for the python script. For instance I have this code.
#! /usr/bin/env python
import os
os.chdir("/home/chekid/work2/")
print os.getcwd()
Unfortunately after running I'm still in the directory of the python script (e.g. /home/chekid) rather than the directory I want to be in. See below.
gandalf(pts/42):~> pwd
/home/chekid
gandalf(pts/42):~> ./changedirectory.py
/home/chekid/work2
gandalf(pts/42):~> pwd
/home/chekid
Any thoughts on what I should do?
Edit: Looks like what I'm trying to do doesn't exist in 'normal' python. I did find a work around, although it doesn't look so elegant to me.
cd `./changedirectory.py`
You can't. The shell's current directory belongs to the shell, not to you.
(OK, you could ptrace(2) the shell and make it call chdir(2), but that's probably not a great design, won't work on Windows, and I would not begin to know how to do it in pure Python except that you'd probably have to mess around with ctypes or something similar.)
You could launch a subshell with your current working directory. That might be close enough to what you need:
os.chdir('/path/to/somewhere')
shell = os.environ.get('SHELL', '/bin/sh')
os.execl(shell, shell)
# execl() does not return; it replaces the Python process with a new shell process
The original shell will still be there, so make sure you don't leave it hanging around. If you initially call Python with the exec builtin (e.g. exec python /path/to/script.py), then the original shell will be replaced with the Python process and you won't have to worry about this. But if Python exits without launching the shell, you'll be left with no shell open at all.
You can if you cheat: Make a bash script that calls your python script. The python script returns the path you want to change directory to. Then the bash script does the acctual chdir. Of course you would have to run the bash script in your bash shell using "source".
The current working directory is an attribute of a process. It cannot be changed by another program, such as changing the current working directory in your shell by running a separate Python program. This is why cd is always a shell built-in command.
You can make your python print the directory you want to move to, and then call your script with cd "$(./python-script.py)". In condition your script actually does not print anything else.

One-word-command from .py (without aliases)

Some programs can be executed from anywhere with a single one-word command. An example of this is youtube-dl, which is a python program that can be executed with the simple command youtube-dl [input]. As far as I have understood, this is simply because there exists a file called /usr/bin/youtube-dl, and /usr/bin is in PATH. However, I do not understand what I have to do to make something like this myself. Right now, I have a python project called testproject that includes a python program like this:
~/testproject/files/myownprogram.py
What do I have to do to make this a binary executable such as youtube-dl?
I know I can make an alias mop="python ~/testproject/files/myownprogram.py", and this is also what I have done, and it works fine. That is, I can write $ mop, and successfully run my program.
But for curiosity's, and learning's, sake, I want to know how to make a file such as the /usr/bin/youtube-dl file, removing the need for aliases.
I find this hard to find information about in search engines...
Any help is greatly appreciated! :-)
Edit:
My question differs from the one marked as duplicate, in that I'm not looking to execute it as a .sh-script. I simply want to execute it as a suffix-less one-word command, similar to all the other executables that are in /usr/bin. :-)
Ex.: $ myown should run my program, without the need for aliases or writing ".sh" or ".py" at the prompt. That is, I want to have a file /usr/bin/myown that somehow runs my testproject at the simple command myownfrom anywhere.
The applications are being executed "from anywhere" because the system has a specific hierarchy of places it looks for these files (the current directory, then the system path). So, it knows to look in /usr/bin because that's in your system path.
As to ensuring it to use python when it's executed, you should add the following to the top of the file (check out some python application source code and you'll see this):
#!/usr/bin/env python
This tells the system to execute the script using the systems "python" command.

Python script gives `: No such file or directory`

I have several python scripts which work just fine but one script has (as of this morning) started giving me this error if I try to run it from the bash:
: No such file or directory
I am able to run the 'broken' script by doing python script_name.py and after looking around a bit the general idea that I picked up was that maybe my line ending of the hashbang got changed (silently) so I looked at the line ending of a working script and a broken script via the :set list option in VI as indicated in this question -> View line-endings in a text file
Both files appear to end using the same character (a $) so I am kind of at a loss on how to proceed from here. Specifically, how to actually 'see' the line ending in case the set list was not the right method.
PS: The script is executable and the shebang is in there, I stated that it's just this 1 script that was working fine before the weekend but it started giving me this error as of this morning.
-- edit: --
Running the script through dos2unix does get it working again but I would like to know of any way to visualize the line ending somehow in VI(M) or why Geany somehow converted the line endings in the first place (as I never work on a dos/windows system anyhow).
From the comments above it looks like you have dos line endings, and so the hashbang line is not properly processed.
Line ending style are not shown with :set list in Vim because that option is only used when reading/writing the file. In memory line endings are always that, line-endings. The line ending style used for a file is kept in a Vim per-file option, weirdly called fileformat.
To see/change the line ending style from Vim, you can use the following commands:
:set fileformat
:set ff
It will show dos or unix. You want unix, of course ;-).
To change it quickly you can save the file with:
:w ++ff=unix
Or if you prefer:
:set ff=unix
And then save the file normally.
So see all the gory details just do :help fileformat, :help file-formats and :help fileformats
You can also use the dos2unix command to convert the file format
dos2unix
This helped me to run the python scripts
This normally happens when we open files in windows do changes and save it.
if you open the file locate the ^M characters at the end of every line
Thanks
Personally, I find it kinda wrong using direct path to python interpreter. As you dont use windows platform, you should have program env, usually in /usr/bin (/usr/bin/env). Try using following shebang:
#!/usr/bin/env python
Different distros store python binary in /bin or /usr/bin (or some weird locations), and this one makes your script config-independent (as far as possible, here we have possibility that env is stored elsewhere; still - it is less possible that env is not in /usr/bin than that python is mislocated).
I had similiar problem (if not exactly the same) and that worked for me.
Also, I have both python interpreters (2.7.x and 3.x) installed, so I need to use "python3" argument for env. AFAIR usually distros link different names to different binaries, so "env python" will run python2.7 on my system, "env python3" (also python33, or smth like that) will run p3k, and "env python2" (also python27, etc) will run python 2.7.x. Declaring which version of interpreter should be used seems like a good idea too.
I came across this problem editing my code on Windows, checking it in with git, and checking out and running it on Linux.
My solution was: tell git to Do The Right Thing. I issued this command on the Windows box:
git config --global core.autocrlf true
Modified the files and checked them in; voila, no such problem any more.
As discussed on the Git documentation.

Categories

Resources