I am trying to write a Python Controller, which would help me automate Git -usage. I've gotten all other commands to work - but I am having difficulties with git push equivalent, when using GitPython Library.
This is where I am right now. This should be working without the SSH Key identification, but I have to squeeze that in.
""" Execute Git Push with GitPython Library.
Hardcoded values: 'branch' environment.
TODO: This is not working. """
def push(self, repo_path, branch, commit_message, user):
repo = Repo(repo_path)
repo.git.add('--all')
repo.git.commit('-m', commit_message)
origin = repo.remote(name=branch)
origin.push()
This is what I have on my Initialization. (Cleared some values due to privacy.)
load_dotenv()
self.BRANCH = "TBD" # Hardcoded Value
self.REPO_PATH = os.getenv('REPO_PATH')
self.REPO = Repo(self.REPO_PATH)
self.COMMIT_MESSAGE = '"Commit from Controller."'
# TODO: These should be changed, when deployed.
self.GIT_SSH_KEY = os.path.expanduser('/home/user/.ssh/id_rsa')
self.GIT_SSH_CMD = "ssh -i %s" % self.GIT_SSH_KEY
self.GIT_USER = "user" # This needs to be changed.
From my understanding from this (GitPython and SSH Keys?) the tactic here is to use GIT_SSH environment variable to provide executable, which will call the ssh - but since I am a beginner, I am having trouble understanding what exactly that environment variable should contain, and how to wrap that with the push function.
Thank you in advance!
First, setting values on self isn't going to accomplish anything by itself, unless there are parts of your code you're not showing us. If you need to set the GIT_SSH environment variable, then you would need to set os.environ['GIT_SSH'].
In general, you shouldn't need to set GIT_SSH unless you require a non-default ssh commandline. That is, if I have:
$ git remote -v
origin ssh://git#github.com/larsks/gnu-hello (fetch)
origin ssh://git#github.com/larsks/gnu-hello (push)
Then I can write:
>>> import git
>>> repo = git.Repo('.')
>>> origin = repo.remote('origin')
>>> res = origin.push()
>>> res[0].summary
'[up to date]\n'
I didn't have to set anything special here; the defaults were entirely appropriate. Under the hood, GitPython just calls the git command line, so anything that works with the cli should work fine without special configuration.
So I'm currently setting up a git pre-commit hook to lint my python files with iSort, and python Black, the issue I'm running into is that when I use git commit --verbose the diff that shows up in the commit editor hasn't actually taken the modifications to the staged files into account.
For example, lets say I have a python file that looks like so:
import re
from os import path
def x():
v = re.compile(r"1")
print(3, v)
def y(v=3):
z = path.join("a", "b")
thing = "a string"
print(thing, z)
Based on the iSort and black settings I have configured, my pre-commit script will change the file to look like so:
import re
from os import path
def x():
v = re.compile(r"1")
print(3, v)
def y(v=3):
z = path.join("a", "b")
thing = "a string"
print(thing, z)
Unfortunately in the git commit editor it still shows the non modified diff. Is there some way to get the editor to have the correct output?
Theoretically I guess it doesn't matter, but it would be nice to see what the diff would actually be.
Instead of a pre-commit hook, try instead a content filter driver, with a smudge/clean script which can:
on checkout make your script one way
on commit (or on git diff) make your script the other way
See an example here or (for clean) here
(image from "Customizing Git - Git Attributes" from "Pro Git book"))
I'm trying to get the patches for a given revision using hglib. I know the hg command is
hg log -pr rev
but I can't find how to do this or equivalent with hglib. It seems there is not functionality to do that, unless I hack the code myself to run the above command. Any help would be greatly appreciated?
The hglib client.log() interface doesn't support what I wanted to do, but I found a simple way to run an arbitrary hg command. This two lines print the patch of revision rev:
out = client.rawcommand([b'log', b'-pr', b'%i'%rev])
print(str(out, 'utf-8'))
May be this is the actual answer!
import hglib
client = hglib.open(<path>)
client.export (revs = str(<revision number>), output = <output file path>)
You can execute the same with subprocess package by yourself to save interpretation time. Rawcommand just builds a command with the parameters we pass and executes with subprocess again.
I try to make git commits programmatically right in Python/Django. The problem I encounter is the syntax of the command. The message of a commit is a variable, that may contain several words. So, I tried to do it this way:
command('git commit -m "%s"'%msg) # command is a custom function that calls
# system Popen function
If I then make a push command to a remote repository at github, then first what I see, is that commit messages are in double quotes like "Test" and another problem is that if a message contains two or more words like "Test message", then it is not even executed. In other words, in a situation like
msg = "Test message"
command('git commit -m "%s"'%msg)
Nothing happens.
I used git from command line, too. This is how I do it here.
import subprocess
def git_commit(message):
return subprocess.check_output(['git', 'commit', '-m', message])
If nothing happens it might be that you need to add the changed files first.
def git_add(file_path):
return subprocess.check_output(['git', 'add', file_path])
Or you add and commit all.
def git_commit_all(message):
return subprocess.check_output(['git', 'commit', '-am', message])
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
What are people's experiences with any of the Git modules for Python? (I know of GitPython, PyGit, and Dulwich - feel free to mention others if you know of them.)
I am writing a program which will have to interact (add, delete, commit) with a Git repository, but have no experience with Git, so one of the things I'm looking for is ease of use/understanding with regards to Git.
The other things I'm primarily interested in are maturity and completeness of the library, a reasonable lack of bugs, continued development, and helpfulness of the documentation and developers.
If you think of something else I might want/need to know, please feel free to mention it.
While this question was asked a while ago and I don't know the state of the libraries at that point, it is worth mentioning for searchers that GitPython does a good job of abstracting the command line tools so that you don't need to use subprocess. There are some useful built in abstractions that you can use, but for everything else you can do things like:
import git
repo = git.Repo( '/home/me/repodir' )
print repo.git.status()
# checkout and track a remote branch
print repo.git.checkout( 'origin/somebranch', b='somebranch' )
# add a file
print repo.git.add( 'somefile' )
# commit
print repo.git.commit( m='my commit message' )
# now we are one commit ahead
print repo.git.status()
Everything else in GitPython just makes it easier to navigate. I'm fairly well satisfied with this library and appreciate that it is a wrapper on the underlying git tools.
UPDATE: I've switched to using the sh module for not just git but most commandline utilities I need in python. To replicate the above I would do this instead:
import sh
git = sh.git.bake(_cwd='/home/me/repodir')
print git.status()
# checkout and track a remote branch
print git.checkout('-b', 'somebranch')
# add a file
print git.add('somefile')
# commit
print git.commit(m='my commit message')
# now we are one commit ahead
print git.status()
I thought I would answer my own question, since I'm taking a different path than suggested in the answers. Nonetheless, thanks to those who answered.
First, a brief synopsis of my experiences with GitPython, PyGit, and Dulwich:
GitPython: After downloading, I got this imported and the appropriate object initialized. However, trying to do what was suggested in the tutorial led to errors. Lacking more documentation, I turned elsewhere.
PyGit: This would not even import, and I could find no documentation.
Dulwich: Seems to be the most promising (at least for what I wanted and saw). I made some progress with it, more than with GitPython, since its egg comes with Python source. However, after a while, I decided it may just be easier to try what I did.
Also, StGit looks interesting, but I would need the functionality extracted into a separate module and do not want wait for that to happen right now.
In (much) less time than I spent trying to get the three modules above working, I managed to get git commands working via the subprocess module, e.g.
def gitAdd(fileName, repoDir):
cmd = ['git', 'add', fileName]
p = subprocess.Popen(cmd, cwd=repoDir)
p.wait()
gitAdd('exampleFile.txt', '/usr/local/example_git_repo_dir')
This isn't fully incorporated into my program yet, but I'm not anticipating a problem, except maybe speed (since I'll be processing hundreds or even thousands of files at times).
Maybe I just didn't have the patience to get things going with Dulwich or GitPython. That said, I'm hopeful the modules will get more development and be more useful soon.
I'd recommend pygit2 - it uses the excellent libgit2 bindings
This is a pretty old question, and while looking for Git libraries, I found one that was made this year (2013) called Gittle.
It worked great for me (where the others I tried were flaky), and seems to cover most of the common actions.
Some examples from the README:
from gittle import Gittle
# Clone a repository
repo_path = '/tmp/gittle_bare'
repo_url = 'git://github.com/FriendCode/gittle.git'
repo = Gittle.clone(repo_url, repo_path)
# Stage multiple files
repo.stage(['other1.txt', 'other2.txt'])
# Do the commit
repo.commit(name="Samy Pesse", email="samy#friendco.de", message="This is a commit")
# Authentication with RSA private key
key_file = open('/Users/Me/keys/rsa/private_rsa')
repo.auth(pkey=key_file)
# Do push
repo.push()
Maybe it helps, but Bazaar and Mercurial are both using dulwich for their Git interoperability.
Dulwich is probably different than the other in the sense that's it's a reimplementation of git in python. The other might just be a wrapper around Git's commands (so it could be simpler to use from a high level point of view: commit/add/delete), it probably means their API is very close to git's command line so you'll need to gain experience with Git.
For the sake of completeness, http://github.com/alex/pyvcs/ is an abstraction layer for all dvcs's. It uses dulwich, but provides interop with the other dvcs's.
An updated answer reflecting changed times:
GitPython currently is the easiest to use. It supports wrapping of many git plumbing commands and has pluggable object database (dulwich being one of them), and if a command isn't implemented, provides an easy api for shelling out to the command line. For example:
repo = Repo('.')
repo.checkout(b='new_branch')
This calls:
bash$ git checkout -b new_branch
Dulwich is also good but much lower level. It's somewhat of a pain to use because it requires operating on git objects at the plumbing level and doesn't have nice porcelain that you'd normally want to do. However, if you plan on modifying any parts of git, or use git-receive-pack and git-upload-pack, you need to use dulwich.
PTBNL's Answer is quite perfect for me.
I make a little more for Windows user.
import time
import subprocess
def gitAdd(fileName, repoDir):
cmd = 'git add ' + fileName
pipe = subprocess.Popen(cmd, shell=True, cwd=repoDir,stdout = subprocess.PIPE,stderr = subprocess.PIPE )
(out, error) = pipe.communicate()
print out,error
pipe.wait()
return
def gitCommit(commitMessage, repoDir):
cmd = 'git commit -am "%s"'%commitMessage
pipe = subprocess.Popen(cmd, shell=True, cwd=repoDir,stdout = subprocess.PIPE,stderr = subprocess.PIPE )
(out, error) = pipe.communicate()
print out,error
pipe.wait()
return
def gitPush(repoDir):
cmd = 'git push '
pipe = subprocess.Popen(cmd, shell=True, cwd=repoDir,stdout = subprocess.PIPE,stderr = subprocess.PIPE )
(out, error) = pipe.communicate()
pipe.wait()
return
temp=time.localtime(time.time())
uploaddate= str(temp[0])+'_'+str(temp[1])+'_'+str(temp[2])+'_'+str(temp[3])+'_'+str(temp[4])
repoDir='d:\\c_Billy\\vfat\\Programming\\Projector\\billyccm' # your git repository , windows your need to use double backslash for right directory.
gitAdd('.',repoDir )
gitCommit(uploaddate, repoDir)
gitPush(repoDir)
Here's a really quick implementation of "git status":
import os
import string
from subprocess import *
repoDir = '/Users/foo/project'
def command(x):
return str(Popen(x.split(' '), stdout=PIPE).communicate()[0])
def rm_empty(L): return [l for l in L if (l and l!="")]
def getUntracked():
os.chdir(repoDir)
status = command("git status")
if "# Untracked files:" in status:
untf = status.split("# Untracked files:")[1][1:].split("\n")
return rm_empty([x[2:] for x in untf if string.strip(x) != "#" and x.startswith("#\t")])
else:
return []
def getNew():
os.chdir(repoDir)
status = command("git status").split("\n")
return [x[14:] for x in status if x.startswith("#\tnew file: ")]
def getModified():
os.chdir(repoDir)
status = command("git status").split("\n")
return [x[14:] for x in status if x.startswith("#\tmodified: ")]
print("Untracked:")
print( getUntracked() )
print("New:")
print( getNew() )
print("Modified:")
print( getModified() )
The git interaction library part of StGit is actually pretty good. However, it isn't broken out as a separate package but if there is sufficient interest, I'm sure that can be fixed.
It has very nice abstractions for representing commits, trees etc, and for creating new commits and trees.
For the record, none of the aforementioned Git Python libraries seem to contain a "git status" equivalent, which is really the only thing I would want since dealing with the rest of the git commands via subprocess is so easy.