Develop Reference

How to get GitLab commits of a file using python gitlab module?

We are trying to get the commits of each file in a Gitlab repository. We are using the Python Gitlab module. We could get the commits of a repository but couldn't get the commits of individual files in the repository. Can someone help us with this?

The commit history of a single file is not exposed through the GitLab API directly. Therefore, there is no direct functionality for this in the python-gitlab gitlab module.
However, you can obtain, effectively, the same information by using available APIs. Specifically, you can either use the repository commits API and diff APIs or the files blame API.
Using the commits API
For example, using the commits API, you can list all commits and their diffs, then associate file changes for each commit.
import gitlab
from collections import defaultdict
TOKEN = 'Your API Token'
gl = gitlab.Gitlab('https://gitlab.example.com', private_token=TOKEN)
project = gl.projects.get(1234)
commits = project.commits.list(all=True)
# file paths and a list of commits which create/modify/delete the file
file_map = defaultdict(list)
for c in commits:
diff = c.diff()
files_changed = set()
for change in diff:
files_changed.add(change['old_path'])
files_changed.add(change['new_path'])
for path in files_changed:
file_map[path].append(c)
# show list of commits which modified README.md
print(file_map['README.md'])
Using the blame API
Using the commits API requires obtaining the diff for every commit, which can take a long time on large repositories.
If you're only interested in the commits which change a single file, traversing the blame tree can be more efficient. However, note that you may also miss commits (for example, commits in other branches or diverged trees) using this method.
def search_blame(project, filename, base_ref=None):
if base_ref is None:
base_ref = project.default_branch
commits = set()
refs_to_check = [base_ref,]
seen = set()
while refs_to_check:
ref = refs_to_check.pop()
if ref in seen:
continue
seen.add(ref)
blame = project.files.blame(filename, ref)
for change in blame:
commit_id = change['commit']['id']
if commit_id not in seen:
refs_to_check.append(commit_id)
refs_to_check.extend(change['commit']['parent_ids'])
for c in change['commit']['parent_ids']:
commits.add(c)
commits.add(commit_id)
return commits
# show commits in blame tree for README.md
# only includes commits in the default branch
print(search_blame(project, 'README.md'))

How to reset head of master branch to a previous commit with GitPython

I want to essentially revert changes in my master branch.
I am able to find my history of commits by doing:
import git
repo = git.Repo('repos/my-repo')
commits = repo.iter_commits('master',max_count=10)
but I am unsure as to how to point the head to, lets say, a commit where the message contains "reset to me". I am aware of repo.git.reset('--hard'), but I don't know how to properly use it. Thank you

If you know the commit number as in Latest Commit = 1, Second = 2, etc. then you can use ~ operator along with HEAD to point to the commit. HEAD~1 = Latest commit, HEAD~2 = second latest commit.
Hence to remove the latest commit, you can use:
import git
repo = git.Repo('repos/my-repo')
repo.head.reset('--hard HEAD~1', index=True, working_tree=True)
Refer this question to learn more about how to identify a commit.

Checking out a branch with GitPython. making commits, and then switching back to the previous branch

I'm using the GitPython library to do some simple Git manipulation and I'd like to checkout a branch, make a commit, and then checkout the previous branch. The docs are a little confusing on how to do this. So far, I have this:
import git
repo = git.Repo()
previous_branch = repo.active_branch
new_branch_name = "foo"
new_branch = repo.create_head(new_branch_name)
new_branch.checkout()
repo.index.commit("my commit message") # this seems wrong
## ?????
I can tell that this works by verifying it via git commands, but I get the feeling that I'm doing this incorrectly. I'm not how to switch back to the previous branch safely sort of using the raw git commands (from within the library directly).

From http://gitpython.readthedocs.io/en/stable/tutorial.html
Switching Branches
To switch between branches similar to git checkout, you effectively need to point your HEAD symbolic reference to the new branch and reset your index and working copy to match. A simple manual way to do it is the following one
# Reset our working tree 10 commits into the past
past_branch = repo.create_head('past_branch', 'HEAD~10')
repo.head.reference = past_branch
assert not repo.head.is_detached
# reset the index and working tree to match the pointed-to commit
repo.head.reset(index=True, working_tree=True)
# To detach your head, you have to point to a commit directy
repo.head.reference = repo.commit('HEAD~5')
assert repo.head.is_detached
# now our head points 15 commits into the past, whereas the working tree
# and index are 10 commits in the past
The previous approach would brutally overwrite the user’s changes in the working copy and index though and is less sophisticated than a git-checkout. The latter will generally prevent you from destroying your work. Use the safer approach as follows.
# checkout the branch using git-checkout. It will fail as the working tree appears dirty
self.failUnlessRaises(git.GitCommandError, repo.heads.master.checkout)
repo.heads.past_branch.checkout()
Or, just below that:
Using git directly
In case you are missing functionality as it has not been wrapped, you may conveniently use the git command directly. It is owned by each repository instance.
git = repo.git
git.checkout('HEAD', b="my_new_branch") # create a new branch
git.branch('another-new-one')
git.branch('-D', 'another-new-one') # pass strings for full control over argument order
git.for_each_ref() # '-' becomes '_' when calling it
And simply do the git.checkout() approach

GitPython nothing appears in working copy after pull

I'm new in PythonGit and I have problem with pulling and pushing. I created locally bare repo and pushed to it an initial commit. After that I tried to init new user repo using PythonGit, fetch it and pull from it. I have no problem with initialize the repo however I can't get anything from remote/bare repo. My code:
import git
repo = git.Repo.init('.')
origin = repo.create_remote('origin', '/home/paweber/git/my-repo.git')
origin.fetch()
repo.create_head('master', origin.refs.master).set_tracking_branch(origin.refs.master)
origin.pull()
In ipython console for fetch and pull I get:
In [5]: origin.fetch()
Out[5]: [<git.remote.FetchInfo at 0x7f4a4d6ee630>]
for fetch and
In [6]: origin.pull()
Out[6]: [<git.remote.FetchInfo at 0x7f4a4d6e6ee8>]
for pull. After pull action, nothing is pulled at all and repo is still empty but exists. What I'm doing wrong?

pull() doesn't do anything as master is already at its destination commit, the one pointed to by origin/master.
This code will work as expected:
import git
repo = git.Repo.init('.')
origin = repo.create_remote('origin', '/home/paweber/git/my-repo.git')
origin.fetch()
# the HEAD ref usually points to master, which is 'yet to be born'
repo.head.ref.set_tracking_branch(origin.refs.master)
origin.pull()

I don't know how properly resolve this problem but the only idea is as phobic say to reset hard repo after create_head.
import git
repo = git.Repo.init('.')
origin = repo.create_remote('origin', '/home/paweber/git/my-repo.git')
origin.fetch()
repo.create_head('master', origin.refs.master).set_tracking_branch(origin.refs.master)
origin.pull()
repo.head.reset('--hard')
After that all further pulls should work properly.

GitPython: get list of remote commits not yet applied

I am writing a Python script to get a list of commits that are about to be applied by a git pull operation. The excellent GitPython library is a great base to start, but the subtle inner workings of git are killing me. Now, here is what I have at the moment (simplified and annotated version):
repo = git.Repo(path) # get the local repo
local_commit = repo.commit() # latest local commit
remote = git.remote.Remote(repo, 'origin') # remote repo
info = remote.fetch()[0] # fetch changes
remote_commit = info.commit # latest remote commit
if local_commit.hexsha == remote_commit.hexsha: # local is updated; end
return
# for every remote commit
while remote_commit.hexsha != local_commit.hexsha:
authors.append(remote_commit.author.email) # note the author
remote_commit = remote_commit.parents[0] # navigate up to the parent
Essentially it gets the authors for all commits that will be applied in the next git pull. This is working well, but it has the following problems:
When the local commit is ahead of the remote, my code just prints all commits to the first.
A remote commit can have more than one parent, and the local commit can be the second parent. This means that my code will never find the local commit in the remote repository.
I can deal with remote repositories being behind the local one: just look in the other direction (local to remote) at the same time, the code gets messy but it works. But this last problem is killing me: now I need to navegate a (potentially unlimited) tree to find a match for the local commit. This is not just theoretical: my latest change was a repo merge which presents this very problem, so my script is not working.
Getting an ordered list of commits in the remote repository, such as repo.iter_commits() does for a local Repo, would be a great help. But I haven't found in the documentation how to do that. Can I just get a Repo object for the Remote repository?
Is there another approach which might get me there, and I am using a hammer to nail screws?

I know this is ages old but I just had to do this for a project and…
head = repo.head.ref
tracking = head.tracking_branch()
return tracking.commit.iter_items(repo, f'{head.path}..{tracking.path}')
(conversely to know how many local commits you have pending to push, just invert it: head.commit.iter_items(repo, f'{tracking.path}..{head.path}'))

I realized that the tree of commits was always like this: one commit has two parents, and both parents have the same parent. This means that the first commit has two parents but only one grandparent.
So it was not too hard to write a custom iterator to go over commits, including diverging trees. It looks like this:
def repo_changes(commit):
"Iterator over repository changes starting with the given commit."
number = 0
next_parent = None
yield commit # return the first commit itself
while len(commit.parents) > 0: # iterate
same_parent(commit.parents) # check only one grandparent
for parent in commit.parents: # go over all parents
yield parent # return each parent
next_parent = parent # for the next iteration
commit = next_parent # start again
The function same_parent() alerts when there are two parents and more than one grandparent. Now it is a simple matter to iterate over the unmerged commits:
for commit in repo_changes(remote_commit):
if commit.hexsha == local_commit.hexsha:
return
authors.append(remote_commit.author.email)
I have left a few details out for clarity. I never return more than a preestablished number of commits (20 in my case), to avoid going to the end of the repo. I also check beforehand that the local repo is not ahead of the remote repo. Other than that, it is working great! Now I can alert all commit authors that their changes are being merged.