am working on a code which I would like to retrieve the commits from a repository on github. Am not entirely sure how to do such a thing, I got git-python but most the api's are for opening a local git repository on the same file system.
Can someone advice?
regards,
For me the following worked best:
Imports:
import os
import datetime
import git
Get current repository, assuming that you're there:
repo = git.Repo(os.getcwd())
Get active branch:
master = repo.head.reference
Current branch:
master.name
Latest commit id:
master.commit.hexsha
Latest commit message:
master.commit.message
Latest commit date:
datetime.datetime.fromtimestamp(master.commit.committed_date)
Latest commit author email:
master.commit.author.email
Latest commit author name:
master.commit.author.name
It seems the easiest thing here is to use the commandline (I'm assuming Linux or any other Unix here, but should be the same on Windows) to clone an existing repository first:
git clone git://github.com/forsberg/misctools.git
This will create the misctools directory.
Now, from python, you can open this repository and update it using pull:
#!/usr/bin/env python
from git import *
repo = Repo("misctools")
o = repo.remotes.origin
o.pull()
master = repo.head.reference
print master.log()
It's all documented at http://packages.python.org/GitPython/0.3.2/tutorial.html
I really advise using only the command line git, git-python its used for macros or complicated things, not just for pulling, pushing or cloning :)
If that's what you're after, I have a bash script to send myself emails about the latest git commits. It runs as a cronjob.
https://github.com/martinxyz/config/blob/master/scripts/email-git-commit-summary.sh
Related
I wrote a simple script in Python.
Now I would like travis to check my code. After travis was successful, the version number should get increased.
Up to now my script has no version number yet. I can store it anywhere where it makes sense for the auto-increment workflow.
How to do this for Python code?
Update
It works now:
run tests
bumpversion
push tag to master
Unfortunately travis does not support "after-all". This means if I want to run the tests for several Python versions, I have no way to bumpversion after the tests of all python versions were successful.
In my case I will check on Python2.7 only until travis resolved this issue: https://github.com/travis-ci/travis-ci/issues/929
Here is my simple script: https://github.com/guettli/compare-with-remote
Solved :-)
It works now:
Developer pushes to github
Travis-CI runs
If all tests are successful bumpversion increases the version
The new version in setup.py get's pushed to the github repo
A new release of the python package gets uploaded to pypi with the tool twine.
I explain the way I do CI with github, travis and pypi here: https://github.com/guettli/github-travis-bumpversion-pypi
If you accept having extra commit for your versioning, you could add this script in continuous_integration/increment_version.py
import os
import pkg_resources
if __name__ == "__main__":
version = pkg_resources.get_distribution("compare_with_remote").version
split_version = version.split('.')
try:
split_version[-1] = str(int(split_version[-1]) + 1)
except ValueError:
# do something about the letters in the last field of version
pass
new_version = '.'.join(split_version)
os.system("sed -i \"s/version='[0-9.]\+'/version='{}'/\" setup.py"
.format(new_version))
os.system("git add -u")
os.system("git commit -m '[ci skip] Increase version to {}'"
.format(new_version))
os.system("git push")
And change your .travis.yml to
after_success:
- python continuous_integration/increment_version.py
I am not sure about how to make the git push part work as it need some testing with the repo rights but I assume that you could probably setup something to allow travis to push in your repo. you could look into that post for instance.
Also note that I used python to perform the git operations but they can be added as extra line in the after_success field:
after_success:
- python continuous_integration/increment_version.py
- git add -u
- git commit -m "[ci skip] version changed"
- git push
I just find it convenient to put the version number in the commit msg.
Also, it is very important to add [ci skip] in the commit message to avoid infinite increment. Maybe it would be safer to trigger the version change on a specific commit msg tag.
Not Python-specific, but this tutorial explains auto-incrementing version numbers by adding .travis.yaml entries which update git tags with each successful build. It seems like a good balance between manual and auto-versioning.
While the tutorial does use npm's package.json for the initial version-check, you could implement a simple equivalent (in Python or otherwise).
Assuming that,
only commits with successful Travis CI builds get merged into the master branch (e.g. using Pull Requests)
the package is always installed with pip from a git repo using, for instance,
pip install git+https://github.com/user/package.git
For auto-incrementing version number, one could simply define the version as the number of commits in the master branch. This can be done with the following lines in setup.py,
minor_version = check_output(['git', 'rev-list',
'--count', 'master']).decode('latin-1').strip()
setup(version='0.{}'.format(minor_version), ...)
I have forked and checked out the youtube-dl repository from github. I would like to import youtube-dl directly from my git directory without having to install it, to test my changes in the code. Is this possible? How do I do this?
You can simply cd into the project root and then import youtube_dl.
To clarify that this is the version you want instead of pip's version, look at the value of youtube_dl.__file__ to see where the module is located:
In [1]: import youtube_dl
In [2]: youtube_dl.__file__
Out[2]: 'C:\\Users\\Me\\Documents\\youtube-dl\\youtube_dl\\__init__.py'
Take a look at the project readme, the developer instruction section is what you are searching for.
If you want to separate your work in the directory from the system version of the module you could look at virtualenv, a tool to isolate your projects dependencies.
I am facing an issue while cloning a git repo.
I am using function clone_from from GitPython library
from git import Repo
Repo.clone_from("git://github.com/facebook/buck.git", "D:\sample")
I am getting error
WindowsError: The system cannot find the file specified
Can someone please tell me if this is how to clone a repo using the library?
You might not have git.ext in your PATH, but that can easily be tested by executing it yourself.
If you see an error, you can either add it to the PATH, or set the GIT_PYTHON_GIT_EXECUTABLEto the executable git-python should execute for git commandline services.
I tried to uninstall a module on heroku with:
heroku run bin/python bin/pip uninstall whatever
Pip shows the module in the /app tree then claims to have uinstalled the module, but running the same command again shows it installed in the same location in the /app tree.
Is there a way to get pip uinstall to succeed?
Heroku run instantiates a new dyno and runs the command specified in that dyno only. Dynos are ephemeral which is why the results of the pip uninstall don't stick.
Updated 2013-09-30: the current way to clear the virtualenv seems to specify a different python runtime version in runtime.txt as stated on Github and in the Heroku's devcenter reference.
Be aware that Heroku currently "only endorses and supports the use of Python 2.7.4 and 3.3.2" so unless your application supports both Python 2.7.4 and 3.3.2, you may want to test it with the runtime you'll want to switch to (currently available at http://envy-versions.s3.amazonaws.com/$PYTHON_VERSION.tar.bz2, though it shouldn't be an issue to switch e.g. between 2.7.4 and 2.7.3 in most cases).
Thanks #Jesse for your up-to-date answer and to the commenters who made me aware of the issue.
Was up-to-date in ~november 2012 (I haven't since updated the linked buildpack, my pull request was closed and the CLEAN_VIRTUALENV feature was dropped at some point by the official buildpack):
As David explained, you cannot pip uninstall one package but you can purge and reinstall the whole virtualenv. Use the user-env-compile lab feature with the CLEAN_VIRTUALENV option to purge the virtualenv:
heroku labs:enable user-env-compile
heroku config:add CLEAN_VIRTUALENV=true
Currently this won't work because there is a bug. You'll need to use my fork of the buildpack until this get fixed upstream (pull request was closed) :
heroku config:add BUILDPACK_URL=https://github.com/blaze33/heroku-buildpack-python.git
Now push your new code and you'll notice that the whole virtualenv gets reinstalled.
Andrey's answer no longer works since March 23 2012. The new style virtualenv commit moved the virtual env from /app to /app/.heroku/venv but the purge branch wasn't updated to catch up so that you end up with a virtualenv not being in PYTHONHOME.
To avoid reinstalling everything after each push, disable the option:
heroku labs:disable user-env-compile
heroku config:remove CLEAN_VIRTUALENV BUILDPACK_URL
There is now a simpler way to clear the pip cache. Just change the runtime environment, for example from 'python-2.7.3' to 'python-2.7.2', or vice versa.
To do this add a file called runtime.txt to the root of your repository that contains just the runtime string (as show above) in it.
For this to work you need to have turned on the Heroku labs user-env-compile feature. See https://devcenter.heroku.com/articles/labs-user-env-compile
By default virtualenv is cached between deploys.
To avoid caching of packages you can run:
heroku config:add BUILDPACK_URL=git#github.com:heroku/heroku-buildpack-python.git#purge
That way everything will be built from scratch after you push some changes. To enable the caching just remove the BUILDPACK_URL config variable.
Now to uninstall specific package(s):
Remove the corresponding record(s) from the requirements.txt;
Commit and push the changes.
Thanks to Lincoln from Heroku Support Team for the clarifications.
I've created some fabfile recipes for Maxime and Jesse answers that allow re-installing the requirements with one fab command: https://gist.github.com/littlepea/5096814 (look at the docstrings for explanation and examples).
For Maxime's answer I've created a task 'heroku_clean' (or 'hc'), it'll look something like that:
fab heroku_clean
Or using an alias and specifying a heroku app:
fab hc:app=myapp
For Jesse's answer I've created a task 'heroku_runtime' (or 'hr'), it sets heroku python runtime and commits runtime.txt (also creates it if it didn't exist):
fab heroku_runtime:2.7.2
If runtime version is not passed it will just toggle it between 2.7.2 and 2.7.3, so the easiest way to change and commit the runtime is:
fab hr
Then you can just deploy (push to heroku origin) you app and the virtualenv will be rebuilt. I've also added a 'heroku_deploy' task ('hr') that I use for heroku push and scale that also can be user together with 'heroku_runtime' task. This is my preferred method of deploying and rebuilding virtualenv - everything happens in one command and I can choose when to rebuild it, I don't like to do it every time like Maxime's answer suggest because it can take a long time:
fab hd:runtime=yes
This is an equivalent of:
fab heroku_runtime
fab heroku_deploy
i wanted to use the pyfire github library https://github.com/mariano/pyfire
this is what pip freeze produced for me:
-e git+ssh://git#github.com/mariano/pyfire.git#db856bb666c4b381c08f2f4bf7c9ac7aaa233221#egg=pyfire-dev
but when doing so the clone that is being done to install the dependencies fails with error code, i can't
heroku run console
to check out the full error log...
any experience with this or ideas?
thanks a lot in advance
pip freeze appears to produce the wrong results you should be able to modify your requirements.txt to:
git+https://github.com/mariano/pyfire.git#db856bb666c4b381c08f2f4bf7c9ac7aaa233221#egg=pyfire-dev
I realized this was already solved for you, but this didn't work for me or #amrox or #tomtaylor.
If you remove the commit part, it works for me. i.e. changing the line in requirements.txt to:
git+https://github.com/mariano/pyfire.git
When I install the git repo including the specific commit locally, git seems to realize the end part is a specific commit, but when I try this on heroku and track the progress, it is pretty clear it is treating the commit part as a tag. Since there is no tag with that name, that's why it fails. They may be using a different version of git.