I'm looking for a Python library that can do basic manipulation of repositories, but is independent of the backend version control system.
By basic manipulation, I'm referring to: initialize a repo, add files, commit, pull, push, get current revision number.
Users of the library could do something this:
import dvcs_wrapper as dvcs
dvcs.set_backend('hg') # could choose 'git', 'bzr'
repo = dvcs.init('/home/me/my_repo')
repo.add('/home/me/my_repo/*.py')
repo.commit('Initial commit')
repo.push('http://bitbucket.org/....')
print('At revision %d' % repo.revision_num)
Any pointers to something like the above? My Google searches turn up nothing...
Update: for what it's worth, I've started working on something like this: code is here with unit tests
for Hg repositories. I might get around to Git and Bazaar; contributions welcome.
There's also the VCS module, which advertises:
vcs is abstraction layer over various version control systems. It is
designed as feature-rich Python library with clean API.
I think you are out of luck.
There are Python wrappers for git but according to this the quality is still less than optimal. Hg and bzr are Python projects but their infrastructure is quite different, so API level integration is not easy. Also different SCMs have different design philosophies, which makes a unified wrapper less plausible.
That being said, if you do need a simple wrapper, you can use the subprocess module and wrap the command lines to get the result you want.
Related
Hi I have a simple pyQt text editor,
Essentially I want to add mercurial support
I have seen in various other editors the ability to support a number of DVCS (Mercurial, GIT,Bazaar, etc), and they give the user the ability to perform functions like commit,update, etc
I really want to know what/how I can integrate mercurial in my pyQt text editor, so that it behaves more or less like other fancy Editors.
Any good tutorials/guides on how to get this done
There are no tutorials around this, generally however there are three approaches:
Command line interface
Mercurials command line interface is considered stable. That means that you can expect Mercurial without extensions to not change the output of a command. Using "-T json" for most commands will also result in an easily parsable Json output. This approach is robust and fairly easy to implement as you only have to call out to Mercurial and parse the json back. Most standard commands like commit, log, etc should be implementable using this
hglib
Mercurial is offering hglib. A library that is available in C and Python which allows you to interface with Mercurial via a local protocol. Mercurial will be started in server mode and you use the library to interact. This approach is also very stable, offers a better abstraction, but relies on the command server being available and implies potential API changes in the library. Note that you also have to take the license of the library into account as you are linking against them.
Embedding Mercurial
Python processes can embedd Mercurial directly by important the right modules. However the Mercurial API is internally not stable and subject to continuous change. This option offers you the most flexibility as you have access to everything, including low-level parsing of datastructures, exposing of hidden functionality such as obsolence markers. The drawbacks are: 1. you have to know what to do otherwise you might corrupt the repository 2. the api changes all the time 3. you are subject to the GPL license.
I am looking for a better patch tool than the ones built into Mercurial, or a visual tool to help me edit patches so that they will get accepted into Mercurial or Gnu patch.
The mercurial wiki has a topic HandlingRejects which shows how simple it is to have Patching fail. I am looking to implement a mercurial based workflow for feature branch version control that is based on feature branches, and relies on exporting and reviewing patches before they are integrated. In my initial testing of this idea, the weakest link in my "patch out", and "review and accept and modify" patches is the way that patch rejection shuts me down.
Here are some common cases where mercurial patch imports fail on me:
Trivial changes to both upstream repoA and feature branch repoB, where a single line is added somewhere, on both branches. Since both branches have a history, it should be possible for a merge tool to see that "someone added one line in repoA, and someone else added one line in repoB". In the case of patch imports, though, this results in a patch import reject and a .rej file turd in your repository which you ahve to manually repair (by editing the .rej file until it can be applied).
The wiki page above mentions the mpatch tool that is found here. I am looking for other Better Merge Tools that (a) work with mercurial, and (b) can handle the trivial case noted in the Handling Rejects wiki page above. Note that mpatch does not work for my purposes, it seems I need something that is more of a rebase tool than a patch tool, and in my case, I may have to make the patch tool be syntax-aware (and thus specific to a single programming language).
I am looking for tools available for working on Windows, either natively, or even via something like cygwin. I am not using a Unix/Linux environment, although I am comfortable with Linux/Unix style tools.
I am not currently using the mq extensions, just exporting ranges of changes using hg export, and importing using hg import, and the rest of the work is my own inventions, however I have tagged this mq, as mq users will be familiar with this .rej handling problem.
Related question here shows ways of resolving such problems while using TortoiseHg.
Emacs is quite capable of handling .rej files.
However, if at all practical, I try to use hg pull --rebase whenever possible. Often I find myself wanting to rebase some lineage of patches onto another changeset that I've already pulled. In these cases I just strip the changeset and pull it in again from .hg/strip-backup, allowing me to use --rebase.
I want to write a module in python (This is the learning project) to enhance my git experience. Is there a python module for various git commands? At least the basic ones (commit/diff/log/add)?
I saw GitPython but I couldn't find the support for (new) commits; its more of a repo browsing framework than a complete GIT interface. (Or did I miss something?)
Also, if there IS a python module for all this, will that be preferable, or executing the shell commands from python code?
In GitPython you create a commit from an index object.
In libgit2 you create a commit from a repository object.
You might also want to look at this question:
Python Git Module experiences?
I think some python source could help beginners like me not to waste precious time on digging docs.
All commits will go to freshly created origin master
Here it is:
from git import Repo
import os
path = '/your/path/here'
if not os.path.exists(path):
os.makedirs(path)
os.chdir(path)
repo = Repo.init(path).git
index = Repo.init(path).index
for x in xrange (1,10):
fname = 'filename' + str(x)
f.open(fname, 'wb+')
f.write()
f.close()
repo.add(fname)
index.commit("initial commit")
Git is designed to consist of "plumbing" and "porcelain". Plumbing components form the foundation, low-level system: Managing objects, repositories, remotes, and so on. Porcelain, on the other hand, means more user-friendly high-level tools that use the plumbing.
Historically, only the most basic/performance-critical parts (mostly plumbing) were implemented in C, the rest used shell/perl scripts. To be more portable, more and more code was rewritten in C.
With this background, I would recommend to just use system calls to the git executable for your python wrapping. Consider your code as part of Git's porcelain. Compared to using a specialized library:
PRO
No need to learn an API -- use the git commands you are familiar with
Complete set of tools -- you can use porcelain and are not restricted to low-level functionality
CONTRA
Need to parse command line output from git calls.
Might be slower
This can be done with GitPython
Install it with:
pip install GitPython
And use it like this:
from git.repo import Repo
repo = Repo('/path/to/repository')
repo.index.add(['some_file'])
repo.index.commit('commit from python')
origin = repo.remotes[0]
origin.push()
Learn more in the documentation.
I'm searching for a library that can extract (at least) the following information from a SVN repository (not a working copy!):
Revision numbers and their author & commit message
Changes in each revision (added, deleted, modified files)
Is there a Python library that can do this?
For the authors and commit messages, I could parse "db/revprops/0/..." (simple format), but looking for changed files does not seem so easy, so I'd rather stick with a library that supports SVN repos.
There are Python bindings to libsvn: http://pysvn.tigris.org/docs/pysvn.html. They facilitate doing pretty much everything the svn command line client can do.
In particular, the Client.log() method does what you are looking for.
I think you want something like py-svn.
One issue that comes up during Pinax development is dealing with development versions of external apps. I am trying to come up with a solution that doesn't involve bringing in the version control systems. Reason being I'd rather not have to install all the possible version control systems on my system (or force that upon contributors) and deal the problems that might arise during environment creation.
Take this situation (knowing how Pinax works will be beneficial to understanding):
We are beginning development on a new version of Pinax. The previous version has a pip requirements file with explicit versions set. A bug comes in for an external app that we'd like to get resolved. To get that bug fix in Pinax the current process is to simply make a minor release of the app assuming we have control of the app. Apps we don't have control we just deal with the release cycle of the app author or force them to make releases ;-) I am not too fond of constantly making minor releases for bug fixes as in some cases I'd like to be working on new features for apps as well. Of course branching the older version is what we do and then do backports as we need.
I'd love to hear some thoughts on this.
Could you handle this using the "==dev" version specifier? If the distribution's page on PyPI includes a link to a .tgz of the current dev version (such as both github and bitbucket provide automatically) and you append "#egg=project_name-dev" to the link, both easy_install and pip will use that .tgz if ==dev is requested.
This doesn't allow you to pin to anything more specific than "most recent tip/head", but in a lot of cases that might be good enough?
I meant to mention that the solution I had considered before asking was to put up a Pinax PyPI and make development releases on it. We could put up an instance of chishop. We are already using pip's --find-links to point at pypi.pinaxproject.com for packages we've had to release ourselves.
Most open source distributors (the Debians, Ubuntu's, MacPorts, et al) use some sort of patch management mechanism. So something like: import the base source code for each package as released, as a tar ball, or as a SCM snapshot. Then manage any necessary modifications on top of it using a patch manager, like quilt or Mercurial's Queues. Then bundle up each external package with any applied patches in a consistent format. Or have URLs to the base packages and URLs to the individual patches and have them applied during installation. That's essentially what MacPorts does.
EDIT: To take it one step further, you could then version control the set of patches across all of the external packages and make that available as a unit. That's quite easy to do with Mercurial Queues. Then you've simplified the problem to just publishing one set of patches using one SCM system, with the patches applied locally as above or available for developers to pull and apply to their copies of the base release packages.
EDIT: I am not sure I am reading your question correctly so the following may not answer your question directly.
Something I've considered, but haven't tested, is using pip's freeze bundle feature. Perhaps using that and distributing the bundle with Pinax would work? My only concern would be how different OS's are handled. For example, I've never used pip on Windows, so I wouldn't know how a bundle would interact there.
The full idea I hope to try is creating a paver script that controls management of the bundles, making it easy for users to upgrade to newer versions. This would require a bit of scaffolding though.
One other option may be you keeping a mirror of the apps you don't control, in a consistent vcs, and then distributing your mirrored versions. This would take away the need for "everyone" to have many different programs installed.
Other than that, it seems the only real solution is what you guys are doing, there isn't a hassle-free way that I've been able to find.