cbrad

Google has announced support for Mercurial on Google Code. Currently it is a preview release for a limited set of users.

There is a Google analysis of Git and Mercurial here.

The Challenge
I have one large (25GB) Subversion repository that partly has a structure like this:

/brad
 /docs
 /finances
 /foo
 ...

I wish to convert the docs subtree (including history) into its own Git repository, without the foo directory.

The Solution
One way to achieve this would have been to dump the repository, filter the history, create a new repository, load the filtered history and then convert with git-svnimport.

Instead, I did the following:

1. Convert the docs subtree into a Mercurial repository, excluding the foo directory.
$ hg convert --filemap filemap --config convert.hg.usebranchnames=False file:///path/to/svnrepos/brad/docs docs-hg

filemap is a file in the current directory with only one line in it:
exclude foo

The effect of --config onvert.hg.usebranchnames=False is to import onto the default branch in Mercurial. Without it, a docs branch would have been used and carried over to Git in the subsequent steps. I wish the final Git repository to just have the conventional master branch.

2. Convert the Mercurial repository to Git.
$ mkdir docs
$ cd docs
$ git init

I installed Mercurial via MacPorts, so to get fast-export to work, I needed to use the right Python:
$ export PYTHON="/opt/local/bin/python2.5"
$ /path/to/fast-export/hg-fast-export.sh -A ../authors.txt -r ../docs-hg

The -A ../authors.txt simply maps the Subversion commit username to a normal Git author format. Same as git-svnimport.

$ git checkout master

3. Remove the intermediate Mercurial repository:
$ cd ..
$ rm -rf docs-hg

I did a diff of the docs subdirectories in the Subversion and Git working copies and did a quick check of the history. Looks like it worked successfully.

I have decided to move from Subversion to a distributed VCS and have been considering Git and Mercurial. I have settled on Git for the following reasons:

• Git seems more granular. I expect this to provide more flexibility to adapt to different circumstances, but at a greater learning time cost.


• The way tags are managed in Mercurial (.hgtags) looks a bit odd.


• The notion Git has of tracking content rather than files is interesting, although I don't understand the ramifications yet.

To be fair, either Mercurial or Git would be suitable for my current needs. Mercurial was initially more attractive as it seemed simpler to get up and going and the Subversion import works better on my existing repository.

There were a couple of interesting posts I found along the way: Experimenting with Git and The Differences Between Mercurial and Git.

John Goerzen has been writing about switching from Mercurial to Git recently on his blog.

I was unsuccessful in my last attempts at converting from a Subversion repository to Mercurial.

Yesterday I thought I would give it another go. I setup a new local Subversion repository and imported a dump from one of my existing repositories. In that repository I have a fairly new project (let's call it foo) which has the standard Subversion layout (trunk, tags, branches), plus an extra directory named artifacts. There are no branches, 2 tags and about 27 commits.

$hg convert file:///path/to/repos/foo

Completed with no errors.

That looked promising, so it was time to try the biggest project in the same repository (let's call it bar). It has 1 branch, 13 tags and about 464 commits.

$hg convert file:///path/to/repos/bar
assuming destination bar-hg
initializing destination bar-hg repository
scanning source...
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 0.9.5)
Traceback (most recent call last):
File "/opt/local/bin/hg", line 14, in ?
mercurial.dispatch.run()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 20, in run
sys.exit(dispatch(sys.argv[1:]))
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 29, in dispatch
return _runcatch(u, args)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 45, in _runcatch
return _dispatch(ui, args)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 348, in _dispatch
ret = _runcommand(ui, options, cmd, d)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 401, in _runcommand
return checkargs()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 357, in checkargs
return cmdfunc()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 342, in
d = lambda: func(ui, *args, **cmdoptions)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 380, in convert
c.convert()
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 257, in convert
parents = self.walktree(heads)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 97, in walktree
commit = self.cachecommit(n)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 204, in cachecommit
commit = self.source.getcommit(rev)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 245, in getcommit
self._fetch_revisions(from_revnum=revnum, to_revnum=stop)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 620, in _fetch_revisions
for entry in self.get_log([self.module], from_revnum, to_revnum):
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 275, in get_log
for p in parent(stdout):
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 262, in parent
raise SubversionException("child raised exception", entry)
libsvn._core.SubversionException: ('child raised exception', 160013)

Not so good. However, if I just try converting the trunk instead of the whole project:

$hg convert file:///path/to/repos/bar/trunk

The conversion completes with no errors.

In summary, trying to convert projects in Subversion to Mercurial has not gone smoothly. Doing a quick google search reveals a recent Mercurial trac issue that looks similar. Time to put this aside for now and switch to evaluating git.

My first attempt at this was the hard way. The easier way is to use MacPorts:

$sudo port install subversion subversion-python24bindings

Unfortunately though, I still get the same errors as in my first attempt, when trying to convert the same Subversion repositories.

If I am to switch over to Mercurial, I need to be able to import my existing projects that are in Subversion. Mercurial includes a Convert extension for this purpose, so I thought I would try and get it going on my Mac.

[EDIT: This is the hard way. See this post , for the easy way.]

I already had Subversion 1.4.3 installed (compiled from source), but didn't have the Python bindings installed, so I used this article and the instructions in the file  subversion/bindings/swig/INSTALL, located in the Subversion source.

I thought I should build against the same version of Python as Mercurial was using. As I had installed Mercurial from MacPorts, Python 2.4 was located under /opt/local/.  This meant my configure statements were as follows.

swig:
$./configure --with-python=/opt/local/bin/python2.4

Subversion:
$./configure --prefix=/usr/local --with-openssl --with-ssl --with-zlib --enable-swig-bindings=python PYTHON=/opt/local/bin/python2.4

The bindings need to be in the python path, so I also created the file:
/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/subversion.pth

with the contents:
$ cat subversion.pth
/usr/local/lib/svn-python

To confirm the bindings are installed correctly, the following command should execute with no output:
$/opt/local/bin/python2.4 -c "from svn import client"

I am not particularly happy with this installation. Compiling source that depends on a port makes me feel uncomfortable. Perhaps it is because I am unfamiliar with MacPorts or just the fact that I have to remember that this dependency exists and not to do anything that may affect it. I also installed swig 1.3.33 (the latest), however the Subversion docs only refer up to version 1.3.29.

So far I have been unable to successfully complete a conversion of a Subversion repository. Trying the example from the Mercurial docs starts working, but after a little while fails:
$ hg convert http://code.sixapart.com/svn/memcached
assuming destination memcached-hg
initializing destination memcached-hg repository
scanning source...
sorting...
converting...
710 Fix for flush_all 1-second window bug. You can now do a "set" immediately
...
698 cleanup, cleanup, cleanup, cleanup.
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 0.9.5)
Traceback (most recent call last):
File "/opt/local/bin/hg", line 14, in ?
mercurial.dispatch.run()
...
File "/usr/local/lib/svn-python/libsvn/ra.py", line 511, in svn_ra_check_path
return apply(_ra.svn_ra_check_path, args)
libsvn._core.SubversionException: ("PROPFIND request failed on '/branches/client-xs-20070328/Cache-Memcached-GetParserXS/const-c.inc'", 175002)

Also tried converting one of the work Subversion repositories. It starts working, but after a little while fails with a different error:
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 0.9.5)
Traceback (most recent call last):
File "/opt/local/bin/hg", line 14, in ?
mercurial.dispatch.run()
...
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 267, in convert
if "\n" in desc:
TypeError: iterable argument required

Haven't tried converting a local Subversion repository yet.

Thought I would jot down my experiences installing Mercurial on a Mac running Tiger. Mac’s are great for some things, but they can be a bit of a pain when it comes to installing unix open source software.

The first thing I tried was installing the latest binary package from the Mercurial site. It is dependent on python, so first I installed MacPython 2.5 by following the link on the Mercurial Page. Then I tried installing the Mercurial binary package, but it complained because it couldn’t find Apple Python. Tried rebooting, didn’t work, go figure.

The second thing I tried was building from source. This is my preferred way of installing this kind of software, because you can explicitly manage the versions yourself. However the build failed because I didn’t have the documentation tools installed (these can be installed via MacPorts). Also, I didn’t seem to have my python path set up properly, because the built hg executable barfed with an error.

The third way I tried was installing via MacPorts. This worked very easily, once MacPorts was installed. The current version of MacPorts required a newer version of Xcode tools than what was already installed. So it was off to the apple developer site (I already had a login) to download the latest version, which is about 1GB.

IMHO, to have to sign up to an Apple site to get the current C compiler so you can build open source software is really poor form from Apple.