As Nelly would say, hurrr we go.
I’ve tested the script several times in the last few days and all the bugs I’ve come across have been quashed, I’m pleased to say. The trickiest one had to do with something caused by that most evil of Clearcase objects, the symlink. Basically, if you have a symlink that you break and replace with an identically named directory/file, it’s difficult for the script to reflect the removal-and-readdition in a single revision. Thankfully, the Subversion dump format took that into account (which I find remarkably foresightful), and I fixed that. The script’s pretty heavily commented, so you can see how, if you’re interested :)
Anyway, the script’s attached, with a detailed set of instructions in the README.txt. Please feel free to send me any questions or comments.
Update: Final, confirmed numbers below. This is for a project where the latest Clearcase snapshot is about 187MB in size.
- Dumping: 257 revisions took 2:23:21, and yielded 8.22GB of data
- Loading: 257 revisions took 2:44:51, and yielded a repository 321MB in size
Also, a little note: the label-importing idea can achieve arbitrary granularity by use of the ‘time’ rule in cspecs. In fact, it might be worthwhile to extend this script to generate and set cspecs based on the time rule instead of labelling, which would save a lot of labelling time for those projects that don’t label regularly and religiously.
Clearcase-to-Subversion Converter: cc2svn.zip
Update:
Matej Spiller-Muys sent in some changes that look quite useful:
- Each label only contains files that have changed (checksum matching)
- After each label dump check for removed files (revision number matching)
- Works with windows PERL (not cygwin, problem with CRLF) – you have specified “os: win” inside yml file.
- Problem with –fmt on windows: this works –fmt \”%Nd\t%n\t%u\n\”"
- Delete dump file before dumping
Unfortunately I haven’t had a chance to look through the script and digest the changes myself, so I’m leaving the old script here and adding his updated one below. If you are able to use it and have no issues please let me know and I’ll get around to deleting the old one. Thanks Matej!
Updated Clearcase-to-Subversion Converter (script only): cc2svn.p
Vamsi Hari wrote in with questions about why the script wouldn’t work on his Windows installation. The answer was pretty standard: line terminations. I’ve converted the files to CRLF, so that should no longer be an issue.
Other things that Vamsi’s email highlighted:
1) The ‘vob already mounted’ message is a warning (as opposed to a fatal error) so it can be ignored.
2) On Windows, Clearcase is often installed in =C:\Program Files=, which is problematic because of the space between ‘Program’ and ‘Files’. If that is the case for your installation, remember to put the path in double-quotes when you update the the config file.
How about migrating clearcase to subversion from separate windows boxes?
Not sure what you mean. Let’s assume that your project is composed of 3 VOBs and you have views on 3 boxes that look at one of the VOBs each. In that case, to use this script you would create a source view on one of those machines that has a config spec that loads up all 3 VOBs and then use the script as usual. There isn’t any special jiggery required with the config spec — you could just concatenate the specs you’re using for the 3 partial views.
It seems the config file is a little wrong — there’s an entry called ‘clearcase’, which should actually be called ‘cleartool’.
Thanks to Samuel for asking about this.
Hi,
I’m trying to convert from cc to svn in a Windows environment… Below, is the error I’m getting, and my cc2svn.yml config file… Can you tell me what I’m doing wrong…???
cleartool : “C:Progra~1RationalClearCasebincleartool.exe”
svnadmin : C:SVNsvnadmin.exe
source_view : comerica_fwd_dev
source_view_path : C:CruiseControlprojectscomericacomerica_fwd_dev
vobs :
– EP_US_PSO_2.X_REL
– EP_1X_REL
– EP_US_PSO_2.X
labels_to_include :
– “^REL_”
nodes_to_exclude :
– “lost found”
– “.labellist”
repository_path : C:repositoriescomerica
dump_dir : “C:Docume~1cstylestmpdump”
out_dir : “C:Docume~1cstylestmpout”
tmp_dir : “C:Docume~1cstylestmp”
C:cc2svn>cc2svn.pl -c cc2svn.yml -mode d
Initializing … cleartool: Error: View tag not found: “comerica_fwd_dev”.
cleartool: Error: /vobs/EP_US_PSO_2.X_REL is not a registered VOB tag.
cleartool: Error: /vobs/EP_1X_REL is not a registered VOB tag.
cleartool: Error: /vobs/EP_US_PSO_2.X is not a registered VOB tag.
done.
Compiling label list … cleartool: Error: Conflicting arguments given.
done (see C:Docume~1cstyles mpout/label_list.txt).
Finished, in 0 seconds.
Hmm, that’s coming from ‘cleartool startview comerica_fwd_dev’. I’ll be you won’t be able you run that successfully in a command prompt because it doesn’t look like a dynamic view.
It’s best to run the script in a dynamic view, the reason being that dumping CC history means changing config specs a lot; in a snapshot view that will mean lots of error-prone updating of files. That’s not to say it can’t be done, but you will have to get your hands dirty a bit, starting with commenting out the line below in the script:
Good luck!
Thanks… I think you answered my question… I actually want to run it on in a dynamic view… I was just trying to run it on my local box in a snapshot view before running it in a dynamic view on my source server, just to see how it works… I do have a couple more questions… Do I have to modify the cc2svn.pl script, or can I just run it as is after putting the correct values for my dynamic view in my cc2svn.yml config file…??? Also, I think I already know the answer to my next question, but I just need confirmation… I have to install the SVN server on my source server in order for the cc2svn.pl script to work, correct…???
I went through the script, and what I gather from it is it pulls the files from CC, converts them to SVN format, then dumps them into an SVN repository that gets created during the execution of the script, correct…??? If you could answer these question for me I really appreciate it, as I have a conversion that I need to have done by this coming Wednesday (July 16, 2008)… Also, if you could give me any information you have that you think can help me with this I’d surely appreciate it… Thanks again…
If you’re running it in a dynamic view, you don’t need to mess with the script, just set up your config file and run it.
No need to run this on your ’source server’, just set up a dynamic view on your local box and run it there.
Everything else is explained in the README. The key points are that it won’t modify your Clearcase repository and that by default it doesn’t check anything into your Subversion repository, so there’s no harm in playing with the script a few times. If you don’t understand the README, I recommend you ask someone with more experience to help you.
Thanks… I understand the README just fine, but there were a few things I wasn’t sure about… Thanks for the help…
I am trying to migrate ClearCase into Subversion. However, the instances of ClearCase and Subversion are hosted on different servers. My idea was to not utilize the “load” mode of the script but have it merely create the dumpfile. From here, I would ftp the dumpfile to the server which subversion is hosted on and then use the svnadmin load command.
My conern/question lies with the config file which requires paths to the svn repository as well as svnadmin.exe. Are these only necessary if using the “load” mode? Will it still work corectly if I leave these fields blank or invalid?
Thanks
Yeah, should be fine.