I've spent the last few weeks researching the switch from SVN to Git (on Windows) and have found the most efficient method to be using Ruby with svn2git, as it seems to automate a lot of stuff I'd otherwise have to do manually (converting tags, etc). I'm using an alternative remote at the moment for the purpose of testing (so I don't screw something up), but had a few questions:
1. While svn2git does convert SVN tags to Git tags, it seems to leave the SVN tag commits intact, which in a graphical display makes it look like a bunch of short branches. The (WoWAce repo) guide recommends pruning empty commits (which makes sense) but doing so eliminates all of those SVN tag commits. It seems to me that this would be a good thing, as it cleans up the history but I'm not certain if it'll cause some sort of problem. Will it?
2. With all of the aforementioned tags converted to Git, will pushing them (--follow-tags, limiting to -as) force the packager to repackage all of them?
3. The above process leaves "remote (tracking?) branches" tied directly to the SVN repo plus some other SVN garbage. My plan was to push only the master (and tags) and then re-clone it so that my working copy was free of all of the SVN trash. Will that work or do I need to take some extra steps?
I've always used http://www.wowace.com/wiki/repositories/convert-svn-to-git/ without issue. I've seen people who don't use that method run into all sorts of issues (such as the packager repeatedly attempting to re-package tags) so that's all I really have to go on.
I read the comment about the tags issue, but I figured it would have been solved by now. It would seem logical to me to have the tags in place (and everything just how you want it) prior to the initial push. And I couldn't get the tag conversion script to actually do anything. Unless you're suggesting abandoning all irrelevant tags and just clean up the empty commits.
The tags issue hasn't been solved because the packager hasn't been touched in years, and won't be since we're preparing to migrate everything over to the new platform we've been working on which has a much better packager. Check http://beta.wowace.com if you haven't seen it already.
So what, just abandon all the old tags, clean up the empty commits and push it? Because I *can* do without the old tags, but I don't want any residual SVN "garbage" left over. The git-svn method creates more garbage than svn2git (or more aptly, svn2git does better at cleaning up after itself).
The link I gave uses "git svn clone" which creates a new Git directory and clones the Subversion commits into it, so there's really nothing to clean up aside from assigning authors and such via the author map step. I'm not sure if it actually discards old tags or if using that method does something that makes the packager not break on them, but as I said - I've never had an issue with converting Subversion repos to Git that way; Ackis Recipe List, Collectinator, _NPCScan, _NPCScan.Overlay, _Cursor, and others I can't remember at the moment.
Well those repos show all of the original tags so clearly I'm missing something. I've ran the clone script (with the -s switch, since it's a standard lay out [ie, trunk, tags, branches]) but the tag conversion doesn't do anything. I think it has something to do with nesting and not providing a --prefix but I'm too tired to fool with it.
Edit: I was looking through the logs after using svn2git and it does exactly what the wiki guide suggests, including creating git-friendly tags and branches out of its svn counterparts. I'm going to see how it translates (after a bit of cleanup). I'll shoot it to an alternate and reclone it before pushing it here.
Thankfully this is a one-time (per add-on) deal. :p
I've always used http://www.wowace.com/wiki/repositories/convert-svn-to-git/ without issue. I've seen people who don't use that method run into all sorts of issues (such as the packager repeatedly attempting to re-package tags) so that's all I really have to go on.
That last command, the one that's supposed to annotate the Git tags so CurseForge will accept them, does nothing. I'm using the Git For Windows git-bash. There's no error when I input that command, but nothing happens either.
Actually, I only tried to use that command after I'd converted Grid's repository using svn2git and Curse wouldn't let me push it, complaining about the lack of annotation on the tags, so that may be contributing to the problem.
Not really - those docs were submitted by anmoch in early 2009, so I'm not sure if Git has just changed that much over the years or if the original line is malformed.
Okay, after spending several more hours chasing the long tail of Google results and laboriously copying and pasting a series of commands for Every. Single. Tag. to convert them to annotated tags because I couldn't get git-bash for windows to run a shell script, it worked. If I ever convert another old WowAce repo from SVN to Git, the tags do not get to come along.
That said, I won't be surprised at all if the Curse packager now gets stuck in an infinite loop of trying to repackage every tag 400 times...
Also all the tags list me as the commit author, but at this point I'm just done trying to make everything perfect.
Edit:
It did seem to create new ZIPs for every tag, at least once... is that going to cause a problem?
Answer: yes, yes it is going to cause problems. Apparently the packager not only recreated ZIPs for every tag, but did so out of order, resulting in some ancient files being the "most recent" on the download page and in the Curse Client.
I don't have time to go through all the 200-ish files it created and manually click and type through the ridiculous deletion process for every single file. I already reported the project; hopefully file moderators do have a bulk-delete function.
Really tempted to just hard-embed libs in all my addons so I don't have to use the unmaintained and bug-infested Curse packager anymore... it seems to screw up nearly as often as it does anything right. :mad:
The packager is STILL causing problems. Curse.com is currently offering a WotLK version of Grid as the "latest version" and the Files tab on the Wowace.com project page seems to indcate that the packager is re-creating ZIPs for every tag AGAIN.
Is there any way to fix this other than to delete all tags and overwrite the entire repository? This is ridiculous.
I did say that not following the instructions could cause exactly this problem.
I have asked that the working copy that the packager works from be nuked from orbit so future runs will function as "normally" as this packager can - this is what we've had to do in the past for conversions that encountered this problem.
When everything moves to the new platform, further conversions can be done whichever way you choose and the new packager will have no issues.
Phanx: In Git, tags are just pointers to a specific commit. Whereas in SVN, tags actually create their own commit. What I did was abandon the idea that I wanted to include all of the old tags since their only real purpose in Git is to trigger/mark a release. And I neither want, nor need those older "releases".
So there's basically two options:
1. Delete all the old tags and only use tags for current/future releases (what I'm doing).
2. Don't push tags along with your commit pushes. Instead, push only the tags you want packaged, separately. That way, you can keep your old tags locally, if only for the sake of history. You could also just leave them un-pushed until the new packager is brought online.
As far as the actual conversion process, I'm using Ruby and SVN2GIT. It makes it so much easier, as it clones the repo and converts the tags/branches to their git equivalents in one fell swoop. Then I just do some clean-up (included removing the old tags, as I've no use for them once I've logged them. But that's optional), push it to a staging repo on BitBucket, clone it back down so that the repo is clean of all the SVN stuff and then push it to WoWAce/GitHub.
Torhal: So with new packager, is it set up to only create packages if the current tag/commit is actually newer and is a higher "version"? Because rebasing dates is a pain in the ass. :P
Random blurb: Rebasing with SourceTree is SO much easier than using Bash if you need to fix typos, etc. Then you can follow it with a blanket rebase using the --committer-date-is-author-date flag for historical goodness. :)
Torhal: So with new packager, is it set up to only create packages if the current tag/commit is actually newer and is a higher "version"? Because rebasing dates is a pain in the ass. :P
If, for some reason, you push a bunch of tags, the new packager will only package the latest 10 by date in the repository. Of course, just like the current packager, you could just turn off packaging before pushing and then turn it back on.
If, for some reason, you push a bunch of tags, the new packager will only package the latest 10 by date in the repository. Of course, just like the current packager, you could just turn off packaging before pushing and then turn it back on.
Unfortunately, that still confused the old packager.
Once you turn packaging back on, it would get into its usual scheme of building broken old packages all the time.
Unfortunately, that still confused the old packager.
Once you turn packaging back on, it would get into its usual scheme of building broken old packages all the time.
Good thing it's got a long-overdue spanking coming. :p
I did say that not following the instructions could cause exactly this problem.
Right, but the instructions don't work as written, so there's simply no way to follow them exactly. I posted about this and your reply was "well, some other guy wrote that years ago, so who knows", so I did my best with the methods I could find that actually worked.
2. Don't push tags along with your commit pushes. Instead, push only the tags you want packaged, separately. That way, you can keep your old tags locally, if only for the sake of history. You could also just leave them un-pushed until the new packager is brought online.
This sounds miserable, as no GUI client for Git I've ever seen has this functionality -- they all just "push tags". If I wanted to type crap on the command line all the time, I'd use Linux. My first several computers ran DOS, and I don't see why anyone would voluntarily go back to that for basic everyday functionality. :p
As far as the actual conversion process, I'm using Ruby and SVN2GIT. It makes it so much easier, as it clones the repo and converts the tags/branches to their git equivalents in one fell swoop.
Yes, that's what I did, but I had to do some extra work to glue the pre-Curse and post-Curse parts of the SVN history together. I suspect that may be what made the packager die, but it just shouldn't be that hard. I just re-discovered jlam's packaging script, so as soon as I can get that to work on Windows, I will probably just turn off the Curse packager entirely. Despite hearing "it's being worked on!" for years, it's still broken just as often as not. If you guys are that strapped for cash to pay for developer time, just stop giving out so many author reward points... I'd rather have a system that Just Works and get less free Amazon gift cards, than one that makes me want to strangle someone every time I have to interact with it.
Right, but the instructions don't work as written, so there's simply no way to follow them exactly. I posted about this and your reply was "well, some other guy wrote that years ago, so who knows", so I did my best with the methods I could find that actually worked.
The cloning script works, but the branch/tag conversion script doesn't.
I think I used:
git for-each-ref --format="%(refname:short) %(objectname)" refs/remotes/tags \
| while read BRANCH REF
do
TAG_NAME=${BRANCH#*/}
BODY="$(git log -1 --format=format:%B $REF)"
echo "ref=$REF parent=$(git rev-parse $REF^) tagname=$TAG_NAME body=$BODY" >&2
git tag -a -m "$BODY" $TAG_NAME $REF^ &&\
git branch -r -d $BRANCH
done
This sounds miserable, as no GUI client for Git I've ever seen has this functionality -- they all just "push tags". If I wanted to type crap on the command line all the time, I'd use Linux. My first several computers ran DOS, and I don't see why anyone would voluntarily go back to that for basic everyday functionality. :p
SourceTree, from Atlassian (BitBucket.org guys), actually has an option to not push tags.
Despite hearing "it's being worked on!" for years, it's still broken just as often as not. If you guys are that strapped for cash to pay for developer time, just stop giving out so many author reward points... I'd rather have a system that Just Works and get less free Amazon gift cards, than one that makes me want to strangle someone every time I have to interact with it.
The new packager isn't being worked on; it's done - WildStar authors have been using it since August of last year.
What is "being worked on!" is feature parity with the legacy platform to support World of Warcraft, which isn't the only game we need to consider. During the development of the new platform, we've had to address the requirements for the author communities of Minecraft, Kerbal Space Program, and WildStar, while ensuring interoperability with the curse.com and the Curse Client.
Being a WoW author myself, I understand your frustration - I, too, want to be rid of the badly-aging beast that is the current CurseForge. This isn't a WordPress site that we can just slap together in a day, however, and the new platform is internally a completely different beast from the old; there are a plethora of subsystems and details to consider, so the process hasn't been as straightforward as you seem to be implying.
That said, we're in the home stretch for the migration and initial relaunch. After that, we'll be actively iterating to bring in improvements, features we didn't get around to adding for the relaunch, and features we never had before.
Rollback Post to RevisionRollBack
To post a comment, please login or register a new account.
1. While svn2git does convert SVN tags to Git tags, it seems to leave the SVN tag commits intact, which in a graphical display makes it look like a bunch of short branches. The (WoWAce repo) guide recommends pruning empty commits (which makes sense) but doing so eliminates all of those SVN tag commits. It seems to me that this would be a good thing, as it cleans up the history but I'm not certain if it'll cause some sort of problem. Will it?
2. With all of the aforementioned tags converted to Git, will pushing them (--follow-tags, limiting to -as) force the packager to repackage all of them?
3. The above process leaves "remote (tracking?) branches" tied directly to the SVN repo plus some other SVN garbage. My plan was to push only the master (and tags) and then re-clone it so that my working copy was free of all of the SVN trash. Will that work or do I need to take some extra steps?
Thanks! :)
Edit: I was looking through the logs after using svn2git and it does exactly what the wiki guide suggests, including creating git-friendly tags and branches out of its svn counterparts. I'm going to see how it translates (after a bit of cleanup). I'll shoot it to an alternate and reclone it before pushing it here.
Thankfully this is a one-time (per add-on) deal. :p
That last command, the one that's supposed to annotate the Git tags so CurseForge will accept them, does nothing. I'm using the Git For Windows git-bash. There's no error when I input that command, but nothing happens either.
Actually, I only tried to use that command after I'd converted Grid's repository using svn2git and Curse wouldn't let me push it, complaining about the lack of annotation on the tags, so that may be contributing to the problem.
Any thoughts?
That said, I won't be surprised at all if the Curse packager now gets stuck in an infinite loop of trying to repackage every tag 400 times...
Also all the tags list me as the commit author, but at this point I'm just done trying to make everything perfect.
Edit:
It did seem to create new ZIPs for every tag, at least once... is that going to cause a problem?
I don't have time to go through all the 200-ish files it created and manually click and type through the ridiculous deletion process for every single file. I already reported the project; hopefully file moderators do have a bulk-delete function.
Really tempted to just hard-embed libs in all my addons so I don't have to use the unmaintained and bug-infested Curse packager anymore... it seems to screw up nearly as often as it does anything right. :mad:
Is there any way to fix this other than to delete all tags and overwrite the entire repository? This is ridiculous.
I have asked that the working copy that the packager works from be nuked from orbit so future runs will function as "normally" as this packager can - this is what we've had to do in the past for conversions that encountered this problem.
When everything moves to the new platform, further conversions can be done whichever way you choose and the new packager will have no issues.
Phanx: In Git, tags are just pointers to a specific commit. Whereas in SVN, tags actually create their own commit. What I did was abandon the idea that I wanted to include all of the old tags since their only real purpose in Git is to trigger/mark a release. And I neither want, nor need those older "releases".
So there's basically two options:
1. Delete all the old tags and only use tags for current/future releases (what I'm doing).
2. Don't push tags along with your commit pushes. Instead, push only the tags you want packaged, separately. That way, you can keep your old tags locally, if only for the sake of history. You could also just leave them un-pushed until the new packager is brought online.
As far as the actual conversion process, I'm using Ruby and SVN2GIT. It makes it so much easier, as it clones the repo and converts the tags/branches to their git equivalents in one fell swoop. Then I just do some clean-up (included removing the old tags, as I've no use for them once I've logged them. But that's optional), push it to a staging repo on BitBucket, clone it back down so that the repo is clean of all the SVN stuff and then push it to WoWAce/GitHub.
Torhal: So with new packager, is it set up to only create packages if the current tag/commit is actually newer and is a higher "version"? Because rebasing dates is a pain in the ass. :P
Random blurb: Rebasing with SourceTree is SO much easier than using Bash if you need to fix typos, etc. Then you can follow it with a blanket rebase using the --committer-date-is-author-date flag for historical goodness. :)
If, for some reason, you push a bunch of tags, the new packager will only package the latest 10 by date in the repository. Of course, just like the current packager, you could just turn off packaging before pushing and then turn it back on.
Unfortunately, that still confused the old packager.
Once you turn packaging back on, it would get into its usual scheme of building broken old packages all the time.
Good thing it's got a long-overdue spanking coming. :p
Right, but the instructions don't work as written, so there's simply no way to follow them exactly. I posted about this and your reply was "well, some other guy wrote that years ago, so who knows", so I did my best with the methods I could find that actually worked.
This sounds miserable, as no GUI client for Git I've ever seen has this functionality -- they all just "push tags". If I wanted to type crap on the command line all the time, I'd use Linux. My first several computers ran DOS, and I don't see why anyone would voluntarily go back to that for basic everyday functionality. :p
Yes, that's what I did, but I had to do some extra work to glue the pre-Curse and post-Curse parts of the SVN history together. I suspect that may be what made the packager die, but it just shouldn't be that hard. I just re-discovered jlam's packaging script, so as soon as I can get that to work on Windows, I will probably just turn off the Curse packager entirely. Despite hearing "it's being worked on!" for years, it's still broken just as often as not. If you guys are that strapped for cash to pay for developer time, just stop giving out so many author reward points... I'd rather have a system that Just Works and get less free Amazon gift cards, than one that makes me want to strangle someone every time I have to interact with it.
The cloning script works, but the branch/tag conversion script doesn't.
I think I used:
Which preserves the original commit message, etc.
SourceTree, from Atlassian (BitBucket.org guys), actually has an option to not push tags.
:)
The new packager isn't being worked on; it's done - WildStar authors have been using it since August of last year.
What is "being worked on!" is feature parity with the legacy platform to support World of Warcraft, which isn't the only game we need to consider. During the development of the new platform, we've had to address the requirements for the author communities of Minecraft, Kerbal Space Program, and WildStar, while ensuring interoperability with the curse.com and the Curse Client.
Being a WoW author myself, I understand your frustration - I, too, want to be rid of the badly-aging beast that is the current CurseForge. This isn't a WordPress site that we can just slap together in a day, however, and the new platform is internally a completely different beast from the old; there are a plethora of subsystems and details to consider, so the process hasn't been as straightforward as you seem to be implying.
That said, we're in the home stretch for the migration and initial relaunch. After that, we'll be actively iterating to bring in improvements, features we didn't get around to adding for the relaunch, and features we never had before.