At work, my team has been working toward converting a chunk of our application to be used as NuGet packages rather than direct project references. I wanted to document some of the issues we’ve been having with the conversion since it’s far more complex than most of the other ones I’ve seen documented. Hopefully others doing similar conversions will find it useful so they can avoid some of the pitfalls we have run into.
Let me start by describing the system as it is. We have roughly 4 million lines of C#, 450+ projects and 200 solutions. One of the solutions is supposed to represent everything; the others represent different vertical slices of the overall application. This produces ~300 console applications, ~50 web endpoints, a couple of old winforms applications and some COM components. This is worked on by roughly 20 different teams, spread across 5 or 6 different locations. It also has, like many complex software applications, a large number of little pieces that are still used but nobody is looking at day to day.
So the thing that jumps out from the previous paragraph is the “supposed to.” We found a lot of things that weren’t in that solution when we started our NuGet conversion, so we were just surprised we hadn’t broken them at some point. It’s also just a lot of code to work with as one unit, compiling it all together and running the minimal subset of tests locally precommit is highly onerous and takes about 4 hours.
The Initial Plan
- Convert the library at the base of the dependency tree to have all of its dependencies as NuGet packages.
- Lock the library in SVN, and migrate to git.
- Set up the CI system to build and publish the library as a package.
- Repeat steps 1-3 for other libraries that have a dependency on the initial library.
- Let other teams convert their code at their convenience to consume the package rather than the version left in SVN.
Our initial concerns with the package change were that we did not want to break any of the code that was dependent on the libraries or have to go and find everything that referenced the libraries and make changes to those directly. This is how the git migration and the NuGet package work got intermingled. We couldn’t really migrate to git with the repo as is, since the repo has 400,000 revisions with a lot of medium sized binary file churn. But, we figured commingling the git migration limited risk and let people move to the package consumption model at their speed, which also might help find chunks of the system that nobody is currently thinking about.
Where are we so far? After about two man-weeks we’re most of the way through step one, things have not worked out quite the way we wanted so far. In converting the existing dependencies over to NuGet we ran into several issues.
First, NuGet was by default restoring files to a location relative to the solution file, but looking for them using paths relative to the project, which was playing havoc with the different vertical solutions. NuGet expects a couple of different normal-styled project layouts, and we weren’t using them. The really annoying part of this was that if you built the solutions in particular order, the libraries would all get restored in the correct place, and that order happened to be the bottom order we tested it to the start with. We resolved this by adding a change to the NuGet config file to have all of the restored packages go to the same place, so the relative paths wouldn’t change in a project depending on which solution it was currently part of. This had another advantage in that you wouldn’t end up with dozens of copies of the same packages strewn all over your dev environment.
The next issue was that multiple teams had written custom build scripts that this broke since we were using automatic/command line package restore . Since we didn’t have access to change those custom build scripts we had reverted our changes and stopped to regroup. If we switched to MSBuild-Integrated package restore, that would fix the build scripts, but would require manual intervention in every solution to set up, adding ~600 MB of binaries to the svn repo. Sticking with the the initial approach would result in an ongoing stream of problems for all of the other related teams with custom build processes set up, especially since it was unclear who had what set up. We ended up switching to the MSBuild-Integrated, mostly due to the overriding need for stability. Technically, we thought it was the lesser choice, but other concerns ended up being primary.
The last issue so far is due to our use of ILMerge to merge output assemblies for some of the console applications. Some of the assemblies that got converted to NuGet package had previously been distributed in the GAC to target machines, and one of those assemblies can’t be used with the ILMerge due to issues with what happens when it gets repackaged. The common alternative to ILMerge doesn’t play nice with NuGet since it requires adding assemblies to projects in a different way. We haven’t resolved this issue fully yet. We added the versions of the assemblies causing issues with ILMerge to the GAC to resolve the problem in the short term. In the long term, we are planning to replace ILMerge with LibZ but haven’t gotten to it yet.
That’s the progress so far. Some other time-sensitive issues have pre-empted this effort, so it has moved to the back burner, but I’ll post a followup once we get further along in the process.
1. There 3 different ways to restore packages: automatic, command line, and MSBuild-Integrated. Automatic is visual studio integrated. Command line is manual invocation of the nuget executable. MSBuild-Integrated hooks into the next tooling layer down, so you only need to set up one thing, but it has some rather unfortunate side effects. Automatic and command line would get used together since the two of them operate on the same data but are used in different contexts. There is a good documentation on pros and cons of all of these options at the NuGet site.