I'm looking for a way to detect problems with assembly references in a large Visual Studio solution:
Binary references to bad locations, like a path not in source control or in the output of another project
Binary references to multiple versions of the same assembly across projects in the solution
Binary references without a path, that may be redirected to the GAC
Binary references that should have been project references
The whole story
I work on a large C# project with almost at 200 projects.
One of the problems that is creeping in over time is that references to assemblies are added but not always to the same version or to the correct location.
For example, a project may get a reference to System.Web.Mvc without a hint path, making it reference what ever version is in the GAC. Visual Studio (and Resharper) will also offer to add a missing reference but may do so by adding a reference to the output folder of another project.
Now the recent Windows Update catastrophy left some team members dead in the water, unable to build the solution. As you can imagine, this bumped up the priority of assembly reference management for us.
To detect some of the most obvious problems I've already setup an msbuild file that can be included in every csproj file and will detect bad references.
However, new project files will need to be edited manually to include that script. So that will inevitably be forgotten.
What I would really like is to check all project files in a solution for 'bad' references during the continuous build, so that all projects will be checked always.
I've been googling for a solution like this for some time and found lots of static analysis and code analysis tools but nothing to analyze project files in a solution.
So, before I go off and roll my own solution, is there a way to do this already?
Update
In order to clean up the code base I've created a bit of ScriptCS code that'll scan all csproj files for referenced to assemblies in Nuget packages and fix them. It's up on GitHub.
You can create a NuGet package where the sole purpose is incorporating a custom .targets file into a project. I recently used this strategy to solve another problem (error messages for missing .snk files).
Testing strong names of NuGet-distributed assemblies
Rackspace.KeyReporting source code
If you create a similar package, it's easy to right click on your solution node and verify that it is installed in all of your C# projects.
If your analysis is more complicated and requires the use of an assembly (custom build tasks) in addition to the .targets file, you can use an approach like I use for the Antlr4 NuGet package, which contains build tasks, resources, and custom .props and .targets files, but no actual assemblies that are referenced by the project it gets installed in.
ANTLR 4 C# Target source code (includes the Antlr4 package source and build scripts)
Instead of adding it to all projects in your solution, why not create some kind of test (unit test, build file, whatever) that can take a project file as input, parse it, and throw an error if OE or more references are incorrect. Much easier than adding (and checking out, committing etc) custom build steps to project files.
Even if you would use a nuget package as proposed earlier, you'd still have to check manually whether all your projects (200 projects? Really?) Reference the package.
Related
I'm working on a large enterprise .NET software. Currently the code is organized in 400+ projects grouped in almost 50 solutions. The folder structure containing code and other artifacts is about 9GB, but after the product is built, it consumes about 70GB. This is becoming an issue for developers because they normally work on different versions of the product and each one is consuming the same amount of disk space.
The main reason for this waste of space is because the build process is producing a very large set of copies of the same assembly even where it is not strictly necessary. Basically every intermediate class library project contains all the dependencies assemblies in its bin folder, where they are not needed at runtime.
For dependencies:
within the same solution we use project references
across solutions we use assembly references
for third parties components we use package references
I know that I could avoid copying dependencies in output bin folder (e.g. using CopyLocal=false or equivalent for project references and package references) but in this way, when a project is consuming an assembly in another solution, transitive dependencies are not copied even when they are really needed (for example when the project is an executable). This is causing issues at runtime or during unit test execution.
I tried to replace all assembly references with project references, even when the project is in another solution. In this way, I noticed that dependencies are correctly managed and transitive dependencies get copied properly. However, I'm a little bit scared of using this approach since it seems not natively supported by the IDE: in fact Visual Studio does not allow you to add a project reference to a project that is not in the current solution. My fear is that developer will have glitches in the IDE if I change dependencies in this way.
Do you have any suggestion or similar experience? The goal would be to minimize the number of copies of the same assembly across all projects and limit them to the minimum in order to reduce the disk space requirement.
I tried to replace all assembly references with project references, even when the project is in another solution. In this way, I noticed that dependencies are correctly managed and transitive dependencies get copied properly.
This is the way I would tackle this, especially since it seems to be working for you. However, as you've discovered, this means everything will need to be in a single solution. Enter filtered solutions.
Take Microsoft's lead, for example. In the dotnet/aspnetcore repository, they have 562 csproj files. However, instead of breaking these projects apart into separate solutions, they have a single solution in the root that contains everything, AspNetCore.sln.
Obviously having a single solution that large isn't going to play very well with the IDE. To solve this problem, Microsoft introduced filtered solutions in Visual Studio 2019. This feature basically provides a way of specifying a subset of projects in the solution to load. This allows teams working on entirely separate aspects of the same solution to be able to load only the projects they need.
We've all done it. We create a new class and type away the constructor adding dependencies and what not. Resharper is there to offer a helping hand and add missing references for us. It's only later on that we realise that we auto-imported references to other parts of our project that we shouldn't have.
So is there an addon for VS that one can configure (using wildcards etc) to have it issue warnings when/if certain .csproj projects are found to contain references to other .csproj projects that they aren't "allowed" to access (architecturally speaking).
Addendum: I am aware that I can achieve this by using pre-build msbuild-logic which parses the .csproj file using regexes and of course this would work. But I just find it kind of ... cumbersome and non-intuitive.
So is there an addon for VS that one can configure (using wildcards
etc) to have it issue warnings when/if certain .csproj projects are
found to contain references to other .csproj projects
AFAIK there's no such kind of extension that does the checking and warning job for you.
The reference to .csproj is actually project references in visual studio.
You can right-click project=>Build Dependencies=>Project Dependencies to check if current project depends on other projects in same solution.But this option will check both project references(add reference to xx.csproj in current.csproj) and project dependencies(ProjectDependencies section in .sln). So only use project references in your solution to manage dependencies between projects, then this option can easily check project references for you.
If the pre-build msbuild-logic which works need similar changes in every project in the solution, maybe Directory.Build.props can make some help if the changes in the project file have similar format. Fetch the pre-build logic into it and put this file in solution or repos directory, it reduces duplicate content in every project file.
I have a smallish solution, with about under twenty projects. The solution used to also contain about six source projects written by a third party service provider, ACME. Now, finally, this other party is supplying us only with a handful of DLLs. I used to just included their source, one project per DLL, in the solution, and so I am looking for a neat way to include all these assemblies in the solution, so they can be referenced from the many projects that need them.
My immediately apparent options are:
Create an AcmeAssembly project, add all DLLs as project items set to copy to output.
Create an AcmeAssembly solution folder. Quicker and simpler than a 'binary-only' source project, but solution folders have the very, very large drawback of having no means of grouping the files without a solution file, i.e. outside of VS.
Create a NuGet package that includes all the required binaries. Then at least we also have a partly 'phycical' grouping in the packages folders. My problem here is I have never written a NuGet package, but I am not asking how to here. I am asking about three candidate solutions, and more will be welcome, and if NuGet wins, I get to learn to write a package.
I can't simply use the project's output bin\debug and bin\release folders. To me, these are strictly output folders, and nothing but other dependency assemblies should also be output there. Deleting the bin folder should have absolutely zero effect on a build, so that is certainly no place to store binaries.
The advantages of nuget over the other solutions are:
Support for versioning
Support is built in into Visual Studio and MSBuild
No 'magic folders' that all developers need to have on their machines
Create your Acme library folder (something like: C:\Source\Library\AcmeLibrary). Put all of your Acme dll's in that folder. Then create the Solution Folder in VS and add the existing items to it (don't add the folder, but the items in the C:\Source\Library\AcmeLibrary folder to the solution folder using "Add Existing Item..." menu selection).
I'm trying to figure out what the best way to handle this scenario is.
Let's say I have a library that's referenced by multiple different non-related solutions, let's call it WebServiceInterface.dll. This library has a dependency on JSON.NET.
Before NuGet
The JSON.NET binary was referenced via a SVN external in the WebServiceInterface project. Other solutions which had a dependency on WebServiceInterface referenced the project (also as an SVN external) and as a result pulled both the project, and it's dependencies.
With NuGet
I haven't figured out how to force the JSON.NET reference to be stored under the WebServiceInterface project (as opposed to the RandomSolution\packages location). I found reference # nu-get to project-level and solution-level pacakges, but I can't seem to find out how to specify this when I add a dependency via nu-get.
The goal here is that when someone checks out WebServiceInterface and adds it to a new solution that it builds (instead of having broken references to JSON.NET which point to the packages directory under whatever the last solution was that checked in).
When I went to find out if Chris B had created a NuGet issue for this, I couldn't find one. EDIT: He did, see his comment below. But I did find a semi-documented feature of NuGet that I used to solve this problem: Allow specifying the folder where packages are installed
Let me break this question into 2 issues:
getting NuGet to allow for multiple solutions to use the same packages location
getting the NuGet packages to automagically fetch from source control when you include a project that has NuGet packages
Problem 1:
By default NuGet stores packages in a packages folder in the solution's folder. To change that location, create a nuget.config file in the solution's root folder with the following contents:
<settings>
<repositoryPath>..\..\..\Utilities\Library\nuget.packages</repositoryPath>
</settings>
<repositoryPath> is relative to your solution; so obviously make it whatever you want. Make each solution have it's own relative path to the same packages folder.
As far as NuGet's flow, from that point, the paths in repositories.config are relative to the folder containing repositories.config, not the solution, so now all projects/packages are managed independent of the solution location.
This allows multiple solutions to use the same packages in source control, and if those solutions use the same projects (that use NuGet packages), those solutions/projects will all be kept in sync no matter which solution updates the package.
Problem 1 completely solved.
Problem 2:
Let me address this from 2 perspectives. This applies to Visual Studio and TFS -- I'll leave SVN for someone else to address.
First: if you have no source code on your drive and do a get of a solution (not a project), I prefer to make it so that you get everything that solution needs to build. There shouldn't be any missing references to go manually grab. That much we can do by adding the package files as solution items. Yes, in each solution. A bit of work, yes, but when it's done the package files will fetch/update from source control automagically.
Second: In a new solution, when you include an existing source control project that has NuGet packages, you have to manually fetch the packages from source control and add them as solution items. At least anyone else getting your solution in the future will automagically get everything they need to successfully build. At least with VS/TFS, this is just the way it is, AFAIK. If projB depends on projA, and you add projB to a new solution, VS/TFS won't automatically grab projA from TFS. You have to do that manually. So then the same goes for dll references (like NuGet packages).
Summary of my solution:
Only one copy of packages in source control for all solutions
Any solution can update packages and all the other solutions will be kept in sync*
* Once one solution updates packages to new paths or file names, they will appear as missing references to the other solutions and you'll have to manually clean that up. But at least you know right where the packages are in source control "(as opposed to the RandomSolution\packages location)."
The packages are always stored at the solution level, so if you install a package into multiple projects, they came from the same place. I don't believe you can configure it so that each project has its own packages folder.
I'm not sure there's a nice way to do what you're trying. You could maybe have a build step on the project that fetches the package, but I don't know how well that will suit you.
I'd recommend posting in the NuGet Issue Tracker to get a discussion going. The people working on it seem pretty active, so it might be something they can add support for in a future version :-)
during development of our application we use a branching structure and while we are developing another team is using earlier builds of our software to create content with it.
To facilitate easy swapping between builds and teams I was hoping to use empty Hintpaths in the content projects' csproj files so that they can use our GAC installed assemblies to build against and in the meantime add a reference path to the projects for our use during development and test cycles where we don't want any assemblies installed in the GAC.
However, it seems reference paths are not stored in the csproj file and thus don't get sourcecontrolled. Since there will be extensive branching it would be less than ideal to have to set all reference paths again when a developer pulls another branch from sourcecontrol.
I've been searching for a bit now and can't seem to find ways to do this. Anybody know of a way to force the reference path in and out of sourcecontrol?
We're talking Visual Studio 2008 and TFS 2008 here.
Cheers,
Anton.
Ok, I seem to be a little clearer in the head after a good night's sleep, took the logical step, namely investigate where exactly the information was stored and how. It turned out the information was stored in the .user file for the project in the project's folder and as it turens out this file contains mbsuild xml.
I then did what I wanted as follows:
Create the Reference path as I required it to facilitate both scenarios without any work.
Browse to the Project's .user file
Copy the PropertyGroup containing the ReferencePath
Paste the PropertyGroup in all the necessary Projects' .csproj xml.
Reload and build.
Done.
The references are stored in the *.csproj file. The nodes are ItemGroup/Reference...
Thomas
This is pretty simple--we do this in our shop.
First, in the Workspace (using Windows Explorer, browse to the Solution folder), create a folder. We name it "Referenced Assemblies". In here, drop all your DLLs.
Now, in the Solution, add a new folder to match the one created in Windows Explorer. Into that folder, add all the DLLs you just dropped in.
Finally, in each project, set up your references to use the DLLs that were added to the solution.
Now your project references DLLs that are part of the solution, so that when the build runs, it will grab the DLL from Source Control to generate the build.
Also, I'd recommend not using the GAC at all if you can avoid it. In my experience, reference behavior is strange. It seems references go first to the GAC, then to the DLL in the local folder, meaning that if the DLL is updated, the one in the GAC is used instead of the DLL in the local folder (which is likely the updated one).