Pain-free local development while also referencing NuGet packages - c#

I am attempting to publish and consume versioned NuGet packages of class libraries while avoiding headaches for local development. Here is a sample Visual Studio solution layout:
| Libraries
| LibraryA
| LibraryB
| LibraryC
| Applications
| ApplicationD
| ApplicationE
This is a single solution containing both shared class libraries and multiple applications. Currently references to the class libraries by the applications are local in-solution references.
What I would like to do is to publish the libraries (A,B,C) as versioned NuGet packages which are then referenced by the applications as needed (D,E). This allows a change to a shared library to be independent from an update to an application which is deployed. Without this, changing one library could cause the binaries to change in a dozen or more applications, all of which would technically need to be tested. This is undesirable, and versioning with NuGet fixes this.
However, let us say that I want to update the content of LibraryA and ApplicationD at the same time. In order to do this after we have switched to NuGet, I will have to make changes to LibraryA, commit them, wait for the package to be created, tell ApplicationD to update its reference to LibraryA, and then test or develop in ApplicationD. This is far more complicated than simply working with both at the same time using local in-solution references.
What is a better way to get both the robustness of versioned NuGet packages for my shared class libraries while also keeping development simple even if it spans over multiple projects and applications? The only other solutions I have found all involve too much overhead or headache, such as having to constantly change the references for ApplicationD between the NuGet package and the local project.
EDIT: To clarify the premise, this question assumes the following:
The architecture (solution and project organization) cannot be significantly reorganized
Shared libraries are going to change at a non-trivial frequency
Changing a shared library cannot force any application to be updated
Applications can reference different versions of shared libraries

Although it takes some work, it is possible to hand-edit .csproj files in order to set up conditional referencing by adding a Condition attribute to the appropriate references.
EDIT I've moved these conditions into ItemGroups, as it seems this is how my mentioned production code is working, and there has been mention of this being a possible issue in VS 2013.
<ItemGroup Condition="'$(Configuration)' == 'Debug Local'">
<!-- Library A reference as generated by VS for an in-solution reference, children unmodified -->
<ProjectReference>...
</ItemGroup>
<ItemGroup Condition="'$(Configuration)' == 'Debug NuGet'">
<!-- Library A reference as generated by NuGet, child nodes unmodified -->
<Reference Include="LibraryA">...
</ItemGroup>
This would allow you to have, on the Projects D & E, configurations of "Debug NuGet" vs. "Debug Local" which reference the libraries differently. If you then have multiple solution files which have their configurations mapped to the appropriate configurations on the projects within, the end user would never see more than "Debug" and "Release" for most operation, since those are the solution configs, and would only need to open the full solution for editing the A, B, & C projects.
Now, as for getting the A, B, & C projects out of the way, you could set them up under a folder marked as a subrepo (assuming you're using an SCM that supports this, such as Git). Most users would never need to pull the subrepo since they're not accessing the ABC projects, and are instead grabbing from NuGet.
Maintenance wise, I can guarantee that VS will not edit the conditional references, and will respect them during compilation -I have gone through both VS 2010 and 2013 (EDIT: Professional version, though I have delved into doing the same with express) with the same conditional reference projects at work. Keep in mind than in VS, references can be made version-agnostic, making NuGet the only place from which version need be maintained, and that can be done like any other NuGet package. While I'm hopeful, I have NOT tested whether NuGet will fight with the conditional references.
EDIT It may also be prudent to note that conditional references can cause warnings about missing DLLs, but does not actually hinder compilation or run.
EDIT For those still reading this, I'm now (7/2019) hearing that the IDE isn't as friendly to these changes anymore, and either it or the Package Manager may override them. Proceed with caution, and always read your commits!

Update for .NET Core (2.x ++)
.NET Core 2.x actually has this functionality built in!
If you have a project reference to project A in project B, and project A is a .NET Standard or Core project with proper package information (Properties -> Package with Package id set to your NuGet package ID), then you can have a regular project reference in project B's .csproj file:
<ItemGroup>
<ProjectReference Include="..\..\A\ProjectA.csproj" />
</ItemGroup>
When you pack (dotnet pack) project B, because of the Package id in project A, the generated .nuspec file will be set up with a NuGet dependency to that Package ID, together with other NuGet references you might have, instead of just including the built DLL file.
<dependencies>
<group targetFramework=".NETStandard2.0">
<dependency id="Project.A" version="1.2.3" exclude="Build,Analyzers" />
<dependency id="Newtonsoft.Json" version="12.0.2" exclude="Build,Analyzers" />
</group>
</dependencies>

I know this is a 2-years old post, but just found it while facing the same situation. Also found this for VS2015, I'm in the process of testing it. I'll come back and adjust my answer accordingly.
https://marketplace.visualstudio.com/items?itemName=RicoSuter.NuGetReferenceSwitcherforVisualStudio2015

I also faced a similar problem. One approach that worked was using local repository (which is basically just a folder in local) and adding post-build script in the libraries. For example: let's say you need to update your implementation for LibraryA, then include following 3 steps in your post-build event for LibraryA:
Check if local repository has that version of package; if yes then delete it
rd /s /q %userprofile%\.nuget\packages\LibraryA\#(VersionNumber) -Recurse -ErrorAction Ignore
Create a nuget package
nuget pack LibraryA.csproj
Push it to local repository
nuget push LibraryA#(VersionNumber) -Source %userprofile%\.nuget\packages
These steps will make sure that the package is always updated for that version after each build (we had to do this since nuget packages are immutable)
Now in ApplicationD, you can point to local repository (%userprofile%.nuget\packages) to get LibraryA; such that after each build of LibraryA, you will receive an updated version of it in ApplicationD
PS: Inorder to get version number of you library you can use this : Determine assembly version during a post-build event

Unfortunately, there really isn't a way to have the best of both worlds. Internally in my company, we've mitigated it somewhat with a fast build/deploy process, which counteracts most of the burdens with always referencing a NuGet package. Basically, all of our applications use a different version of the same library hosted in a local NuGet repository. Since we use our own software to build, deploy, and host the packages, it makes it pretty quick to update the library, then update its NuGet package in another solution. Essentially, the fastest workflow we've found is this:
Make changes to library
Automatically build and deploy version of library incremented by 1 to internal NuGet feed
Update NuGet package in consumer application
The whole process from check-in to updating the consuming project takes around 3 minutes. The NuGet repository also has a symbol/source server which helps tremendously with debugging.

In the properties of ApplicationD, go to the "Reference Paths" tab and add the path of the output folder of LibraryA. Then, if you change and build LibraryA, the next build of ApplicationD will use the modified LibraryA.
When you are finished, don't forget to remove the "Reference Paths" and update the referenced NuGet package version.

My not-so-clean yet fastest solution so far is:
Assuming the following two separate solutions:
VS Solution 1: contains libraries published as nuget packages:
Solution1
|_ my.first.library
|_ my.second.library
VS Solution 2: contains applications, which consume one or more of the above libraries as PackageReferences:
Solution2
|_ my.first.application
| |_ depends on nuget my.first.library (let us say v1.0.1)
|
|_ my.second.application
In case, I'm making changes to my.first.library
I proceed as follows:
Make code changes to my.first.library and rebuild
Navigate to the build output directory of my.first.library (e.g. <Solution1 directory>/my.first.library/bin/debug/netstandard2.0) and copy the .dll and .pdb files
Navigate to the my.first.library's local directory of the currently being used nuget feed (for example at: C:\Users\user.name\.nuget\packages\my.first.library\1.0.1lib\netstandard2.0) and replace the .dll and .pdb files there with the ones generated in step 1 (possibly making backup).
Changes get reflected in my.first.application. Continue working and repeat steps 1-4, when needed
Advantages:
being completely local. No secondary nuget feeds needed.
zero changes to .csproj/.sln files
Caution:
While this solution offers you flexibility, make sure you clear your nuget cache before acting on them, for example by publishing to a nuget server. Thanks #Benrobot

Related

Use local source code of NuGet package to debug and edit code

I have a solution with an application project (ASP.NET Core) and multiple library projects. I want to separate some of the library projects into a separate solution and turn them into NuGet packages.
With the libraries in the same solution I could of course simply edit something in a library, run the application and see how it works (and debug, if necessary).
However, when I turn the libraries into a NuGet package, the application references the packages from our private NuGet feed instead of the project file.
My question is: is it possible to locally "override" the package reference and use the local source code instead? That way I could still edit the libraries and see the effects in the application. This is a lot easier than having to publish a new package for every small change (especially when trying to fix an issue or implementing a new feature).
DNT (Dot Net Tools) does this. You can specify which packages to switch and where they are.
See the 'switch-to-packages' and 'switch-to-projects' command line switches.
Its a bit fiddley as (when I last tried) you had to create a config file that holds the mapping, and it seems to be easy to break the switching. But its something.
https://github.com/RicoSuter/DNT
I've not tried it, but maybe you can use it to switch to packages on a commit for the build server to work correctly? (Or to ensure the references are correct in source control?)
If you want to use nuget in your project and debug, even modify the source files of the nuget packages, this is not a good choice because you should build the nuget project(generate the new changed dll) and repack it as a nuget package, then reinstall, to enable the changes. It is too complex.
Once you install the nuget, no matter how many changes you make, it’s useless. The nuget installed at this time is the version you made before any changes. No matter how you change it, it is the previous version. The version stays at that timestamp, unless you repackage the project. Generate nupkg and update the nuget version.
So nuget is not a good choice for your situation, you should use ProjectReference.
Directly use the ProjectReference to reference two source projects, build at the same time, and get the changed parts at the same time.
ProjectReference could cross two different solutions.
Add this on the main project:
<ItemGroup>
<!--add any nuget project'csproj file like this to debug its source code-->
<ProjectReference Include="..\xxx\xxx.csproj">
</ProjectReference>
</ItemGroup>
If the proejct is out of the solution, you could directly use the full path of the nuget project's csproj to connect it.
I'm not sure what you mean by "override" but you can always add the library project to your ASP.NET Core solution and reference it like normal project references. A project referenced within a solution doesn't have to be physically placed in the same folder as the solution itself.
This, however, does require that any developer on the project has both GIT repositories cloned locally (given your two solutions are located in separate GIT repos) in order to be able to build the ASP.NET Core solution. But I don't really see that as a downside.

Developing and debugging projects in mutli-repository in Visual Studio

This is the basics of the problem I am having (of course it is oversimplified for the sake of the question):
I have 2 projects in a solution stored in mono-repository:
Project1 - outputs a library
Project2 - outputs an executable
When Project2 references Project1.
I would like to move to multi-repository where Project1 will be stored in different Repository from Project2.
Project1 will output a nuget package and Project2 will reference it instead the project itself.
The issue I am facing: in current situation (mono-repository) when during development I introduce a feature in
Project2 that also requires a change in Project1 it is not a problem. If there are problems I can discover it
during development time.
In a new way (multi-repository) I first need to make a change to Project1, create nuget and push it to nuget
store, then update reference in Project2. If I would have problems in Project1, I must go back to Project1,
fix the issue and push it again, update reference in Project2 and so on. Also, losing the benefit of debugging
both projects.
Is there a solution to this approach? To focus my question: if I have source codes of both projects on my dev machine,
is it possible somehow to instruct Visual Studio to use source code instead of referenced Nuget for debugging?
Hope I explained it right and clear as possible...
If you have two different projects and both of them are in different repositories, you could add the library as a submodule of the first project. I'm assuming you're using git. You could do this simply by:
git submodule add <link for the other repo>
In visual studio you just add the project to the same solution and then reference library from the executable. I'm not sure if that would do the trick for you, but I hope that works.
We have the exact same problem in our company.
This is 2 part problem.
First part is to have both repositories near each other... There are more options, we were deciding between these 2:
git submodule
meta repository (it is a compromise between mono-repository and multi-repository) you have 2 repositories you want to connect, so you create 3rd repository as a meta-repository via https://github.com/mateodelnorte/meta
Second part is how to connect those 2 repositories so that they are debuggable, but still apart each other... We though of these 3 ways:
new .sln in meta repository that will reference both projects (this didn't cut it, because we already been in a position when we had multiple .sln files and maintaining them is not that easy, because when you add some project into one, you have to add it to the other solutions and this goes sideways really quick)
using Reference and PackageReference with Condition - locally for debugging when .dll is build in ProjA then the ProjB would use "Reference", if not the ProjB would use PackageReference => this was our main solution to our problem until we did it like that... (Because we have more than 2 projects ProjA -> ProjB -> ProjC.. The problem here was when it was built locally, referenced via Reference DLL, then ProjA was not visible from ProjC, but when built via CI and referenced via PackageReference, then ProjA was visible from ProjC)
using only Nugets - Every build of ProjA it will create nuget locally (pre-release) and in ProjB, we would reference that via wild-cards. This works, until you make a second change into ProjA, because ProjB will cache that nuget in C:/Users//.nuget/packages :( so when building ProjA and packing the nuget we clear the newly built nuget from this packages folder. You have to restore ProjB every time you make change into ProjA, but this is where we landed as a final solution for now.
So the final solution for us is:
meta-repository for like 5 other repositories
for debugging we use locally built nugets with constant version of "major.minor.patch.65534-local"
for CI we use the same nuget packaging but we override the local version with
feature branches "major.minor.patch.build_number-branch_name" (having -something after the version makes that nuget pre-release)
master/main branch "major.minor.path.build_number"

Best way to maintain .NET project dependencies and updated binaries in large multi solution application

I know this is a vague question, but I spent hours of searching and couldn't find a solution.
I have a class library project - A
I have another class library project - B
B has a reference to A
I have another console application project - C
C has a references to both A and B.
Now, what happens is:
A's code is being updated. A is being rebuilt.
Now B needs to get that new A binary, and then B needs to be rebuilt.
Now C needs to get that new A binary AND that new B binary.
Now imagine that I maintain several projects, each in its own solution, and that chain of dependencies is longer.
What is the best way to maintain all those binaries and keep them up to date?
How can I make sure that C always has the most up-to-date versions of A and B?
One solution I found is to create a nuget server in which there will be a nuget package for A and for B,
and when A or B's binaries are updated, new nuget packages will be rebuilt, and because C will use these nuget packages, it will always have the most up-to-date binary versions of A and B.
We work with SVN but I don't see how it can help managing dependencies between projects.
Any suggestion/direction will be very helpful
Thanks
The dependency chain you are describing is handled differently in different Visual Studio versions what depends on type of the project you are using. In old C# project version (pre VS 2017) when project reference is used (it is displayed in meta folder of the project as a references) and Visual Studio is unable to track and copy dependencies of project reference. This is why it was necessary to have your project C to reference both project A and project B.
Whereas in new format of csproj in Visual Studio 2017 when dependencies are used instead of references MSBuild is capable of traversing whole dependency tree and will handle properly situation where your project C references only project B which references project A. All required by application assemblies will be built and copied into output directory of project C.
In both cases there is no need to use NuGet server.
Now imagine that I maintain several projects, each in its own solution, and that chain of dependencies is longer.
What is the best way to maintain all those binaries and keep them up to date? How can I make sure that C always has the most up-to-date versions of A and B?
Once you have broken your dependency chain handled easily by MSBuild by separating your projects into separate solutions your build system lands in unsupported directly by Visual Studio territory. For pure managed .NET applications I would avoid that at any cost. For examples of building large, complex managed projects have a look at Roslyn compiler.
To solve that problem without using NuGet server at all - and it is my recommendation - you can create tree of dependencies spanning through solution boundaries by referencing directly from projects of one solution their dependencies (projects) from another solution. This will make your build much easier to manage (and it is particularly important for CI and DevOps solutions) and expand.
If you cannot modify current project/solution structure just create a new one as an overlay over existing project/solution structure and configure it properly. You will end up with one solution/project system which is perhaps easier to work with during code editing and testing and a second one which spans all maintained projects which is better suited for building whole application.
If the above option is not working just modify your all projects to output all final artifacts to one common output directory (move bin of every project to common top level bin directory, you may want to move obj intermediate directories as well).
If the choice of the NuGet server is a must the easiest way to do it it is to use common output directory and use NuGet.config in all projects consuming dependent projects by pointing them to new package source.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<!-- NOTE: Leave this file here and keep it in sync with list in dir.props. -->
<!-- The command-line doesn't need it, but the IDE does. -->
<packageSources>
<clear/>
<add key="private NuGet" value="file:///e:/src/myproject/.nuget" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
<config>
<add key="repositoryPath" value="..\packages" />
</config>
<packageRestore>
<!-- Automated package restore in VS does not work at this time with
this project and it causes build failures in VS. Disable it. -->
<add key="automatic" value="false" />
</packageRestore>
</configuration>
In the above NuGet.config file:///e:/src/myproject/.nuget points to main directory of your NuGet server (simple file system directory) where all .nupkg(s) are stored - it will require setting common output directory for nupkg(es) from all projects. This will not work on network shares yet as this nuget.client feature is under development right now. See nuget.clinet pull request which is still open at the time of writing Adding support for Network Share Hosted Package Source and compatability with generic build tools.
If you want to use full blown NuGet server first I would seriously consider using it as an external service i.e. myget.org. They have free plan if your packages can be public or paid plans if you want to keep your packages private.
Finally if you want to use your very own NuGet server go to NuGet project site and choose one you would prefer. NuGetGallery - equivalent to current nuget web site or smaller one NuGet.Server and follow installation instrcutions.
I have my own local NuGet repository so I think that my method should work on some shared network disk where you will have newest nuget packages.
First you need to install nuget standalone. You will download it from official website and you will have nuget.exe. I suggest adding it to the folder which will be added to system PATH environment variable. By doind this it will be accessible from your computer everywhere just by typing 'nuget'.
Now you need to get to the folder where is your .csproj file from your library.
Initialize your .nuspec file by typing nuget nuspec
You should have file [project_name].nuspec. Open it and add this line as second line of file after <xml> tag. It's weird that nuspec do not add this automatically.
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
authors tag and description tag on my computer could not be fetched from .csproj file so i needed to input values there manually.
I defined post-build event to my .csproj file as this:
nuget.exe pack $(ProjectPath) -IncludeReferencedProjects -OutputDirectory "D:\programowanie\nugetPackages"
powershell.exe D:\programowanie\nugetPackages\install.ps1
First command packs your project files into .nupkg files and send them directly to nugetPackages folder on disk D (there you could use some shared network drive).
-IncludeReferencedProjects description from specification:
Indicates that the built package should include referenced projects either as dependencies or as part of the package. If a referenced project has a corresponding .nuspec file that has the same name as the project, then that referenced project is added as a dependency. Otherwise, the referenced project is added as part of the package.
Install.ps1 contents:
nuget init D:\programowanie\nugetPackages D:\programowanie\nugetServer\
It creates folder structure in NugetServer directory for my packages and copy .nupkg files to those folders.
Now you need to add new package source inside your viual studio. You just enter path to the D:\programowanie\nugetServer\ in my case and name. You do it here:
I created solution for my .csproj library. I do not know if I done it by accident or is it really needed.
It was my first approach to having my own nuget package folder to maintain dependencies. I think that my solution is still not very good and lack a lot of automation. If you would like to have more info on the topic i suggest to stick with oficcial guidelines.
I do not have so complicated dependencies as in your case. But maybe creating solution which includes all of those libraries and definining good build order will enable to automatically build and deploy them to nuget server with my method.

MSBuild .NET project that will output existing assembly as it's result

I need to create csproj file that will be usable as project reference in VS2013 and will output prebuilt binary as it's "Build" result.
We use referenced projects for build, however company policy doesn't allow access to some of that projects for everyone. As a result projects need to be updated manually to make them build. This is really a major inconvenience when switching branches and when making edits to project files, so I want to create dummy project that will be bound to pre-built binaries as their "output" and will be placed instead of real projects.
EDIT: Moving that assembly to Nuget package is not an option for now since Nuget has some issues with dev flow (when you need to debug/test/develop package). I saw some VS extension that implements switching between Nuget package and local project which might solve this issue, but I'm not sure if it will be accepted and want to explore other options.
To be clear - the thing I want to avoid is editing project in any way, so that project can be built cleanly after pulling it from Git, and I don't have to clean it every time before commit.
I haven't properly tested it, but the solution seems really simple (if I understand the question properly).
Just add this to the existing .csproj, overriding the Build target to just give the path to the pre-built assembly.
<Target
Name="Build"
Returns="$(TargetPath)" />
This assumes the TargetPath property already defined, and it should automatically be if you're modifying the original .csproj. Otherwise just define it yourself in a <PropertyGroup> before the Build task.
Note that having TargetPath defined is important for the ProjectReferences in your own project to resolve.
How about having those restricted (binary only) projects reside in an internal Nuget package feed, so that Nuget can install the packages as needed, on build?

Nu-Get & issue with project level dependences for projects referenced by multiple solutions

I'm trying to figure out what the best way to handle this scenario is.
Let's say I have a library that's referenced by multiple different non-related solutions, let's call it WebServiceInterface.dll. This library has a dependency on JSON.NET.
Before NuGet
The JSON.NET binary was referenced via a SVN external in the WebServiceInterface project. Other solutions which had a dependency on WebServiceInterface referenced the project (also as an SVN external) and as a result pulled both the project, and it's dependencies.
With NuGet
I haven't figured out how to force the JSON.NET reference to be stored under the WebServiceInterface project (as opposed to the RandomSolution\packages location). I found reference # nu-get to project-level and solution-level pacakges, but I can't seem to find out how to specify this when I add a dependency via nu-get.
The goal here is that when someone checks out WebServiceInterface and adds it to a new solution that it builds (instead of having broken references to JSON.NET which point to the packages directory under whatever the last solution was that checked in).
When I went to find out if Chris B had created a NuGet issue for this, I couldn't find one. EDIT: He did, see his comment below. But I did find a semi-documented feature of NuGet that I used to solve this problem: Allow specifying the folder where packages are installed
Let me break this question into 2 issues:
getting NuGet to allow for multiple solutions to use the same packages location
getting the NuGet packages to automagically fetch from source control when you include a project that has NuGet packages
Problem 1:
By default NuGet stores packages in a packages folder in the solution's folder. To change that location, create a nuget.config file in the solution's root folder with the following contents:
<settings>
<repositoryPath>..\..\..\Utilities\Library\nuget.packages</repositoryPath>
</settings>
<repositoryPath> is relative to your solution; so obviously make it whatever you want. Make each solution have it's own relative path to the same packages folder.
As far as NuGet's flow, from that point, the paths in repositories.config are relative to the folder containing repositories.config, not the solution, so now all projects/packages are managed independent of the solution location.
This allows multiple solutions to use the same packages in source control, and if those solutions use the same projects (that use NuGet packages), those solutions/projects will all be kept in sync no matter which solution updates the package.
Problem 1 completely solved.
Problem 2:
Let me address this from 2 perspectives. This applies to Visual Studio and TFS -- I'll leave SVN for someone else to address.
First: if you have no source code on your drive and do a get of a solution (not a project), I prefer to make it so that you get everything that solution needs to build. There shouldn't be any missing references to go manually grab. That much we can do by adding the package files as solution items. Yes, in each solution. A bit of work, yes, but when it's done the package files will fetch/update from source control automagically.
Second: In a new solution, when you include an existing source control project that has NuGet packages, you have to manually fetch the packages from source control and add them as solution items. At least anyone else getting your solution in the future will automagically get everything they need to successfully build. At least with VS/TFS, this is just the way it is, AFAIK. If projB depends on projA, and you add projB to a new solution, VS/TFS won't automatically grab projA from TFS. You have to do that manually. So then the same goes for dll references (like NuGet packages).
Summary of my solution:
Only one copy of packages in source control for all solutions
Any solution can update packages and all the other solutions will be kept in sync*
* Once one solution updates packages to new paths or file names, they will appear as missing references to the other solutions and you'll have to manually clean that up. But at least you know right where the packages are in source control "(as opposed to the RandomSolution\packages location)."
The packages are always stored at the solution level, so if you install a package into multiple projects, they came from the same place. I don't believe you can configure it so that each project has its own packages folder.
I'm not sure there's a nice way to do what you're trying. You could maybe have a build step on the project that fetches the package, but I don't know how well that will suit you.
I'd recommend posting in the NuGet Issue Tracker to get a discussion going. The people working on it seem pretty active, so it might be something they can add support for in a future version :-)

Categories