Recently we had this case:
We have multiple projects in our solution that depend on each other. With project dependencies we make sure the project is build in the right order.
In the case above a developer forgot to add a project dependency. Therefore 2 things could happen:
1. The build would fail because msbuild builds and outputs the projects in an order that does not work
2. The build would succeed because msbuild builds and outputs the projects in the right order (by accident)
As you can see above, the first nightly build succeeded, the second nightly did not, and the third succeeded again. This can be confusing. I would like to make the builds more repeatable, so that the consecutive builds either all fail or all succeed.
Is there an approach i can take to make this better?
I read about deterministic builds but i am unsure whether that would help my case:
https://github.com/dotnet/roslyn/blob/master/docs/compilers/Deterministic%20Inputs.md
Additional info;
- The code is checkout clean before each build
- We use a plugin-based model so projects are not stitched together with project references, but plugin projects are build separately without dependencies. Then we reference the output dll from our application projects. Therefore the plugin should be built before the application project. We manually add project dependencies to make sure this happens.
Related
I am currently migrating a large solution from the old csproj file format to the new csproj format. I am doing this a few projects at a time, so I have a mixed environment with some projects using the old project file format and some projects using the new project file format.
I have started to notice some builds failing because files are in use. My theory (based on this answer) is that MSBuild is building some projects twice because the properties are different (i.e. the new project file format specify the TargetFramework property while old projects do not).
The projects that seem to have concurrency issues are projects that are referenced by other projects, where the referencing projects are split between the project file formats.
The command I am using to build the project is:
msbuild.exe /maxcpucount:6 /property:Configuration=Debug;Platform=x64 /t:Rebuild my.sln
Is there a way to instruct MSBuild to only build a given project once (regardless of properties) until I am able to convert all of the projects in the solution over to the new project file format?
Note that building single threaded does correct the concurrency issues, but that significantly slows down the build and the projects are still built multiple times.
This is a bug in MSBuild when referencing C++ CLI projects from multi-targeting projects. It appears they are putting in a fix to address this.
To work around the bug, the property can be removed from the reference using the GlobalPropertiesToRemove attribute:
<ProjectReference Include="..\B.CppCLILibrary\B.CppCLILibrary.vcxproj" GlobalPropertiesToRemove="TargetFramework" />
Only you could have correctly solved this since we do not have access to your code. But generally MSBuild and all build systems operate under the rule that a build 'target' is only processed once. No matter how many projects refer to it.
A build system should provide language for the user to specify dependencies between different 'targets'. And then it is up to the build system to figure out in which order to build these things in. (i.e. called a topological sort). Beginning or starting with making the most independent targets all the way to the least independent target.
If something is getting built twice it could be:
1. A bug in the build system.
2. The user forcing the project to build twice.
Anyways glad you got it sorted out.
I am build a system that is similar to project builds. A project has set of inputs and outputs. A project may depend on other projects. I can use topological sort and find the sequence in which I must evaluate projects. But, how do I parallel build. Toposort does not give same ranks to projects that can build in parallel. Also, how do I do incremental builds.
You can parallel build all projects which have not any dependencies. After build remove builded project from graph and again build projects without dependencies. Repeat until graph is empty.
In pseudo code
Build project graph
Validate graph (e.g. circular dependencies)
Find all projects without dependencies and not under build
Start build all found in 3 and mark them under build
Wait for one of builds end and romove builded project from graph
if graph not empty go to 3
Wait for rest builds
If I have a Project which uses my own made .dll and this .dll is not registered with the GAC but simply in the same folder as my projects App. eg C:\Program Files (x86)\MyApp Folder.
Can I, and more importantly how do I properly reference this .dll if I want to for example build a second project which also uses this .dll. It is possible that I build a few small apps that will use this .dll.
In this case must I have it in the GAC or if it is not there what must I do?
If I have a Project...
You should never put yourself in a situation where you have just a project. You first and foremost have a solution. A collection of projects that, together, build an app. Projects of course have a dependency on each other, you use a project dependency to tell the compiler about. Which automatically takes care of reference assemblies, the output of one project becomes the reference of another. And any changes you make to the source code of such a project automatically propagate to the others.
This is usually as far as teams take it.
This however tends to not work so well on very large solutions with dozens of projects, Visual Studio tends to get sluggish and building can take a long time. An important step to take in such a case is to freeze a root project. A programmer needs to get an explicit permission to make changes to such a core project. Because such a change tends to be very destabilizing, requiring many changes in dependent projects. And effectively destroys many hours of testing and validation time.
You do this by explicitly removing a project from a solution. Which now automatically makes it difficult to make changes to it. The dependent projects need to be updated to use an explicit reference assembly instead of the project dependency. Picking a well-known location for the assembly is important. Either source control or (preferrably) a build server is instrumental to be the source of the assembly. A tool like Nuget can be very useful.
I have a small project with a lot of dependencies. I changed one of the files in the project and noticed that MSBuild re-compiles all the dependencies also. What might cause this? Is this the default behavior and if so how do I change it?
If it matters I'm compiling via the command line (not from visual studio)
It could be building them all, or it could just be checking the dependencies to determine if they need a rebuild.
First, what kinds of projects are in the solution? More obscure project types may have targets that were not written with incremental builds in mind and may have buggy behavior (such as 2012 SQL Projects always rebuilding because it attempts to resolve SQLVAR params to file locations).
Here's some advice to get you on your way:
Prefer a file logger allowing you to troubleshoot without relying on the console buffer.
msbuild.exe MySolution.sln /fl3 /flp3:warningsonly;logfile=msbuild.wrn /fl4 /flp4:errorsOnly;logfile=msbuild.err /fl5 /flp5:Verbosity=diag;logfile=msbuild.log
This will generate three logs, two for warnings and errors, and a log with the greatest level of detail. Open msbuild.log and identify the build for one of the dependencies.
Does the build log indicate targets were skipped?
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
or that they were rebuilt?
Building target "SqlBuild" completely.
MsBuild can build your projects in parallel.
Another trick to enhancing performance is to build your projects in parallel. MsBuild is the scheduler and can build your dependencies in parallel using the magical "/m" switch!
I am having problems getting my C# Solution to build "Fresh". If I clean the solution and build it again it will not build (I can do it a few times and it will build). It has an error about the azure project getting build before the worker and web projects that it is dependent on. Also about how most of the projects in the solution are looking for
WAT070 : The referenced assembly {...}/Worker.dll was not found.
Please make sure to build the role project that produces this assembly before building this Windows Azure Cloud Service Project.
{...}\VisualStudio\v12.0\Windows Azure Tools\2.2\Microsoft.WindowsAzure.targets 1252 5 AzureProjectName
Now if I build the projects in the order listed in the Project Dependencies -> Build Order everything works. Also, the web and work role are listed before Azure Project.
Solution is very simple.
You can set your project build order by right click on 'Project Solution' and select "Select Project Build Order" option.
For Example, I have a WpfFormApplicaiton1 and two class with title "ClassLibrary1" and "ClassLibrary2". By default Visual Studio sets it as follow:
My requirement is such as "ClassLibrary2" will be used by "ClassLibrary1" & "ClassLibrary1" will be used by "WpfFormApplication1". So, in order to fulfill this requirement I have to change the default project build order.
Go to Project Dependencies; Select the "ClassLibrary1" and set the "ClassLibrary2" as its dependency.
Similary, select the "WpfFormApplication1" and set the "ClassLibrary1" as its dependency.
Now, the desired project build order is set; confirmed by Project Build Order's options.
Solution is taken from my blog.
Your issue may be that the Dependencies are not defined. Even though the Build Order shows the order in which projects are built, if you do not define the dependencies for each one under Project Dependencies, msbuild will not know to wait for the dependencies build to complete before moving on.
To clarify: Unless you actually check the box that an item is a dependency, the projects in the build order list may build in parallel and not sequentially.
You can see under Tools->Options->Project and Solutions->Build and Runthat there is a default value for the number of parallel projects to build.
So to make the build process wait for dependencies to build make sure that all of the "Depends on" fields are checked for the projects needed under Project Dependencies -> Dependencies.
We had an issue where the project guids differed in case. Editing the project files solved the issue. We changed all Guids to uppercase
I just had this issue too. In my case the issue was that I had several project references within the solution. The other projects were using a different version of the framework to my Worker Role (4.5.1 vs 4.5).
When I changed all projects to use the same version of the framework the solution builds and runs successfully.
I had this issue. In my case, the solution's project build order, as determined by VS.Net, was not correct. Specifically, my web project was listed above three of its dependencies.
These three dependencies were listed under in the References node of the web project in the Solution Explorer. However, in the dialogue Project Dependencies, the web project did not depend on any project.
Also, I noticed that the web project had a small blue exclamation mark, with hoover-over message "The Web project '' requires SQL Server Express, which is not installed on this computer. ...". After fiddling with the web.config, based on ASP.NET Web Api: Project requires SQL Server Express and reloading the project, the exclamation disappeared and the project dependencies were correctly checked in the dialogue Project Dependencies and the build order reflected this correctly. However, when I reverted the changes to the web.config, as a test, the dependencies were not removed, so I am quite unsure what fixed my issues.
Anyway, in the dialogue Project Dependencies, you can manually check any project which was not automatically identified as a dependency.
Check to make sure there aren't any residual old files in your working folder. These can cause confusions with MSBuild. To avoid that, simply blow away all the old files in the path and get latest.