We Keep Coding

How to properly package a Nuget distributed custom tool that generates source code as part of the build

Thanks to this awesome article by Nate McMaster, I know how to package a .NET core console application as a Nuget package that automatically installs itself as a (pre, in this instance) build task.
To test if everything works, I simply had my custom tool write out a public C# class.
Here is the complete and runnable sample on Github.
However, the file that my custom tool adds isn't really part of the build (the first one that actually generates the file) and therefore the introduced class is not in the assembly after the first build (see Line 38 here). However, because the .NET core projects now automatically include all .cs files alongside the project, it builds the new class into the output on subsequent builds (see Line 57 here).
The generated files don't go away on clean, though and generally don't behave like something an MSBuild task would output. However, because the exec happens in a targets file, we ought to have access to all the machinery to make this happen. So my question is:
How do I correctly execute a custom build tool (console app) that needs to examine the project, its files and generate source code (preferably in obj/ as say <foo>.g.cs that gets compiled into the resulting assembly as part of a single build? Ideally, this generated file(s) shouldn't appear in the solution explorer, either.
Help!

When generating the intermediate file (CustomTool.g.cs) in the intermediate folder (you'll need to resolve it, see example in Refit library: https://github.com/reactiveui/refit/blob/5b4e14aaf8a1fcc27396b7c08171d100aba1b97d/Refit/targets/refit.targets#L11); you need to explicitly add it as a compile item.
Taking your example targets file (https://github.com/aniongithub/CustomTool/blob/master/CustomTool/RunCustomTool.targets#L13):
<Project>
<PropertyGroup>
<IntermediateOutputPath Condition="$(IntermediateOutputPath) == '' Or $(IntermediateOutputPath) == '*Undefined*'">$(MSBuildProjectDirectory)obj\$(Configuration)\</IntermediateOutputPath>
<!-- Command to invoke CustomTool -->
<CustomTool>dotnet "$(MSBuildThisFileDirectory)/netcoreapp2.2/CustomTool.dll"</CustomTool>
<!-- Other variables -->
<CustomVariable>"$(MSBuildProjectDir)"</CustomVariable>
</PropertyGroup>
<Target Name="CustomTool" BeforeTargets="CoreCompile" DependsOnTargets="PrepareForBuild">
<Exec Command="$(CustomTool) $(ProjectPath) $(IntermediateOutputPath)CustomTool.g.cs" />
<!-- add generated file as a compile item, otherwise it won't get picked up -->
<ItemGroup Condition="Exists('$(IntermediateOutputPath)\CustomTool.g.cs')">
<Compile Include="$(IntermediateOutputPath)\CustomTool.g.cs" />
</ItemGroup>
</Target>
</Project>

project.assets.json not found with msbuild

I have an ASP.NET Core application that i wish to build on a jenkins machine with MSBuild 15.
When i try to build i get the following error:
C:\Program Files\dotnet\sdk\2.1.502\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(198,
5): error NETSDK1004: Assets file 'C:\sync\Src\Util\myUtil\ob
j\project.assets.json' not found. Run a NuGet package restore to generate this file
I understand that i need to do nuget restore somehow, but i failed to make it work.
My build process:
Running a batch filed with the following command:
call "%VS150COMNTOOLS%VsDevCmd.bat"
MSBuild DailyBuild.proj /t:DailyBuild /p:VersionNumber=%2 /l:FileLogger,Microsoft.Build.Engine;logfile=Build.log
The DailyBuild.proj file look like this:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<SourcesPath>$(MSBuildProjectDirectory)\..\..\</SourcesPath>
<CSCleanProperties>BuildType=Clean;Configuration=Release;IsFormalBuild=true</CSCleanProperties>
<CSBuildProperties>BuildType=ReBuild;Configuration=Release;PauseBuildOnError=false;PublishWebSites=true;VersionName=myProd-$(VersionNumber)</CSBuildProperties>
</PropertyGroup>
<Target Name="DailyBuildWithClean">
<MSBuild Projects="$(MSBuildProjectDirectory)\Make.proj" Targets="Clean" Properties="$(CSCleanProperties)"/>
<MSBuild Projects="$(MSBuildProjectDirectory)\Make.proj" Properties="$(CSCleanProperties)"/>
<MSBuild Projects="$(MSBuildProjectDirectory)\Make.proj" Targets="FormalBuild" Properties="$(CSBuildProperties)"/>
</Target>
<Target Name="DailyBuild">
<MSBuild Projects="$(MSBuildProjectDirectory)\Make.proj" Targets="SW;PreparePackFolder" Properties="$(CSBuildProperties)"/>
</Target>
</Project>
The Make.proj is a proj file containing definitions for many applications to be built, one of them is my ASP.NET Core app.
How do i fix this problem? thank you.
SOLUTION EDIT:
Thanks to solution by Martin Ullrich:
Added in the DailyBuild.proj the target Restore, also added in the Make.proj a target called restore as suggested
(IE:
<Target Name="Restore">
<MSBuild Projects="$(SourcesPath)\my.sln" Targets="Restore" />
</Target>
)

Add -r (-restore//Restore) to your MSBuild command to trigger a restore before the main build.
The restore parameter will build the Restore target, clear internal caches and then run the rest of the build as specified.
Since you build a custom MSBuild project, you then need to add a Restore target to it:
<Target Name="Restore">
<MSBuild Projects="$(SourcesPath)\my.sln" Targets="Restore" />
</Target>
(or alternatively add another Restore target on the make.proj file and forward it from there to the solution or individual projects that you need to be restored)

Be careful using Restore Nuget packages directly in MS build task configuration.
This option is deprecated, as mentioned here for Azure DevOps. (However, I am not sure how context-dependent that is.)
(Important) This option is deprecated. Make sure to clear this checkbox and instead use the NuGet Installer build step.
Source Link: MSBuild
However, I already used that step (in TFS), so this obviously would not fix it for me.
I tried removing the packages-folder in Source Control Explorer as mentioned here, but that did not fix it either.
Inspired by this, I upgraded the TFS NuGet Installer build step to use Nuget 4.0 (in "Advanced" options), and that did fix it. (Maybe in combination with the removal of the packages-folder?)

Simply rebuild your project in another location(directory/folder ) and run your solution it works 100%.

Using msbuild.exe to build a project with a dependency requiring a different Platform property

I have two fairly straightforward C# projects: An executable that can build as either x86 or AnyCPU, which references (via <ProjectReference>) a DLL project that only has an AnyCPU configuration. This all works as expected within Visual Studio.
I am trying to build the x86 version of the executable project (and its dependencies) from the command line, with /p:Platform="x86". This causes the build of the DLL project to fail. (Whereas /p:Platform="AnyCPU" works, presumably because it is valid for both projects.)
The full command line I am using is:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild MyProject\MyProject.csproj /t:Build /p:Configuration="Release" /p:Platform="x86"
What are my options for getting this build to work from the command line? Preferably without modifying the DLL project at all, or modifying the projects in ways that interfere with using them normally in Visual Studio.
(The ultimate goal here is a batch file that can build a clean version of the project for distribution.)
Additional info:
Both projects have "Debug" and "Release" configurations. The executable project has "x86" and "AnyCPU" available under Platform. The DLL project has only "AnyCPU" available under Platform. The "Platform target" option matches the "Platform" in all cases. (There is no "Prefer 32-bit" option, as I am on VS2010.)
The error seems to be a compilation-related error ("no unsafe code allowed") in the DLL, which -- although I am not 100% sure -- seems to be because none of the <PropertyGroup> elements in the DLL project are being matched (due to Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' " etc) that would specify necessary parameters for compilation (eg AllowUnsafeBlocks and DefineConstants).

Answering my own question... First of all, MSBuild can build solution files. This does exactly what you'd expect:
msbuild MySolution.sln /p:Configuration="Release" /p:Platform="x86"
The result is an x86 executable with an AnyCPU DLL (as the solution specifies).
There are a few people out on the internet suggesting that its behaviour is not a perfect match for Visual Studio in some obscure cases. But it seems to work just fine for my purposes. (I think they were having issues with the order things get built in.)
I knew that MSBuild could build a solution file, but -- oops -- I neglected to test it on my simple reproduction case, after it failed on the more complex thing I'm working on.
The above, alone, isn't enough for a fully satisfying answer, particularly if there is a need to customise things. The way MSBuild builds solution files is to create a dummy project file, based on the solution file. This can be inspected by first setting an environment variable like so:
set MSBuildEmitSolution=true
This will emit the dummy project file next to the solution file, which can then be inspected.
I haven't completely analysed what it is doing, but it looks fairly straightforwardly like it is using the <MSBuild> task with the Projects parameter that is itself passing in the solution-specified Configuration and Platform appropriate for each project. According to the documentation it seems to be using the ones specified in AdditionalProperties. (This also seems useful to know.)
For reference, here is some relevant code extracted from the generated project file:
<Target Name="Build" Outputs="#(CollectedBuildOutput)">
<MSBuild Projects="#(ProjectReference)" BuildInParallel="True" Properties="BuildingSolutionFile=true; CurrentSolutionConfigurationContents=$(CurrentSolutionConfigurationContents); SolutionDir=$(SolutionDir); SolutionExt=$(SolutionExt); SolutionFileName=$(SolutionFileName); SolutionName=$(SolutionName); SolutionPath=$(SolutionPath)" SkipNonexistentProjects="%(ProjectReference.SkipNonexistentProjects)">
<Output TaskParameter="TargetOutputs" ItemName="CollectedBuildOutput" />
</MSBuild>
</Target>
Where #(ProjectReference) is grabbing data from:
<ItemGroup>
<ProjectReference Include="X:\Solution\MyProject\MyProject.csproj">
<ToolsVersion>
</ToolsVersion>
<SkipNonexistentProjects>False</SkipNonexistentProjects>
<AdditionalProperties>Configuration=Release; Platform=x86; VisualStudioVersion=10.0</AdditionalProperties>
<Configuration>Release</Configuration>
<Platform>x86</Platform>
</ProjectReference>
<ProjectReference Include="X:\Solution\DLLProject\DLLProject.csproj">
<ToolsVersion>
</ToolsVersion>
<SkipNonexistentProjects>False</SkipNonexistentProjects>
<AdditionalProperties>Configuration=Release; Platform=AnyCPU; VisualStudioVersion=10.0</AdditionalProperties>
<Configuration>Release</Configuration>
<Platform>AnyCPU</Platform>
</ProjectReference>
</ItemGroup>
(Note the different AdditionalProperties.)

Ensure subdirectory builds before rest of project

Working on a very large C# project with multiple subportions. One of these portions creates and places dlls in a specific location for consumption. As per a recent change we're now trying to place these dlls with every root build call made. (Use to be manually placed every milestone or so)
So I'm referencing the dirs.proj file that is compiling the subdirectory which creates and places the dlls. That all works fine. The problem is that for one reason or another other portions of the project start to look for these dlls before that part has finished compiling.
How can I ensure that this part of the project gets compiled and places the dlls before beginning to compile the rest of it? I have a very brief understanding of and but really don't know how to use them to do what I want.
Thanks for any and all help!

Read up on Build Targets and the MakeDir task.
You'll want to include the logic creating the new directory in a task that is scheduled to occur prior to the default Build task.
<Target BeforeTargets="Build" Name="CreateRequiredFolder" Condition="!EXIST('$(MyDirectory)')" >
<MakeDir Directories="$(MyDirectory)" />
</Target>
Note how we use BeforeTargets to schedule this before Build, and the condition on the target causes it to be skipped if the desired folder already exists.

Here is what I ended up doing to get it to work.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="......." InitialTargets="BuildWebshared">
<ItemGroup>
<WebsharedProjects Include="..\location\dirs.proj" />
</ItemGroup>
<Target Name="BuildWebshared">
<MSBuild
Projects="#(WebsharedProjects)"
Targets="Build">
</MSBuild>
</Target>
....

MSBuild Post-Build

I've got an MSBuild script that is just about doing everything that I need it to do apart from my post-build step (see a previous question that I asked: MSBuild conditional Exec?).
What I'm looking to do is build many csproj files and optionally perform post-build steps if and only if the project was built. I don't want to perform my post-build step all the time or else the timestamp on my final output will be modified unnecessarily (and it makes the build process very time consuming for no reason).
In my MSBuild script I've got something like the following for each of my csproj files:
<Target Name="ProjectName">
<MSBuild Projects="PathToProject" Properties="Configuration=$(buildtype)" />
</Target>
Edit:
I think what I really want to do is detect when the CoreCompile task runs for each project. If there were some way to check for this in a condition?
Any ideas?
I'm new to MSBuild so maybe I'm on completely the wrong track!
Thanks,
Alan

You can also do it based on the configuration selected in your build process. For CI, you should always use "Release" or "Production" (you can define your own).
<Exec Condition="'$(ConfigurationName)'=='Release'" Command="your command goes here ..."/>

After much searching for a simple solution to this problem I didn't find one and ended up coming up with a solution of my own that works but may not be the best solution. However, I wanted to share it with anyone else that is having the same problem so that you can at least have a working solution and hopefully saving you a lot of head banging.
To recap, what I wanted to do was run a command line tool after my project was built but only if the assembly was updated (i.e. the timestamp changed). I didn't want to put this into the post-build section of every project because I only wanted the post-build to happen on our build server (not development machines).
I didn't find any way of doing this externally in my main .proj file and did end up altering the post-build section of each .csproj file. However, I prefixed it with an if condition something like this:
if '$(ExecuteCommand)' == 'true' command.exe
This means that the command will never be executed on the development machine but when I invoke the build from my .proj file I can set that flag to true like this:
<!-- Define common properties -->
<PropertyGroup>
<ExecuteCommand>true</ExecuteCommand>
</PropertyGroup>
<Target Name="YourTarget">
<!-- Build project -->
<MSBuild Projects="Path to project" Properties="ExecuteCommand=$(ExecuteCommand)" />
</Target>
As I said, I don't think it is the most graceful solution but it certainly works and will be sufficient for me for the time being. However, I'd still be interested to hear what the proper way of achieving this is so that I can improve my script.
Thanks,
Alan

If you can add the following to each of your projects:
<Target Name="DoStuffWithNewlyCompiledAssembly">
<Exec Command="command.exe" />
</Target>
... then you only need to add a property:
<Target Name="Name">
<MSBuild Projects="" Properties="TargetsTriggeredByCompilation=DoStuffWithNewlyCompiledAssembly" />
</Target>
This works because someone smart at Microsoft added the following line at the end of the CoreCompile target in Microsoft.[CSharp|VisualBasic][.Core].targets (the file name depends on the language and MSBuild/Visual Studio version).
<CallTarget Targets="$(TargetsTriggeredByCompilation)" Condition="'$(TargetsTriggeredByCompilation)' != ''"/>
So if you specify a target name in the TargetsTriggeredByCompilation property, your target will run if CoreCompile runs-- and your target will not run if CoreCompile is skipped (e.g. because the output assembly is already up-to-date with respect to the code).