We have a VS2012 solution with a dozen or so C# projects in it. One of the C# projects makes use of a text file, which is converted to SQL by a separate C# tool we've written. We want to automatically invoke this TXT-2-SQL file conversion during a build whenever the TXT file has changed (Release and Debug build).
Furthermore, we'd like the Debug build to cause the SQL to be loaded into the local DB; ie, invoke one of our existing batch files, populate_db.bat, with the SQL filename as an argument (this just wraps the invocation of the mysql client and causes the SQL to be executed).
What's the best way to do this?
From my own research, I've found the following choices:
Custom MSBuild Task - create a separate assembly containing a class implementing ITask; reference it from our csproj; let that task do the work just described.
Custom Build Tool (Single File Generators) - create a separate assembly containing a class implementing IVsSingleFileGenerator; register that on each dev machine; use that tool name in the 'Custom Build Tool' item in the properties of the aforementioned TXT file in our existing csproj
C++ Makefile project - use CustomBuild or CustomBuildTool statements in an otherwise empty C++ Makefile project; Include TXT file of interest in that other project file; Add that project to solution.
Target + Execs - Add custom target nodes to our existing csproj:
<Target Name="TXT2DB"
AfterTargets="Build"
Condition=" '$(Configuration)' == 'Debug' "
Inputs="MyTextFile.txt"
Outputs="MySqlFile.sql"
Label="TXT2DB" >
<Exec Command="$(ProjectDir)\Tools\txt2sql\txt2sql.exe MyTextFile.txt MySqlFile.sql" />
<Exec Command="populate_db.bat MySqlFile.sql" />
</Target>
Post Build Events - use post build events. Mentioned for completeness, but discarded b/c it happens in both debug and release, and doesn't seem to be sensitive to a TXT file change.
Currently, we've selected #4 as the easiest to impl and maintain solution, and satisfies all the requirements.
Related
Thanks to this awesome article by Nate McMaster, I know how to package a .NET core console application as a Nuget package that automatically installs itself as a (pre, in this instance) build task.
To test if everything works, I simply had my custom tool write out a public C# class.
Here is the complete and runnable sample on Github.
However, the file that my custom tool adds isn't really part of the build (the first one that actually generates the file) and therefore the introduced class is not in the assembly after the first build (see Line 38 here). However, because the .NET core projects now automatically include all .cs files alongside the project, it builds the new class into the output on subsequent builds (see Line 57 here).
The generated files don't go away on clean, though and generally don't behave like something an MSBuild task would output. However, because the exec happens in a targets file, we ought to have access to all the machinery to make this happen. So my question is:
How do I correctly execute a custom build tool (console app) that needs to examine the project, its files and generate source code (preferably in obj/ as say <foo>.g.cs that gets compiled into the resulting assembly as part of a single build? Ideally, this generated file(s) shouldn't appear in the solution explorer, either.
Help!
When generating the intermediate file (CustomTool.g.cs) in the intermediate folder (you'll need to resolve it, see example in Refit library: https://github.com/reactiveui/refit/blob/5b4e14aaf8a1fcc27396b7c08171d100aba1b97d/Refit/targets/refit.targets#L11); you need to explicitly add it as a compile item.
Taking your example targets file (https://github.com/aniongithub/CustomTool/blob/master/CustomTool/RunCustomTool.targets#L13):
<Project>
<PropertyGroup>
<IntermediateOutputPath Condition="$(IntermediateOutputPath) == '' Or $(IntermediateOutputPath) == '*Undefined*'">$(MSBuildProjectDirectory)obj\$(Configuration)\</IntermediateOutputPath>
<!-- Command to invoke CustomTool -->
<CustomTool>dotnet "$(MSBuildThisFileDirectory)/netcoreapp2.2/CustomTool.dll"</CustomTool>
<!-- Other variables -->
<CustomVariable>"$(MSBuildProjectDir)"</CustomVariable>
</PropertyGroup>
<Target Name="CustomTool" BeforeTargets="CoreCompile" DependsOnTargets="PrepareForBuild">
<Exec Command="$(CustomTool) $(ProjectPath) $(IntermediateOutputPath)CustomTool.g.cs" />
<!-- add generated file as a compile item, otherwise it won't get picked up -->
<ItemGroup Condition="Exists('$(IntermediateOutputPath)\CustomTool.g.cs')">
<Compile Include="$(IntermediateOutputPath)\CustomTool.g.cs" />
</ItemGroup>
</Target>
</Project>
I have a project with some nuget packages referenced.
In output folders (bin\Debug or bin\Release), all referenced libraries lie next to the executable.
How to specify output folder for libraries?
I want all nuget libraries in bin\Release\Libs and executable in bin\Release.
I woke up early this morning and decided to have a go at doing it myself. Turned out to be pretty quick, but that may be because of my (unfortunate) experience with looking into MSBuild files. Writing this post took me far longer than writing the target.
From your question, I assume you're using a traditional project, since SDK style projects only create the project's assembly in the bin directory. However, I much prefer SDK style projects because use can quickly and easily use the dotnet cli to create test projects and the csproj is much more easily editable. So, I'll give you my steps to find my solution for SDK style projects, and you need to follow along to do something similar with a traditional project.
So, we want to change where a files are being copied, which means we need to modify some items. Everything in MSBuild runs in a target, so we'll need to know when to run our custom target, what items to modify and probably what metadata of those items to modify. I created a new project, added some NuGet references then ran dotnet msbuild -t:publish -bl and opened the msbuild.binlog file.
What metadata to change
Searching for the name of a dll that came from a nuget package, I find a message saying copied from ... to ..., so I click on it to go to the entry, and follow the tree back to the task, which I see is the built-in Copy task. The target path to the task is Publish -> _PublishBuildAlternative -> ComputeAndCopyFilesToPublisDirectory -> CopyFilesToPublishDIrectory -> _CopyResolvedFilesToPublishAlways. Double clicking the copy task I see
<Copy SourceFiles = "#(_ResolvedFileToPublishAlways)"
DestinationFiles="#(_ResolvedFileToPublishAlways->'$(PublishDir)%(RelativePath)')"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForPublishFilesIfPossible)"
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForPublishFilesIfPossible)">
So, I can guess I need to modify the RelativePath metadata of an _ResolvedFileToPublishAlways item.
What item to change
Side note: MSBuild doesn't have public/private modifies, so instead a convention is generally used. Anything starting with an underscore should be considered to be an implementation detail that could change between releases, so it's better to use things that do not start with an underscore, and the teams who maintain the targets file should try harder not to break compatibility.
So, since _ResolvedFileToPublishAlways starts with an underscore, let's find out where it was created. Searching for it takes me to a target where the binlog tells me it was added, in a target called _ComputeResolvedFilesToPublishTypes, and its definition is
<Target Name="_ComputeResolvedFilesToPublishTypes">
<ItemGroup>
<_ResolvedFileToPublishPreserveNewest Include="#(ResolvedFileToPublish)"
Condition="'%(ResolvedFileToPublish.CopyToPublishDirectory)'=='PreserveNewest'" />
<_ResolvedFileToPublishAlways Include="#(ResolvedFileToPublish)"
Condition="'%(ResolvedFileToPublish.CopyToPublishDirectory)'=='Always'" />
</ItemGroup>
</Target>
So, I can see that it's simply copying ResolvedFileToPublish items to new item names. Looking for where those items are created, it's in a target named ComputeFilesToPublish, and expanding the tree to see all the items created and their metadata, I'm going to guess the items I want to modify all have AssetType = runtime, which is perfect for a condition we're going to need to use.
When to run our target
Ideally I would run just before CopyFilesToPublishDirectory, however looking at its definition I see
<Target Name="CopyFilesToPublishDirectory"
DependsOnTargets="_CopyResolvedFilesToPublishPreserveNewest;
_CopyResolvedFilesToPublishAlways" />
The problem is that when MSBuild executes a target it runs in this order:
Any targets listed in DependsOnTargets
Any target that lists the current target as BeforeTargets
The current target
Any targets that lists the current target as AfterTargets
So, while I want to run BeforeTargets='CopyFilesToPublishDirectory', the DependsOnTargets will run before my target, so I can't do that. So I'll choose to run AfterTargets="ComputeFilesToPublish". There are other targets that run in between those, and one sounds like that it might add ResolvedFileToPublish items, but with my current project the target doesn't run because of conditions, so my custom target might not be generic enough to work for all projects.
Writing our custom target
So now we know when our target will run, which items it will modify and how we will modify their metadata.
<Target Name="RedirectRuntimeFilesToBinDirectory" AfterTargets="ComputeFilesToPublish">
<ItemGroup>
<ResolvedFileToPublish Condition=" '%(ResolvedFileToPublish.AssetType)' == 'runtime' ">
<RelativePath>lib\%(RelativePath)</RelativePath>
</ResolvedFileToPublish>
</ItemGroup>
</Target>
Unfortunately the binlog doesn't show the details about the metadata being modified, which is a real pain in the arse when trying to debug build issues and why some items have unexpected values, but in any case I've now successfully changed the destination of NuGet dependencies, and probably project to project references, to a lib\ directory.
Grace to the zivkan's investigation I found the answer. Traditional project has target CopyFilesToOutputDirectory which depends on _CopyFilesMarkedCopyLocal target. In this last one we have task Copy:
<Copy
SourceFiles="#(ReferenceCopyLocalPaths)"
DestinationFiles="#(ReferenceCopyLocalPaths->'$(OutDir)%(DestinationSubDirectory)%(Filename)%(Extension)')"
SkipUnchangedFiles="$(SkipCopyUnchangedFiles)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForCopyLocalIfPossible)"
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForCopyLocalIfPossible)"
Condition="'$(UseCommonOutputDirectory)' != 'true'"
>
And here I found metadata DestinationSubDirectory which is exactly what I need to change.
So finally
First, we need to change csproj file and add these lines:
<ItemDefinitionGroup>
<ReferenceCopyLocalPaths>
<DestinationSubDirectory>lib\</DestinationSubDirectory>
</ReferenceCopyLocalPaths>
</ItemDefinitionGroup>
Second, we need to change app.config file to let the assembly know the path to the libraries:
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="lib;libs" />
</assemblyBinding>
</runtime>
That's all. All referenced libraries will be copied into subfolder lib
What is the easiest way to create a build configuration in approximately 50-100 projects (same solution) IF Visual Studio has already detected the build configuration from another project?
Our team uses a set of common projects (namespace is simply "Common") within several solutions. The Common namespace has it's own master solution with its own set of build configurations. Common's solution contains five build configurations ("Debug-QA", "Debug-Dev", etc.).
Whenever these projects are used within a NEW solution (ie "MyNewSolution"), Visual Studio shows the build configurations from Common's master solution. Unfortunately, these configurations have not yet been created in MyNewSolution or any of MyNewSolution's projects. This creates a problem for ADDING the build configurations to the other projects, or including the projects in these build configurations, since there is no way to CREATE a build configuration if the name already exists (which Visual Studio thinks it does, thanks to the Common projects being included).
My goal is to add the same configurations (ie "Debug-QA", "Debug-Dev", etc.) into MyNewSolution, and its projects, so that all of the projects and solutions match. The only way I can see to do this is to manually create the build configuration on each new project... which is torture since MyNewSolution has approximately 50-100 projects.
FYI: I'm using Visual Studio 2012
This is more of a hack than a proper solution, but you could always:
Create a a copy of the configuration you want for the entire solution with a GUID as its name.
Remove the copy from the projects which already have the original configuration (undo changes).
Rename the GUID to the original configuration's name using the "Find/Replace in Files" tool.
Even if it isn't a very viable option, it is a pretty good quick fix.
EDIT:
How to manually remove configurations from solutions:
With the solution file opened in a text editor, you will see a block called Global which contains sections. The SolutionConfigurationPlatforms section contains the configuration's definitions. There is also an other section called ProjectConfigurationPlatforms where the configurations are assigned. Simply remove the references to the configuration from both groups and that should do it. If you have more complex solutions, there might be other references to remove. This is just a base case.
How to manually remove configurations from projects:
Again, with the project file opened in a text editor, you will see many references to the configuration you want to remove. C# projects have a PropertyGroup with the configuration as a condition. You can simply remove the group entirely. There might be other references to the configuration around the file so make sure to clean everything properly.
Make sure that you have a backup of your files if something goes wrong.
If there is very little project-level customization of the target configurations, this would probably be simplest to manage via an externalized configuration imported into your individual projects via the MSBuild Import element.
In order to allow project-specific overrides, this Import should be placed near the top of the project files. e.g.:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\..\CommonConfig.targets" Condition="Exists('..\..\CommonConfig.targets')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
...
This would, unfortunately, require you to edit all the project files once to add the Import. However, once this is done, you would be able to add a configuration to the imported .targets file and have it automatically propagate to all the projects.
I can't tell you what will work best for you, but I can say from experience that it becomes very easy to create new solution spaces with 50+ projects from old solution spaces when you don't let Visual Studio write the project files. Instead, you can take off-the-shelf third-party software that will read some configuration files you give it and in a few seconds will spit out a solution and all the projects you need, all configured the way you need them to be.
The same software would also generate the standalone "Common" solution when you just want to compile the common projects.
The right software will give you the flexibility and power that makefiles provide when you're setting up your projects, but you still get to do your work in Visual Studio.
I've used CMake extensively in that role, but for C++ rather than C#. I'm very happy with CMake; I have used it in environments with 50+ projects whose source code is scattered so far and wide that I use scripts to find it all, with some third-party libraries that are brought in as precompiled DLLs or LIBs. Also, from experience I know there's nothing stopping someone from concurrently maintaining their own hand-crafted set of VS project files for the same source code if they really want to. But of course you may want to do your own shopping around for the best software for your environment.
It is a significant investment of time to convert several dozen projects from hand-crafted project files to a more makefile-like system, but in an environment that requires reconfiguring or recombining the projects many times, I felt the investment paid back rather quickly.
I am trying to set global rules for my team. We are using VS2012 with TFS for our C# projects. I'd like to suppress some of the warnings and also treat some of the warnings as errors. I found the way to do it on the project level - project properties -> build tab.
But we have solution with more than hundred projects and I am looking for some easier way to set those rules globally.
A solution is just a (pretty dumb) container for projects. If you open it in a text editor you'll quickly see you can't extend it, only add projects/items.
What you want is one or more common msbuild files specifying all needed options for compiler/linker/whatever tools you use, and Import it in every single project. We've been using this for years and it's very convenient (though part of the convenience is probably we also wrote a small tool to generate project files to automatically import the global properties so we don't have to mess with them manually)
Alternatively you could add a machine wide file, look in $(MSBuildToolsPath)\Microsoft.CSharp.targets to see where to place those files. I'm not going to copy/paste the content here, but the very first lines basically check if there are user definded files in eg $(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\$(MSBuildThisFile)\ImportBefore and if so they're all imported before all common msbuild stuff. Likewise the end of the Microsoft.CSharp.targets contains similar logic to import files after all common msbuild stuff.
As of MSBuild 15 (circa 2017) you can use Directory.Build.Props file in the top folder of your solution. The syntax is the same as csproj, fsproj, or vbproj file and the entries are treated as though they are injected into all project files. You will need to restart Visual Studio to apply the changes. (thanks Jumbo!)
<Project>
<PropertyGroup>
<WarningsAsErrors>CS4014, CS1998</WarningsAsErrors>
</PropertyGroup>
</Project>
We're trying to use T4 with Visual Studio 2010 (SP1) to build scripts for another language that are based upon some of our existing C# classes. I'm hoping for the following:
The template needs to load our existing assembly and use objects from a namespace in that assembly.
The transformation needs to run on every build on every development machine and build server without any additional installations.
(1) and (2) need to work together.
(1) was fairly straightforward:
<## assembly name="$(TargetDir)RequiredProject.dll" #>
<## import namespace="RequiredProject.RequiredNamespace" #>
Using the $(TargetDir) macro allowed me to reference the dll with a fully qualified UNC path (per the instructions found here).
(2) is a bit roundabout, but I think I've got it solved: I installed the required text transformation SDKs on a different machine and copied the required .targets and .dlls into a folder in my solution and then updated my .csproj file to reference the local .targets file.
(3) is where I run into problems. It seems like the <TransformOnBuild>true</TransformOnBuild> property doesn't play nicely when a referenced assembly needs to be built prior to the transformation. Everytime I enable transform on build with referenced assemblies, I get the following error:
Compiling transformation: Metadata file '$(TargetDir)RequiredProject.dll' could not be found.
However, I'm using the same assembly instruction that I was using in (1) to reference the assembly. In fact, going to the .tt template directly and saving it still produces the expected output -- it just doesn't work during the "build" step. Am I doing something wrong, or is there a way to ensure that the template transformations occur after the assemblies they depend on are built? (Or, more simply, that template transformations occur last?)
Unfortunately, the msbuild T4 host doesn't yet support embedded macro or msbuild variables in assembly names.
However, it does support Windows environment variables "%foo%", so although it means some machine-level setup, you can get something that works across in-IDE and build time transforms.
My understanding is that Visual Studio 2013 will finally solve this problem, but that doesn't do me much good as I'm still on Visual Studio 2012. After a lot of effort I finally ran across a solution.
In the project that has the template you wish to run, add the following as a pre-build step on the Build Events tab of the project properties page.
set textTransformPath="%CommonProgramFiles(x86)%\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\TextTransform.exe"
if %textTransformPath%=="\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\TextTransform.exe" set textTransformPath="%CommonProgramFiles%\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\TextTransform.exe"
set ProjectDir=$(ProjectDir)
%textTransformPath% "%ProjectDir%StringGenerator.tt"
The first two lines take care of the differences between locating TextTransform.exe on 32-bit and 64-bit systems. The third line is the key. I need the path to the project location inside my template, so I set a local environment variable equal to the value of the build's $(ProjectDir) property. Inside my template, just use the following:
var projectDir = Environment.GetEnvironmentVariable("ProjectDir");
This has solved my issue.
I created a seperate solution that contained my needed referenced assemblies. The I had my buildscript build the reference solution first, then transform the templates, then build the solution containing the generated code.
If you want to reference dependency assemblies within a T4 script using macros and have text templating succeed during build-time, then you have to use project properties.
Within your project:
<Import Project="$(ProgramFiles)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\TextTemplating\Microsoft.TextTemplating.targets" />
<PropertyGroup>
<T4ProjectDir>$(ProjectDir)</T4ProjectDir>
</PropertyGroup>
<ItemGroup>
<T4ParameterValues Include="T4ProjectDir">
<Value>$(T4ProjectDir)</Value>
<Visible>false</Visible>
</T4ParameterValues>
</ItemGroup>
Where the path to your text templating environment may be different.
Then use $(T4ProjectDir) as you would use any other macro in your text template.
Or you could also simply refer to existing properties:
<ItemGroup>
<T4ParameterValues Include="ProjectDir">
<Value>$(ProjectDir)</Value>
<Visible>false</Visible>
</T4ParameterValues>
</ItemGroup>