We Keep Coding

msbuild cannot copy manifest because it was not found

We have a c# framework (not .core) solution with several projects in it. It is build by TFS. Sometimes (not all the times) I got a build error:
error MSB3030: Could not copy the file
"c:\BuildAgent_work\27\b\Console.Admin\Product.Admin.exe.manifest"
because it was not found.
[c:\BuildAgent_work\27\s\Console.Admin\Console.Admin.csproj]
The app.manifest file is added to the project Properties folder, and I checked it exists on buildagent source folder. I checked it is not exists on the binaries folder. I don't know why it is not copied there during the build.
In fact I don't know if I need this manifest thing at all. I think I don't. This whole thing was added to the project by one of my collegaue for a reason is unknown for me. Is it required to create publish package for web projects? In this case why it is required for a console project? For what reason is it added?
The msbuild parameters (for building the solution) are the following, and the [x] Clean option is checked in the TFS build solution step.
/p:OutDir=$(Build.BinariesDirectory)
/p:GenerateProjectSpecificOutputFolder=true
/p:DeployOnBuild=true /
p:PackageAsSingleFile=true
/p:GenerateDocumentation=true
/p:DisableAllVSGeneratedMSDeployParameter=true
/t:Clean,Build,Publish
/p:RunCodeAnalysis=$(CodeAnalysis.Run);CodeAnalysisRuleSet=$(CodeAnalysis.RuleSet).ruleset;CodeAnalysisIgnoreGeneratedCode=true
/p:IncludeAppPool=true
/p:PrecompileBeforePublish=true;EnableUpdateable=false

An application manifest is an XML file that describes and identifies the shared and private side-by-side assemblies that an application should bind to at run time. These should be the same assembly versions that were used to test the application. Application manifests may also describe metadata for files that are private to the application. You can refer to this document.
So, you first need to check in csproj for any additional operations on the manifest. When the MSBuild creates the publish files it copies files base on the build definition , you can try to edit the .csproj file as bellow which copied the relevant files. app.manifest is your file path .
<ItemGroup>
<None Update="app.manifest">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>

IIS Session Lost on Publish Web

Previously with my Web project in Visual Studio 2012 I was able to hit the 'Publish Web' button and as long as I did not make any changes to any .config or .cs files most of the time my session data would persist so I wouldn't have to log in again every time I make a small change to a .css or .html file.
After messing around with some settings in order to get debugging functioning with w3wp.exe now the session data is lost every single time I click Publish Web, even if I made absolutely no changes between publishes.
I don't know what I did to change this but I really need to be able to make changes to static files without having so sign in every single time. How do I stop the session from being killed?

It depends on what you're publishing. Even if you didn't make any changes knowingly, the app domain will unload if:
web.config was copied again (contents don't matter)
bin folder was modified (again, same dlls with newer timestamps are considered modification)
AFAIK, Publish always updates web.config. One thing I'm not sure is whether it updates web.config if you don't have any transforms (.debug.config or .release.config) or not.
There are couple of things you can try:
Set web.config's Build Action to "None". That'd prevent it being copied to output altogether.
Define a new Build Configuration, say "Static Content". Using this Build Configuration, define a new publish profile, say "Static Content". Add these lines to your .csproj file
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
...
<OutputPath>bin\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<ExcludeFilesFromDeployment>web.config</ExcludeFilesFromDeployment>
<ExcludeFoldersFromDeployment>bin</ExcludeFoldersFromDeployment>
</PropertyGroup>
This would prevent these 2 folders from being deployed so your app domain won't unload and session won't be lost. And you can still use the other profile to do a full release when needed.
In your Publish > Settings > File Publish Options, ensure you have "Delete all existing files..." option unchecked.
Hopefully, one or all of these workarounds will resolve your issue.
Update: Based on the comments, here's a simple AfterBuild Target that recursively copies only changed files to publish folder. Simply copy the following code into your .csproj file and tweak the paths appropriately.
<!-- define static content to be copied over -->
<ItemGroup>
<SourceFiles Include="CSS\**\*.css" />
<SourceFiles Include="Scripts\**\*.js" />
</ItemGroup>
<!-- The target that gets executed after build finishes and copies files recursively -->
<Target Name="AfterBuild">
<Copy
SourceFiles="#(SourceFiles)"
DestinationFolder="C:\Publish\YourApp\%(RecursiveDir)"
SkipUnchangedFiles="True"
/>
</Target>
Now when you build, you should see the static content getting published to your publish folder.
Notes:
%(RecursiveDir) copies files recursively, which helps in rebuilding the folder tree. Helpful if you have nested folders such as css\fonts and css\images.
Copy task has several other attributes (such as ContinueOnError, OverwriteReadOnlyFiles) that you may want to tinker with.
You can also use standard VS macros in the destination path (e.g. $(SolutionRoot), $(OutputDir) etc.).

Do you mean asp.net session? If yes then it is restored each time when the application pool is restarted . It could be caused by:
web config change
bin folder content change
IIS app pool recycle
Generally it is a wrong practice to rely on session data. You have to always check for the data existence and restore if not exit without forcing the user to log in.
But you can keep the session alive if you run it in SateServer or SqlServer mode:
Session-State Modes

Solution 1:
Setup a new site in IIS to hold all of your static files. E.g.
-Make sure it uses a different Application Pool than your other site
NewSite/Css/.css
NewSite/Scripts/.js
NewSite/JQuery
NewSite/BootStrap
(don't actually create this structure, you can manage it with a new vs project)
etc
Give it a localhost host header (MachineName and Port) internal access only.
Now Create a new Project in the Solution with your existing project, move said files to it matching a structure like above.
Setup a new publish profile to deploy your css, scripts, etc etc to the New site you made http://{machinename}:port
Call the Profile something like Dev_PublishStatics or Prod_PublishStatics (w/e you want)
Now you can make changes to CSS/Scripts in Project 2 and publish them with resetting Site A
-Lastly
In your Main site, Create virtual directories that point to the directories for Css, Scripts etc etc in the site you made above.
--Notes
You can also move ASPX files etc etc to the new Site and map them via virtual directories too if you want, but the dlls would need to be in both sites Bin folders. But it works, just remember that if you changed code you need to publish to both sites.

Solution 2:
Change the app pool settings,
IIS Manager
-> Application Pools
->-> Select Site Application Pool
->->-> Advanced Settings on the actions menu
Now set Disable Recycle for Configuration Changes to True, this will prevent the site from resetting on configuration changes. You will need to remember to reset it yourself when new code needs loaded.
I also recommend setting
Start Mode = Always Running
Disable Overlapped Recycle = true

Clickonce and postbackevent

I have a WPF project and a post build event which copy files from a folder outside of the solution into the output directory.
I want to publish my application with Clickonce publish. The only problem is that the copied files are not included in the publish or the manifest.
I tried using MageUI.exe , msbuild /target:publish from the visual studio cmd and even tried to change the project file by hand including a beforepublishevent but none of that worked.
I am open to suggestions, but what I want is to take the output folder and make the installer install the output files. (e.g. install the .net 4 framework and visual c++ runtime libs)

If you don't want them in the solution, but you do want them in ClickOnce then the only possibility is to use an external tool to create the ClickOnce manifests as you can't add non-project files to the ClickOnce output in Visual Studio.
There are two options I know of:
Use MageUI to manage your ClickOnce manifests. You can add additional files to your application in Mage.
You could use some software that I created and sell called ClickOnceMore (www.clickoncemore.net). ClickOnceMore will allow you to add all the files in one directory to your ClickOnce manifests by adding a single folder include. You can also then control which sub folder on the client they get deployed to. It was designed to make scenarios like this simple. You can download a free trial on the web site.
I hope it helps. Apologies for the marketing plug, but I do think ClickOnceMore can solve your problem perfectly.

Instead of copying the files through a post-build event, why don't you include them inside the project, and mark Copy to Output Directory = Copy if Newer?
If they are DLLs, you can add them as reference instead.
There are other options to create installers such as WiX+SharpSetup. It's much more flexible, but also more complicated, and takes more time to create simple installers.

Multiple .NET Configuration Files and Setup Project Problem

In order to handle settings for different deployment targets, I moved application settings from app.config to its own file and included that file in app.config via configSource. I also created a settings file for each target.Here is an illustration:
Project A
app.config (references settings.config)
settings.config
settings.Release.config
settings.Debug.config
During post-build, I copy the appropriate settings.{configuration}.config to the output directory. This is working fine so far and I can see settings.config file in the project output directory containing settings for the current build configuration: Release, Debug, etc.
However, I am having a problem with the setup project that I have for this project (Project A). Initially, it was not including settings.config file. So I set the build action for settings.config file as Content and I added content files from Project A to the setup project. This ensured that settings.config file was included in the setup. However, since the setup project appears to be picking settings.config file from the project directory instead of the output directory, settings.config file included in the setup is not what it should be. I want the one from the output directory to be included in the setup program since that one is the correct one for the current build configuration. I tried the following:
Added settings.config as a file to the setup project. However, it seems like I can only specify absolute path. So when I add it from the output directory of a particular build configuration (..bin\debug\settings.config), it does not work in other build configuration since (..bin\debug\settings.config) does exist in the directory specified. I looked into using relative paths or dynamic paths in the setup project where the build configuration could be specifed as part of the path but I could not find anything.
I considered using pre-build event to actually modify settings.config file in the project directory and then have it copied over the output directory by setting its 'Copy to Output Directory' to copy always or copy if newer. This should ensure that the appropriate settings.config is copied to the output directory just like the post-build based solution and should also ensure that the contents of settings.config file is updated before the setup project includes it. However, I don't like this solution because I would have to make sure settings.config file is writeable before I can make any changes since it is source controlled. If it is readonly, then I need to flip it to writeable, make changes, and then set it to readonly again. It is adding extra complexity.
I was wondering if anyone has a better idea or knows a setup project trick that allows me to include settings.config file appropriate for the current build configuration in the setup program.
Thanks

If I had to approach this problem, I'd start by asking the following question:
Why does settings.config have to be under source code control if settings.Debug.config or settings.Release.config provide the same information?
The answer, if I read your question correctly, is because you needed to force a settings.config file to appear as part of the build output. I'm guessing this is because your setup project is using the built in "Primary output" choice.
What you can do instead is add that file to your setup project as an explicit file reference. Right-click on the setup project and choose add / file, then select the file you want to include. As you'll notice (unless it's been fixed in VS2008 which sadly I'm not yet allowed to use at work), there is a very annoying limitation placed on manually added files - there is no way to make the path build configuration aware. You can work around that by copying the appropriate settings.config file to a common location (e.g. bin/Configuration) and picking it up from there. This does limit you to building Debug and Release versions sequentially, rather than in parallel, but for many this probably isn't a huge problem.
If you aren't required to use VS setup projects, I strongly encourage you to take a look at WiX (Windows Installer XML - see http://wix.sourceforge.net/ for more information). That will easily allow you to accomplish what is necessary, although if you are unfamiliar with the internal workings of Microsoft Installer the initial learning curve could be a little steep. Microsoft use WiX themselves for some pretty significant setup tasks (e.g. Office 2007, SQL Server, etc.). It had been hoped that WiX would become part of Visual Studio (for VS 2010), but sadly that is no longer the case.

I decided to go about achieving the same result (being able to have different configuration settings for different target environments) in a different way. So here is how I implemented it and it is working great. I read some of the posts here at SO about XmlMassUpdate task from MSBuild Community Tasks and decided to utilize it. Here is what I did:
1) For each project that needs to have different settings depending on the target environment, I added an xml file called app.config.substitutions.xml or web.config.substitutions.xml to the project. So, the project looked like
Project A
app.config
app.config.substitutions.xml
app.config.substitutions.xml file has the settings substitutions that XmlMassUpdate will process and apply to app.config file. Below is a sample substitution file that I use:
<configuration xmlns:xmu="urn:msbuildcommunitytasks-xmlmassupdate">
<substitutions>
<Development>
<appSettings>
<add xmu:key="key" key="SomeSetting" value="DevValue" />
</appSettings>
</Development>
<Test>
<appSettings>
<add xmu:key="key" key="SomeSetting" value="TestValue" />
</appSettings>
</Test>
<Release>
<appSettings>
<add xmu:key="key" key="SomeSetting" value="ReleaseValue" />
</appSettings>
</Release>
</substitutions>
</configuration>
For details on how to specify substitutions, take a look at the documentation for XmlMassUpdate or just do a search on it.
2) Now I need to run XmlMassUpdate as part of build automation (TeamBuild/MSBuild). So in BeforeCompile in TeamBuild build definition file (basically a proj file), I added the following to run XmlMassUpdate on config files that have a corresponding .substitution.xml file
<PropertyGroup>
<SubstitutionFileExtension>.substitutions.xml</SubstitutionFileExtension>
<TargetEnvironment>Test</TargetEnvironment>
</PropertyGroup>
<Target Name="BeforeCompile" Condition="'$(IsDesktopBuild)'!='true'">
<CreateItem Include="$(SolutionRoot)\**\app.config;$(SolutionRoot)\**\web.config">
<Output ItemName="ConfigurationFiles" TaskParameter="Include"/>
</CreateItem>
<CreateItem Include="#(ConfigurationFiles)" Condition="Exists('%(FullPath)$(SubstitutionFileExtension)')">
<Output ItemName="ConfigFilesWithSubstitutions" TaskParameter="Include"/>
</CreateItem>
<Message Text="Updating configuration files with deployment target specific settings..."/>
<XmlMassUpdate
ContentFile="%(ConfigFilesWithSubstitutions.FullPath)"
SubstitutionsFile="%(ConfigFilesWithSubstitutions.FullPath)$(SubstitutionFileExtension)"
ContentRoot="/configuration"
SubstitutionsRoot="/configuration/substitutions/$(TargetEnvironment)"/>
</Target>
Note that config files are read-only during the build, I make sure to set them writeable before running this task. I actually have another custom MSBuild task that runs before XmlMassUpdate that handles common settings throughout all of the config files such as connection strings. That task makes the config files writeable. I also don't check modified config files back to the source control. They're (appropriate config file for the deployment target) included in the installer.

Getting Content Files and Primary Output programmatically

For some reason, we have a script that creates batch files to XCOPY our compiled assemblies, config files, and various other files to a network share for our beta testers. We do have an installer, but some don't have the permissions required to run the installer, or they're running over Citrix.
If you vomited all over your desk at the mentions of XCOPY and Citrix, use it as an excuse to go home early. You're welcome.
The code currently has hundreds of lines like:
CreateScripts(basePath, "Client", outputDir, FileType.EXE | FileType.DLL | FileType.XML | FileType.CONFIG);
It used to be worse, with 20 int parameters (one per file type) representing whether or not to copy that file type to the output directory.
These hundreds of lines create upload/download batch files with thousands of XCOPY lines. In our setup projects, we can reference things like "Primary output from Client" and "Content Files from Client". I'd love to be able to do that programmatically from a non-setup project, but I'm at a loss.
Obviously MS does it, either using an API or by parsing the .csproj files. How would I go about doing this? I'm just looking for a way to get a list of files for any of the setup categories, i.e.:
Primary Output
Localized Resources
Content Files
Documentation Files
EDIT:
I have a setup project like Hath suggested, and it's halfway to what I'm looking for. The only problem keeping that from being a perfect solution is that multiple projects depend on the same assemblies being in their own folder, and the setup will only copy the file once.
Example:
Projects Admin, Client, and Server all rely on ExceptionHandler.dll, and Admin and Client both rely on Util.dll, while Server does not. This is what I'm looking for:
Admin
Admin.exe
Admin.exe.config
ExceptionHandler.dll
Util.dll
Client
Client.exe
Client.exe.config
ExceptionHandler.dll
Util.dll
Server
Server.exe
Server.exe.config
ExceptionHandler.dll
Since the referenced assemblies are all the same, what I get is this:
Admin
Admin.exe
Admin.exe.config
ExceptionHandler.dll
Util.dll
Client
Client.exe
Client.exe.config
Server
Server.exe
Server.exe.config
This causes a FileNotFoundException when either Client or Server can't find one of the two DLLs it's expecting.
Is there a setup property I'm missing to make it always copy the output, even if it's duplicated elsewhere in another project's output?
EDIT AGAIN: All referenced DLLs are set to "Copy Local", and always have been. I found a decent article on using NAnt and XSLT to grab the list of files, so that may be a possible solution as well, as neouser99 suggested.
ACCEPTED SOLUTION: I'm pretty much back where I started. All .exe and .dll outputs are put into a "bin" directory in the setup project, loosely packed. The other per-application folders contain shortcuts to the executable in that directory.
The difference now is, I'm going to add a custom action to the installer to use reflection, enumerate the dependencies for each executable output, and copy the .exe and .dll files to the separate directories. Bit of a pain, as I just assumed there was a way to programmatically detect what files would be included via some setup library.

why not use another setup project and just set the 'Package files' setting to As Loose uncompressed files (setup project->properties)? then share the folder.. or something.
edit:
I see, you have 3 folders for your outputs. but the setup project only detects the ExceptionHandler.dll and Util.dll once, so it will just pick the first folder and put it in there.
You could do a setup project for each project - bit annoying maybe..
You could manually add in the dll's to the projects that are missing the assembly's
either by adding in the File by 'add file' or 'add assembly' or 'add project output' if you have those projects in the same solution.. (I doubt that's the case though).
or just dump all of them into one output directory...

Although it's designed as a build tool, you might find NAnt to be extremely useful in what you are talking about. The tasks (build, copy, move, delete, etc.) that you can define allow for very fine-grained file lookups, up to general, full folders. If you also incorporate NAnt into your build process, I think you could find that it helps out in more ways then one.

Another approach that has worked for me in the past is to add the shared resource (Assembly, DLL or project) as a reference to each of the Admin, Server and Client projects. Then open the properties panel for the referenced item in each project and set "Copy Local" to true.
Now when you build the projects, each will have its own instance of the Assembly copied into its output folder.
This should also cause the shared components added in this manner to be replicated in each of the output folders in the setup package.

A completely different approach could be to set them up as symbolic links on the network share. A symbolic link is basically a short-cut where the file-system hides the fact that it is a short-cut, so all other applications actually believes that the file has been copied (http://en.wikipedia.org/wiki/NTFS_symbolic_link).
One advantage of this approach is that the file is updated immediately as the file changes and not only when you build your projects. So when you for instance save one of the config-files with a text-editor the update is applied immediately.

The following MSBuild script part can build your SLN file (you can replace it with .csproj) and will report a list of all projects that were build (Dlls, EXEs).
<MSBuild Projects="MySolution.sln" Targets="Clean; Rebuild" Properties="Configuration=$(BuildMode);">
<Output TaskParameter="TargetOutputs"
ItemName="AssembliesBuilt" />
</MSBuild>
Now, this doesn't really solve your problem, but it gets you a list of everything that was build. You also have copylocal, so you could probably just take AssembiesBuild and copy all DLL and .CONFIG files from there.
Example:
AssembliesBuild = c:\myproj\something1\build.dll
you'd go to c:\myproj\something1\ and simply search for all *.dll and *.config files and include them. You can do this pretty easily with MSBuild or powershell, if you have it installed. To output a XCOPY script from MSBuild, I think you'll need MSBuild contrib projct installed.