Running Fortify on .dll files as well as .cs - c#

.NET version of existing question Can I run fortify on .jar files instead of .java?
My team is using Fortify 5.15.0.0060. We are scanning our .NET project in Visual Studio, and project settings shows the following command line arguments:
-vsversion 10.0 "-b" "project" "-machine-output" "-scan" "-f" "C:\Users\user\AppData\Local\Fortify\VS2010-3.80\project\Scan.fpr" "-format" "fpr"
This highlights issues in our .cs source code files, but does not appear to look at the DLLs that form part of our solution (and which are copied to our output directory along with our binaries). For many of these libraries we do not have access to .pdb files, but were hoping that Fortify would be able to do a limited amount of scanning anyway.
Is it possible to add a command line argument to include DLL files, with/without PDB files?

When it comes to .Net, Fortify can only scan assemblies that have a .pdb because it uses ildasm.exe to decompile the assembly and then it uses the .pdb to match to the source files. Unfortunately there is no way for you to scan these extra assemblies.
If you have the .pdb files, you can have Fortify scan them, however you may not see the full source in your results. To include these assemblies, you need to specify them in your Translation options. If you are doing this all from the command line, then this is how you would do it:
sourceanalyzer -b project -vsversion 10.0 projectPath\Additional.Assembly.1.dll projectPath\Additional.Assembly.2.dll projectPath\Additional.Assembly.3.dll
sourceanalyzer -b project -scan -f MyResults.fpr
If you have not yet updated your Fortify version, I would highly recommend that you do so.

Related

How does C# compiler search for libraries during compilation?

A C# program uses .Net framework libraries or third party libraries by using directive in the C# code.
When compiling the program using csc,
how shall we specify the search path for the used libraries?
does the compiler search for the used libraries in some search order?
I am comparing csc to gcc and javac in terms of specifying search paths for libraries and searching order. For gcc and javac, there are multiple ways to specify search paths,
command line options to them, such as -L to gcc and -classpath to javac;
environment variables, such as LIBRARY_PATH for gcc and CLASSPATH for javac;
default paths, such as current directory, some system-wide directory
Does csc also have multiple ways, and have a search order between them?
Same questions if compiling the program using msbuild which calls csc.
Thanks.
how shall we specify the search path for the used libraries?
Use the /lib command line option.
Syntax
/lib:dir1[,dir2]
dir1
A directory for the compiler to look in if a referenced assembly is not found in the current working directory (the directory from which you are invoking the compiler) or in the common language runtime's system directory.
dir2
One or more additional directories to search in for assembly references. Separate additional directory names with a comma, and without white space between them.
OP
Same questions if compiling the program using msbuild which calls csc
If you use msbuild you are more than likely using a .proj file so it would use the path setup for your environment I would expect
Tell me more
-lib

C# analysis fails without .pdb files

I'm trying out the new C# plugin v3.0 with SonarQube 4.2.
According to plugin documentation, I need .pdb files to run analysis which includes FxCop rules, and indeed I get a failure message when executing sonar-runner if said .pdb files are not present in the output folder; FxCop exits with code 1536.
The thing is, the analysis seems to require .pdb files for ALL the binnaries, and the output folder contains several dependencies in addition to my own compiled code. These are NuGet packages and I cannot find symbol files for all of them.
So, is there any way I can have the analysis to skip the files without debugging symbols?
The regular SonarQube file exclusion lists seem to apply to source code files only (e.g. *.designer.cs) and not for binnaries.
This is a bug on the SonarQube C# plugin side. I have created the following ticket to fix it: https://jira.codehaus.org/browse/SONARFXCOP-29
Note that this is not related to the presence of *.pdb files, but only on the presence of the referenced assemblies.

C# commandline decompiler

I'm looking to decompile a lot of assemblies into C#. I found several tools that do a visual decompile. However I was wondering if there is a tool that can do decompiling from the command line (similar to ildasm /OUT).
The motivation is that there are 100+ of assemblies and I don't want to open each one and save as an .cs file. .NET Reflector seems to have a batch load of assemblies, but it doesn't have a batch save. So I was thinking of writing a script that goes through each assembly and decompiles it using a command line command.
If you are looking to have a program that generates the C# code for an assembly, Jon Gallant recently had a blog post about doing this using JustDecompile from Telerik. There are a couple of assemblies that you link to and then you can control the generation of the code without a UI.
The only thing you need is this open source decompiler called dnSpy -> https://github.com/0xd4d/dnSpy
includes a command line tool:
Examples:
dnSpy.Console.exe -o C:\out\path C:\some\path
Decompiles all .NET files in the above directory and saves files to C:\out\path
dnSpy.Console.exe -o C:\out\path -r C:\some\path
Decompiles all .NET files in the above directory and all sub directories
dnSpy.Console.exe -o C:\out\path C:\some\path\*.dll
Decompiles all *.dll .NET files in the above directory and saves files to C:\out\path
dnSpy.Console.exe --md 0x06000123 file.dll
Decompiles the member with token 0x06000123
dnSpy.Console.exe -t system.int32 --gac-file "mscorlib, Version=4.0.0.0"
Decompiles System.Int32 from mscorlib
Just as a continuation of the answer from above, dnSpy seems to be discontinued and crashed for me when I tried to use it (like here).
Maybe consider using dnSpyEx, which worked like a charm.

What is function of each file in a simple C# project?

I want to ask about the files that are found in the folder of a simple C# Project .
like
(.pdb file , .vshost file , .manifest file ) in bin folder
( .csproj.fileListAbsolute.txt file , .pdb file ) in debug folder
(AssemblyInfo.cs ) in Properties folder
what is the function of each file of them ?
and which of them is in MSIL , if there is no file of them in this language how can I get file in MSIL ?
another question : what is the specific part that converts C# code into MSIL ?? , Is it the C# Compiler ? Is there any specific name to it ?
Technically, the pdb, vshost, and manifest file are not part of a C# project, the are part of the output generated when you build the project.
The PDB file contains symbol information used by the debugger to associate code within the assembly with your sources files. This allows the debugger to identify which line of source code corresponds a set of instruction in the MSIL of an assembly.
The vshost.exe file is a hosting process that visual studio generates that helps accelerate the debugging of an application. It caches app domain for your process to reduce the startup time for the debugger.
The manifest file contains information about the assemblies in a project and some common metadata about them. Its contents varies by project type.
The csproj file is xml-based representation of the code and resources for a project. It's the file that Visual Studio uses to figure out what code and resuorces to compile into an assembly. This file also contains project settings and options that control how the compiler (and other tools) process the assets of a project.
None of these files contains MSIL. The MSIL is part of the .exe and .dll files generated when you build a project. These files are encoded using the Microsoft PE (portable executable) format, and contain resources, data, and MSIL code.
The exe and dll, and pdb files are generated by the C# compiler - csc.exe. The other files (IIRC) are generated by other tools invoked by the msbuild process that controls the entire build cycle.
The executable (exe) or assembly (dll) will contain code in MSIL. The MSIL can be viewed using a tool like ILDASM or .NET Reflector. The C# compiler is csc.exe.
Here's a quick list:
PDB - These are debugging symbols used by Visual Studio and other debuggers.
VSHOST - This is a special executable that's hosted by Visual Studio (hence the name) I think this is used to provide intellisense and debugging help (e.g. breakpoints).
MANIFEST - Depends on the kind of project you're working on. Usually this file lets you specify special OS-level needs for your application (e.g. UAC admin rights... security model etc. etc.)
CSPROJ - Think of this as a Makefile/Buildfrile for your application. (CSPROJ means a CSharp Project) - This file list all the files necessary to build your application and also any extra processing steps (like a Makefile target) that might be necessary.
AssemblyInfo.cs - This is just a normal C# code file, but usually reserved for adding assembly-level attributes. (a .NET assembly metadata, as opposed to a normal EXE file metadata which is used by the operating system)
MSIL is Microsoft Intermediate Language which is an intermediate representation of the code resulting from compiling C# source code (or other .NET languages) using the C# compiler (csc.exe).
The .NET Runtime then interprets or JIT-compile the IL into actual machine code when you run the application.

ILMerge Best Practices

Do you use ILMerge? Do you use ILMerge to merge multiple assemblies to ease deployment of dll's? Have you found problems with deployment/versioning in production after ILMerging assemblies together?
I'm looking for some advice in regards to using ILMerge to reduce deployment friction, if that is even possible.
I use ILMerge for almost all of my different applications. I have it integrated right into the release build process so what I end up with is one exe per application with no extra dll's.
You can't ILMerge any C++ assemblies that have native code.
You also can't ILMerge any assemblies that contain XAML for WPF (at least I haven't had any success with that). It complains at runtime that the resources cannot be located.
I did write a wrapper executable for ILMerge where I pass in the startup exe name for the project I want to merge, and an output exe name, and then it reflects the dependent assemblies and calls ILMerge with the appropriate command line parameters. It is much easier now when I add new assemblies to the project, I don't have to remember to update the build script.
Introduction
This post shows how to replace all .exe + .dll files with a single combined .exe. It also keeps the debugging .pdb file intact.
For Console Apps
Here is the basic Post Build String for Visual Studio 2010 SP1, using .NET 4.0. I am building a console .exe with all of the sub-.dll files included in it.
"$(SolutionDir)ILMerge\ILMerge.exe" /out:"$(TargetDir)$(TargetName).all.exe" "$(TargetDir)$(TargetName).exe" "$(TargetDir)*.dll" /target:exe /targetplatform:v4,C:\Windows\Microsoft.NET\Framework64\v4.0.30319 /wildcards
Basic hints
The output is a file "AssemblyName.all.exe" which combines all sub-dlls into one .exe.
Notice the ILMerge\ directory. You need to either copy the ILMerge utility into your solution directory (so you can distribute the source without having to worry about documenting the install of ILMerge), or change the this path to point to where ILMerge.exe resides.
Advanced hints
If you have problems with it not working, turn on Output, and select Show output from: Build. Check the exact command that Visual Studio actually generated, and check for errors.
Sample Build Script
This script replaces all .exe + .dll files with a single combined .exe. It also keeps the debugging .pdb file intact.
To use, paste this into your Post Build step, under the Build Events tab in a C# project, and make sure you adjust the path in the first line to point to ILMerge.exe:
rem Create a single .exe that combines the root .exe and all subassemblies.
"$(SolutionDir)ILMerge\ILMerge.exe" /out:"$(TargetDir)$(TargetName).all.exe" "$(TargetDir)$(TargetName).exe" "$(TargetDir)*.dll" /target:exe /targetplatform:v4,C:\Windows\Microsoft.NET\Framework64\v4.0.30319 /wildcards
rem Remove all subassemblies.
del *.dll
rem Remove all .pdb files (except the new, combined pdb we just created).
ren "$(TargetDir)$(TargetName).all.pdb" "$(TargetName).all.pdb.temp"
del *.pdb
ren "$(TargetDir)$(TargetName).all.pdb.temp" "$(TargetName).all.pdb"
rem Delete the original, non-combined .exe.
del "$(TargetDir)$(TargetName).exe"
rem Rename the combined .exe and .pdb to the original project name we started with.
ren "$(TargetDir)$(TargetName).all.pdb" "$(TargetName).pdb"
ren "$(TargetDir)$(TargetName).all.exe" "$(TargetName).exe"
exit 0
We use ILMerge on the Microsoft application blocks - instead of 12 seperate DLL files, we have a single file that we can upload to our client areas, plus the file system structure is alot neater.
After merging the files, I had to edit the visual studio project list, remove the 12 seperate assmeblies and add the single file as a reference, otherwise it would complain that it couldnt find the specific assembly. Im not too sure how this would work on post deployment though, could be worth giving it a try.
I know this is an old question, but we not only use ILMerge to reduce the number of dependencies but also to internalise the "internal" dependencies (eg automapper, restsharp, etc) that are used by the utility. This means they are completely abstracted away, and the project using the merged utility doesn't need to know about them. This again reduces the required references in the project, and allows it to use / update its own version of the same external library if required.
We use ILMerge on quite a few projects. The Web Service Software Factory, for example produces something like 8 assemblies as its output. We merge all of those DLLs into a single DLL so that the service host will only have to reference one DLL.
It makes life somewhat easier, but it's not a big deal either.
We had the same problem with combining WPF dependencies .... ILMerge doesn't appear to deal with these. Costura.Fody worked perfectly for us however and took about 5 minutes to get going... a very good experience.
Just install with Nuget (selecting the correct default project in the Package Manager Console). It introduces itself into the target project and the default settings worked immediately for us.
It merges the all DLLs marked "Copy Local" = true and produces a merged .EXE (alongside the standard output), which is nicely compressed in size (much less than the total output size).
The license is MIT as so you can modify/distribute as required.
https://github.com/Fody/Costura/
Note that for windows GUI programs (eg WinForms) you'll want to use the /target:winexe switch. The /target:exe switch creates a merged console application.
I'm just starting out using ILMerge as part of my CI build to combine a lot of finely grained WCF contracts into a single library. It works very well, however the new merged lib can't easily co-exist with its component libraries, or other libs that depend on those component libraries.
If, in a new project, you reference both your ILMerged lib and also a legacy library that depends on one of the inputs you gave to ILMerge, you'll find that you can't pass any type from the ILMerged lib to any method in the legacy library without doing some sort of type mapping (e.g. automapper or manual mapping). This is because once everything's compiled, the types are effectively qualified with an assembly name.
The names will also collide but you can fix that using extern alias.
My advice would be to avoid including in your merged assembly any publicly available lib that your merged assembly exposes (e.g. via a return type, method/constructor parameter, field, property, generic...) unless you know for sure that the user of your merged assembly does not and will never depend on the free-standing version of the same library.
We ran into problems when merging DLLs that have resources in the same namespace. In the merging process one of the resource namespaces was renamed and thus the resources couldn't be located. Maybe we're just doing something wrong there, still investigating the issue.
We just started using ILMerge in our solutions that are redistributed and used in our other projects and so far so good. Everything seems to work okay. We even obfuscated the packaged assembly directly.
We are considering doing the same with the MS Enterprise Library assemblies.
The only real issue I see with it is versioning of individual assemblies from the package.
I recently had issue where I had ilmerged assembly in the assembly i had some classes these were being called via reflection in Umbraco opensource CMS.
The information to make the call via reflection was taken from db table that had assembly name and namespace of class that implemented and interface. The issue was that the reflection call would fail when dll was il merged however if dll was separate it all worked fine. I think issue may be similar to the one longeasy is having?
It seems to me like the #1 ILMerge Best Practice is Don't Use ILMerge. Instead, use SmartAssembly. One reason for this is that the #2 ILMerge Best Practice is to always run PEVerify after you do an ILMerge, because ILMerge does not guarantee it will correctly merge assemblies into a valid executable.
Other ILMerge disadvantages:
when merging, it strips XML Comments (if I cared about this, I would use an obfuscation tool)
it doesn't correctly handle creating a corresponding .pdb file
Another tool worth paying attention to is Mono.Cecil and the Mono.Linker [2] tool.
[2]: http:// www.mono-project.com/Linker

Categories