We Keep Coding

C# Roslyn to use previously compiled class in next compilations

I ask with example,
Lets say I have the following code.
fullcommand = #"public class oldTest
{
public static void oldTestMethod(){
Console.WriteLine(""oldTest Class"");
}
}"
var syntaxTree = CSharpSyntaxTree.ParseText(fullCommand);
var compilation = CSharpCompilation.Create(
assemblyName,
new[] {syntaxTree},
references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary,allowUnsafe:true));
var ms = new MemoryStream();
var result = compilation.Emit(ms);
And I will compile the above code with Roslyn in memory.
next i want to compile another code in memory to use the above compiled class, lets say the below.
new_fullcommand = #"public class newTest
{
public static void newTest(){
oldTest.oldTestMethod();
}
}"
var syntaxTree = CSharpSyntaxTree.ParseText(new_fullcommand);
var compilation = CSharpCompilation.Create(
assemblyName,
new[] {syntaxTree},
references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary,allowUnsafe:true));
var ms = new MemoryStream();
var result = compilation.Emit(ms);
how can i make the second code to use the first code as its reference? or use it?

The easiest way would probably be to pass in multiple syntaxTree objects when you create your compilation.
However, if you want to build up your compilation incrementally I believe you can use Compilation.AddSyntaxTrees to your first compilation object.

Problem with the Serializer for a Xslt2.0 transformation with Saxon

This is my first try to program a Xslt2.0 transformation with SaxonHE 9.9 in C#, so the problem here is when I create the serilizer I get the error that the class Saxon.Api.Serializer contains no constractor with 0 arguments.
I know what this error means, but not why it occurs, cause each example that I see creates the serializer like this.. This question sounds a bit stupid, but I cannot find a answer to get it work.
using Saxon.Api;
namespace XY
{
class Program
{
static void Main(string[] args)
{
String SourceFilename = "./test/test.xml";
String StylesheetFilename = "./scripte/xml-to-html.xsl";
String OutputFilename = "./Output/test.html";
using (FileStream streamXml = File.OpenRead(SourceFilename))
{
using (FileStream streamXsl = File.OpenRead(StylesheetFilename))
{
Processor processor = new Processor();
DocumentBuilder builder = processor.NewDocumentBuilder();
Uri uri = new Uri("urn:test");
builder.BaseUri = uri;
XdmNode input = builder.Build(streamXml);
XsltTransformer transformer = processor.NewXsltCompiler().Compile(streamXsl).Load();
transformer.InitialContextNode = input;
Serializer serializer = new Serializer();
serializer.SetOutputFile(OutputFilename);
transformer.Run(serializer);
}
}
Console.WriteLine("test.html created successfully");
}
}
}
EDIT
using System;
using Saxon.Api;
using System.IO;
using System.Reflection;
namespace XY
{
class Program
{
static void Main(string[] args)
{
string currentDirectory = Directory.GetCurrentDirectory();
String SourceFilename = ".\\test\\test.xml";
String StylesheetFilename = ".\\scripte\\xml-to-html.xsl";
String OutputFilename = ".\\Output\\result.html";
if (StylesheetFilename.StartsWith(".\\"))
{
StylesheetFilename = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "\\" + StylesheetFilename;
}
if (SourceFilename.StartsWith(".\\"))
{
SourceFilename = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "\\" + SourceFilename;
}
var uri_source = new System.Uri(SourceFilename);
var uri_xsl = new System.Uri(StylesheetFilename);
Processor processor = new Processor();
XdmNode input = processor.NewDocumentBuilder().Build(uri_source);
processor.SetProperty("http://saxon.sf.net/feature/preferJaxpParser", "true");
XsltCompiler compiler = processor.NewXsltCompiler();
XsltExecutable executable = compiler.Compile(uri_xsl);
XsltTransformer transformer = executable.Load();
transformer.InitialContextNode = input;
Serializer serializer = processor.NewSerializer();
System.IO.StreamWriter stream = new StreamWriter(OutputFilename);
serializer.SetOutputWriter(stream);
transformer.Run(serializer);
stream.Close();
}
}
}
I change also some other thinks and now it works, thanks for the answers.

I'll log a bug on the fact that there are sample apps and/or documentation that use the "new Serializer()" form.
We dropped this from the Java product in 9.8 because it caused constant trouble that the Serializer doesn't (necessarily) have access to all the configuration options (held in the Processor); also using a factory method Processor.newSerializer() potentially allows us to to create a subclass of Serializer, so it's more flexible. We then followed this pattern on .NET in the 9.9 release, partly for the same reasons, and partly because the .NET API has now been rewritten as a very thin layer on top of the Java API, which helps us to maintain commonality, and simplifies testing.
We try hard to maintain backwards compatibility in the main product APIs but it's not a requirement that overrides all others; if we feel that we got something badly wrong, then we fix it. As some people say to justify the policy, "the future is longer than the past".
LATER
We have done some checking and we think the 9.9 documentation and sample applications are correct; you must be using an older version. If I'm wrong, please identify the specific location where you found incorrect information.

In 9.9 you can (or really need to) create a Serializer with the various overloads of processor.NewSerializer (see http://saxonica.com/html/documentation/dotnetdoc/Saxon/Api/Processor.html#NewSerializer(Stream))..

Here is my solution for the problem:
using System;
using System.IO;
using Saxon.Api;
namespace Project1
{
public static class ClassMain
{
public static string TransformXml(string xmlData, string xslData)
{
var xsltProcessor = new Processor();
var documentBuilder = xsltProcessor.NewDocumentBuilder();
documentBuilder.BaseUri = new Uri("file://");
var xdmNode = documentBuilder.Build(new StringReader(xmlData));
var xsltCompiler = xsltProcessor.NewXsltCompiler();
var xsltExecutable = xsltCompiler.Compile(new StringReader(xslData));
var xsltTransformer = xsltExecutable.Load();
xsltTransformer.InitialContextNode = xdmNode;
var results = new XdmDestination();
xsltTransformer.Run(results);
return results.XdmNode.OuterXml;
}
public static void Main()
{
var xmlData = File.ReadAllText("a.xml");
var xslData = File.ReadAllText("a.xsl");
var data = TransformXml(xmlData, xslData);
Console.WriteLine(data);
Console.ReadKey();
}
}
}

Adding support for dynamic file loaders in c#

If I have a Console App in C# that reads in files of a certain format and converts them to business objects, I design this by having an IReader interface so that I can support different formats, eg XML, CSV, pipe delimited etc, and have different concrete classes for each file format.
If the requirement is to be able to load new file readers (new formats) in dynamically without having to recompile, is there a way I can accomplish that?
The only way I can think of is somehow using XSD or reg expressions but it seems to me there should be a better solution

This sounds like you want a plugin mechanism for loading your IReaders dynamically. There are plenty of examples out there.
Simple plugin mechanism sample
SO discussion

You could use reflection. Each implementation of IReader could go in a distinct DLL. You would also create an Attribute to tag each implementation of IReader that states which file format it handles.
public sealed class InputFormatAttribute : Attribute
{
private string _format;
public string Format
{
get { return format; }
}
public InputFormatAttribute(string format)
{
_format = format;
}
}
[InputFormat("CSV")]
public class CSVReader : IReader
{
// your CSV parsing code here
public BusinessObject Parse(string file)
{}
}
BusinessObject LoadFile(string fileName)
{
BusinessObject result = null;
DirectoryInfo dirInfo = new DirectoryInfo(Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location));
FileInfo[] pluginList = dirInfo.GetFiles("*.DLL");
foreach (FileInfo plugin in pluginList)
{
System.Reflection.Assembly assem = System.Reflection.Assembly.LoadFile(fileInfo.FullName);
Type[] types = assem.GetTypes();
Type type = types.First(t => t.BaseType == "IReader");
object[] custAttrib = type.GetCustomAttributes(typeof(InputFormatAttribute), false);
InputFormatAttribute at = (InputFormatAttribute)custAttrib[0];
if (at.Format.Equals(Path.GetExtension(fileName).Substring(1), StringComparison.CurrentCultureIgnoreCase))
{
IReader reader = (IReader)assem.CreateInstance(type.FullName);
return reader.Parse(fileName);
}
}
// got here because not matching plugin found
return null;
}

Depends on the complicity of Readers, you may decide to use CodeDom to let someone to write the code directly. Short example:
// create compiler
CodeDomProvider provider = CSharpCodeProvider.CreateProvider("C#");
CompilerParameters options = new CompilerParameters();
// add more references if needed
options.ReferencedAssemblies.Add("system.dll");
options.GenerateExecutable = false;
options.GenerateInMemory = true;
// compile the code
string source = "using System;namespace Bla {public class Blabla { public static bool Test { return false; }}}";
CompilerResults result = provider.CompileAssemblyFromSource(options, source);
if (!result.Errors.HasErrors)
{
Assembly assembly = result.CompiledAssembly;
// instance can be saved and then reused whenever you need to run the code
var instance = assembly.CreateInstance("Bla.Blabla");
// running some method
MethodInfo method = instance.GetType().GetMethod("Test"));
var result = (bool)method.Invoke(_formulas, new object[] {});
}
But probably you'll have to provide sort of editor or accomplish the task partially (so only necessary code have to be written).

Building programmatically a project

I need to build a project programmatically for a .csproj I am creating on the fly.
While searching Google I found the classes and API provided by the MS for the MSBuild Engine. With that information, I create a process which executes msbuild.exe and then reads the output, but now I want to use the namespace Microsoft.Build.Execution to build the project. This is my program:
public class Compiler
{
private static string locationOfMSBuilldEXE = "";
public static void Build(string msbuildFileName)
{
BuildManager manager = BuildManager.DefaultBuildManager;
ProjectInstance projectInstance = new ProjectInstance(msbuildFileName);
var result = manager.Build(new BuildParameters()
{
DetailedSummary = true
},
new BuildRequestData(projectInstance, new string[] { "Build" }));
var buildResult = result.ResultsByTarget["Build"];
var buildResultItems = buildResult.Items;
string s = "";
}
}
The results show that this is building fine, but I need to know the detailed output from the compile and how to view it. It would be really helpful if someone can give me link to a good tutorial or a book on MSBuild.

Thanks #ritchmelton. Though I figured it out myself.
Here is my code : I have used an in built logger ConsoleLogger
public class Compiler
{
private static string locationOfMSBuilldEXE = "";
public static void Build(string msbuildFileName)
{
ConsoleLogger logger = new ConsoleLogger(LoggerVerbosity.Normal);
BuildManager manager = BuildManager.DefaultBuildManager;
ProjectInstance projectInstance = new ProjectInstance(msbuildFileName);
var result = manager.Build(
new BuildParameters()
{
DetailedSummary = true,
Loggers = new List<ILogger>(){logger}
},
new BuildRequestData(projectInstance, new string[] { "Build" }));
var buildResult = result.ResultsByTarget["Build"];
var buildResultItems = buildResult.Items;
string s = "";
}
}

You need to add a instance of a class that implements the ILogger interface to your BuildParameters. You can add a new instance of one of the supplied loggers in the Microsft.Build.Logging namespace, or you can implement ILogger yourself as it is very small and there is a helper class in the Microsoft.Build.Utilities namespace called Logger that is easy to extend.
Build loggers
ILogger interface
Logger helper

If you just want to build a project or solution, without elaborate parameters, you can do it more simply. Pseudocode:
using namespace Microsoft.Build.Evaluation;
var p = Project.Load("path to project");
p.SetGlobalProperty("Configuration", "Release");
p.Build(...);
That's it! BuildParameters and so forth are for quite advanced scenarios. Visual Studio itself uses them.
Dan (msbuild dev)

.NET assemblies: understanding type visibility

I am trying to reproduce something that System.Xml.Serialization already does, but for a different source of data.
For now task is limited to deserialization only.
I.e. given defined source of data that I know how to read. Write a library that takes a random type, learns about it fields/properties via reflection, then generates and compiles "reader" class that can take data source and an instance of that random type and writes from data source into the object's fields/properties.
here is a simplified extract from my ReflectionHelper class
public class ReflectionHelper
{
public abstract class FieldReader<T>
{
public abstract void Fill(T entity, XDataReader reader);
}
public static FieldReader<T> GetFieldReader<T>()
{
Type t = typeof(T);
string className = GetCSharpName(t);
string readerClassName = Regex.Replace(className, #"\W+", "_") + "_FieldReader";
string source = GetFieldReaderCode(t.Namespace, className, readerClassName, fields);
CompilerParameters prms = new CompilerParameters();
prms.GenerateInMemory = true;
prms.ReferencedAssemblies.Add("System.Data.dll");
prms.ReferencedAssemblies.Add(Assembly.GetExecutingAssembly().GetModules(false)[0].FullyQualifiedName);
prms.ReferencedAssemblies.Add(t.Module.FullyQualifiedName);
CompilerResults compiled = new CSharpCodeProvider().CompileAssemblyFromSource(prms, new string[] {source});
if (compiled.Errors.Count > 0)
{
StringWriter w = new StringWriter();
w.WriteLine("Error(s) compiling {0}:", readerClassName);
foreach (CompilerError e in compiled.Errors)
w.WriteLine("{0}: {1}", e.Line, e.ErrorText);
w.WriteLine();
w.WriteLine("Generated code:");
w.WriteLine(source);
throw new Exception(w.GetStringBuilder().ToString());
}
return (FieldReader<T>)compiled.CompiledAssembly.CreateInstance(readerClassName);
}
private static string GetFieldReaderCode(string ns, string className, string readerClassName, IEnumerable<EntityField> fields)
{
StringWriter w = new StringWriter();
// write out field setters here
return #"
using System;
using System.Data;
namespace " + ns + #".Generated
{
public class " + readerClassName + #" : ReflectionHelper.FieldReader<" + className + #">
{
public void Fill(" + className + #" e, XDataReader reader)
{
" + w.GetStringBuilder().ToString() + #"
}
}
}
";
}
}
and the calling code:
class Program
{
static void Main(string[] args)
{
ReflectionHelper.GetFieldReader<Foo>();
Console.ReadKey(true);
}
private class Foo
{
public string Field1 = null;
public int? Field2 = null;
}
}
The dynamic compilation of course fails because Foo class is not visible outside of Program class. But! The .NET XML deserializer somehow works around that - and the question is: How?
After an hour of digging System.Xml.Serialization via Reflector I came to accept that I lack some kind of basic knowledge here and not really sure what am I looking for...
Also it is entirely possible that I am reinventing a wheel and/or digging in a wrong direction, in which case please do speak up!

You don’t need to create a dynamic assembly and dynamically compile code in order to deserialise an object. XmlSerializer does not do that either — it uses the Reflection API, in particular it uses the following simple concepts:
Retrieving the set of fields from any type
Reflection provides the GetFields() method for this purpose:
foreach (var field in myType.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
// ...
I’m including the BindingFlags parameter here to ensure that it will include non-public fields, because otherwise it will return only public ones by default.
Setting the value of a field in any type
Reflection provides the function SetValue() for this purpose. You call this on a FieldInfo instance (which is returned from GetFields() above) and give it the instance in which you want to change the value of that field, and the value to set it to:
field.SetValue(myObject, myValue);
This is basically equivalent to myObject.Field = myValue;, except of course that the field is identified at runtime instead of compile-time.
Putting it all together
Here is a simple example. Notice you need to extend this further to work with more complex types such as arrays, for example.
public static T Deserialize<T>(XDataReader dataReader) where T : new()
{
return (T) deserialize(typeof(T), dataReader);
}
private static object deserialize(Type t, XDataReader dataReader)
{
// Handle the basic, built-in types
if (t == typeof(string))
return dataReader.ReadString();
// etc. for int and all the basic types
// Looks like the type t is not built-in, so assume it’s a class.
// Create an instance of the class
object result = Activator.CreateInstance(t);
// Iterate through the fields and recursively deserialize each
foreach (var field in t.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
field.SetValue(result, deserialize(field.FieldType, dataReader));
return result;
}
Notice I had to make some assumptions about XDataReader, most notably that it can just read a string like that. I’m sure you’ll be able to change it so that it works with your particular reader class.
Once you’ve extended this to support all the types you need (including int? in your example class), you can deserialize an object by calling:
Foo myFoo = Deserialize<Foo>(myDataReader);
and you can do this even when Foo is a private type as it is in your example.

If I try to use sgen.exe (the standalone XML serialization assembly compiler), I get the following error message:
Warning: Ignoring 'TestApp.Program'.
- TestApp.Program is inaccessible due to its protection level. Only public types can be processed.
Warning: Ignoring 'TestApp.Program+Foo'.
- TestApp.Program+Foo is inaccessible due to its protection level. Only public types can be processed.
Assembly 'c:\...\TestApp\bin\debug\TestApp.exe' does not contain any types that can be serialized using XmlSerializer.
Calling new XmlSerializer(typeof(Foo)) in your example code results in:
System.InvalidOperationException: TestApp.Program+Foo is inaccessible due to its protection level. Only public types can be processed.
So what gave you the idea that XmlSerializer can handle this?
However, remember that at runtime, there are no such restrictions. Trusted code using reflection is free to ignore access modifiers. This is what .NET binary serialization is doing.
For example, if you generate IL code at runtime using DynamicMethod, then you can pass skipVisibility = true to avoid any checks for visibility of fields/classes.

I've been working a bit on this. I'm not sure if it will help but, anyway I think it could be the way. Recently I worked with Serialization and DeSerealization of a class I had to send over the network. As there were two different programs (the client and the server), at first I implemented the class in both sources and then used serialization. It failed as the .Net told me it had not the same ID (I'm not sure but it was some sort of assembly id).
Well, after googling a bit I found that it was because the serialized class was on different assemblies, so the solution was to put that class in a independent library and then compile both client and server with that library. I've used the same idea with your code, so I put both Foo class and FieldReader class in a independent library, let's say:
namespace FooLibrary
{
public class Foo
{
public string Field1 = null;
public int? Field2 = null;
}
public abstract class FieldReader<T>
{
public abstract void Fill(T entity, IDataReader reader);
}
}
compile it and add it to the other source (using FooLibrary;)
this is the code I've used. It's not exactly the same as yours, as I don't have the code for GetCSharpName (I used t.Name instead) and XDataReader, so I used IDataReader (just for the compiler to accept the code and compile it) and also change EntityField for object
public class ReflectionHelper
{
public static FieldReader<T> GetFieldReader<T>()
{
Type t = typeof(T);
string className = t.Name;
string readerClassName = Regex.Replace(className, #"\W+", "_") + "_FieldReader";
object[] fields = new object[10];
string source = GetFieldReaderCode(t.Namespace, className, readerClassName, fields);
CompilerParameters prms = new CompilerParameters();
prms.GenerateInMemory = true;
prms.ReferencedAssemblies.Add("System.Data.dll");
prms.ReferencedAssemblies.Add(Assembly.GetExecutingAssembly().GetModules(false)[0].FullyQualifiedName);
prms.ReferencedAssemblies.Add(t.Module.FullyQualifiedName);
prms.ReferencedAssemblies.Add("FooLibrary1.dll");
CompilerResults compiled = new CSharpCodeProvider().CompileAssemblyFromSource(prms, new string[] { source });
if (compiled.Errors.Count > 0)
{
StringWriter w = new StringWriter();
w.WriteLine("Error(s) compiling {0}:", readerClassName);
foreach (CompilerError e in compiled.Errors)
w.WriteLine("{0}: {1}", e.Line, e.ErrorText);
w.WriteLine();
w.WriteLine("Generated code:");
w.WriteLine(source);
throw new Exception(w.GetStringBuilder().ToString());
}
return (FieldReader<T>)compiled.CompiledAssembly.CreateInstance(readerClassName);
}
private static string GetFieldReaderCode(string ns, string className, string readerClassName, IEnumerable<object> fields)
{
StringWriter w = new StringWriter();
// write out field setters here
return #"
using System;
using System.Data;
namespace " + ns + ".Generated
{
public class " + readerClassName + #" : FieldReader<" + className + #">
{
public override void Fill(" + className + #" e, IDataReader reader)
" + w.GetStringBuilder().ToString() +
}
}";
}
}
by the way, I found a tiny mistake, you should use new or override with the Fill method, as it is abstract.
Well, I must admit that GetFieldReader returns null, but at least the compiler compiles it.
Hope that this will help you or at least it guides you to the good answer
regards