.net Framework 4.7.2...
Compiled expressions can access private fields. When I use exactly the same expression and write it to a dynamic assembly, using CompileToMethod, I get a System.FieldAccessException while trying to read the private field.
Is there anything I can do to allow the dynamic assembly to have the same access privileges that a compiled expression does? There is ancient lore that says you cannot. But I can't find anything resembling a primary source for that claim. I can't believe there is not some form of assembly attributes or permissioning that would allow access.
Can I do it if I save the assembly instead? (Writing cached marshalling assemblies to disk is a likely feature in the future).
The application is marshalling of structs to streams in a domain-specific computer music language. Serialization is not an option (another example of dynamic code in dynamic assemblies that violates access).
Example code:
The lambda expression successfully reads the value of the private field of ComplexStruct (given below). If the same expression is emitted to a dynamic assembly using CompileToMethod, it fails with an access exception.
ComplexStruct s = new ComplexStruct();
s.String1 = "abc";
// Pick a private field (one of the backing fields for a property)
FieldInfo fieldInfo = typeof(ComplexStruct).GetFields(BindingFlags.NonPublic | BindingFlags.Instance)[0];
var structArgument = Expression.Parameter(typeof(ComplexStruct));
var lambda = Expression.Lambda<Func<ComplexStruct,String>>(
Expression.Field(structArgument, fieldInfo), // return the value of the private field.
structArgument);
Func<ComplexStruct,String> fn = lambda.Compile();
String result = fn(s);
Assert.AreEqual(structArgument.String1, result);
A struct with private fields:
// (Complex compared to simple struct where all fields
// are public or the struct is unmanaged in case you were wondering)
public struct ComplexStruct : IEquatable<ComplexStruct>
{
public String String1 { get; set; } // the backing field for this property gets read.
public String String2 { get; set; }
public String String3 { get; }
public ComplexStruct(String v1, String v2)
{
String1 = v1;
String2 = v2;
}
public bool Equals(ComplexStruct other)
{
return String1 == other.String1 && String2 == other.String2;
}
}
Creation of the assembly:
AppDomain myAppDomain = Thread.GetDomain();
AssemblyName myAsmName = new AssemblyName();
myAsmName.Name = "DynamicAssembly";
this.saveAssembly = ServiceBase.DEBUG;
assemblyBuilder = myAppDomain.DefineDynamicAssembly(
myAsmName,
saveAssembly? AssemblyBuilderAccess.RunAndSave: AssemblyBuilderAccess.RunAndCollect);
After reviewing .net sources it seems pretty certain that there is no way for an Assembly to bypass field access checks.
thehennyy pointed out the backdoor that Linq Expressions use. DynamicMethod constructors provides a skipVisibility parameter which allows generation of IL that can access non-public fields and methods. But there's no way to integrate DynamicMethods with Dynamic Assemblies, or emit them to a saved assembly.
Given the limitations of DynamicMethods, there doesn't seem to be any reason to prefer them over Linq Expressions, given that Linq Expression APIs are infinitely easier to use that IL.Emit APIs.
In the end, I used call-outs to templated classes that generate struct serialization delegates produced by Linq Expressions in static constructors.
If you're following the same path, you may want to look at "unmanaged" structs, introduced in C# 7.2, which allow optimized serialization of structs composed entirely of ValueType members. Given that Strings are reference classes, that usually has limited value. But given that I'm trying to write allocation-free serializers, they were useful for my purposes.
See my answer to this question on more details on the undocumented IgnoreAccessCheckToAttribute which is doing what you probably want.
Here is sample code:
One legitimate use is when compiling dynamically assemblies that needs access to internal classes, as in the following code:
var assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(new AssemblyName(name),
AssemblyBuilderAccess.RunAndCollect);
var assmemblyNames = typesToAccess.Where(t => t is not null)
.Select(t => t!.Assembly.FullName.Split(',').First()) // Extract the name from fullName
.Distinct().ToList();
var ignoreAccessCtor = typeof(IgnoresAccessChecksToAttribute)
.GetConstructor(new Type[] { typeof(string) });
foreach (var refName in assemblyNames)
{
var builder = new CustomAttributeBuilder(ignoreAccessCtor, new object[] { refName });
assemblyBuilder.SetCustomAttribute(builder);
}
Related
I'm trying to implement static reflection for all fields in the class.
In other words, I have to create get and set using name for all these fields.
I use the answers on Set field value with Expression tree, and came to the following solution
public class ExpressionSetterGetter
{
public class SetterGetter<T> where T : class
{
public Delegate getter;
public Action<T, object> setter;
}
public static Dictionary<string, SetterGetter<T>> GetFieldSetterGetterExpressions<T>() where T : class
{
var dic = new Dictionary<string, SetterGetter<T>>();
SetterGetter<T> setterGetter;
var type = typeof(T);
var fields = type.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
foreach (var fieldInfo in fields)
{
var targetExp = Expression.Parameter(type, "target");
var valueExp = Expression.Parameter(typeof(object), "value");
var fieldExp = Expression.Field(targetExp, fieldInfo);
var assignExp = Expression.Assign(fieldExp, Expression.Convert(valueExp, fieldExp.Type));
var fieldSetter = Expression.Lambda<Action<T, object>>(assignExp, targetExp, valueExp).Compile();
ParameterExpression objParm = Expression.Parameter(type, "obj");
MemberExpression fieldExpr = Expression.Field(objParm, fieldInfo.Name);
var fieldExprConverted = Expression.Convert(fieldExpr, typeof(object));
var fieldGetter = Expression.Lambda(fieldExprConverted, objParm).Compile();
setterGetter = new SetterGetter<T>() { setter = fieldSetter, getter = fieldGetter };
dic.Add(fieldInfo.Name, setterGetter);
}
return dic;
}
}
Now, I have these two issues.
I use type.GetFields(), but MSDN says it is a reflection method.
What means that the compiler doesn't know the type before runtime.
Am I right? If it is correct, what is the reason to use the
expression trees. As far as I know expression trees translate to the
code at compile-time time, what means almost no additional costs.
The same logic. What if I put as a parameter a list with name of
fields what have to be wrapped. In other words, instead of
type.GetFields() I simply put the names of fields as parameters.
public static Dictionary<string, SetterGetter<T>> GetFieldSetterGetterExpressions<T>(IEnumerable<string> fieldNames)
Obviously, the list can be not known at compile-time. Again, how the CLR will react?
Expression trees' output is not known in compile time, they are compiled at runtime, which is something different. Advantage of using (cached) expression trees compilation is that reflection heavy lifting is performed only once, during the expression construction and compilation. From there, invocation of the resulting delegate is very fast.
In case you are sure the list specifies only the fields that actually exist in your class and you provide only a subset of all the fields, you could, in theory, somewhat reduce the performance impact of the reflection. But you should keep in mind that you will still need the FieldInfo instances for each of them which could only be retrieved using reflection. So the overall price of invoking GetField for every known field will probably be even higher than just calling GetFields for all of them.
The easiest way to set fields value in compile time is converting your private fields to public properties. Then, you can set them as you wish.
And this is not a joke on you, it's an explanation of the difference - C# doesn't allow setting fields from outside, except using reflection API's. And those API's designed for runtime, not for compile time.
If you'd like to bend the rules and produce such a code at compilation time (precisely, at post-compilation time), you should use some third party library such as PostSharp or Fody.
And that's the end of the story, so long as you are using C# in it's current or one of the previous versions.
What is the best general purpose way to get a System.Type from Microsoft.CodeAnalysis.ISymbol for different types of symbols ? (e.g. class declarations, variable, properties, etc)
I want to be able to do various checks on the type e.g. as checking if the type implements any interface or is cast-able to any interface, just like one can check on System.Type.
The problem I am having is that most of the concrete classes used to represent the symbol are internal (see http://sourceroslyn.io/) and I could not find tye type information in the ISymbol.
SourceNamedTypeSymbol
LocalSymbol
I retrieve the ISymbol using the following code
var objectSymbol = (ISymbol)model.GetDeclaredSymbol(obj.Node);
Short answer: you can't. There is no proper way to get a System.Type (reflection) from an ISymbol (Roslyn).
One option to do go in the direction you want is constructing the fully-qualified name of your type and then looking that up through reflection (example).
You should probably ask yourself whether this is something you need to do in the first place though -- reflection and Roslyn aren't really intended to work together.
What you are interested in, however, can be done through Roslyn as well. The key here is using the semantic model which has all this information for you.
All declarations (opposed to usages) have a specific overload available that allows you to get the declaring symbol and return it in the appropriate type (such as INamedTypeSymbol in this case).
Take the following example:
const string source = #"
using System;
namespace MyNamespace
{
class MyClass : IDisposable
{
void Method()
{
MyClass nameOfVariable, another;
}
}
}
";
var tree = CSharpSyntaxTree.ParseText(source);
var compilation = CSharpCompilation.Create("MyCompilation", new[] { tree }, new[] { MetadataReference.CreateFromFile(typeof(object).Assembly.Location) });
var semanticModel = compilation.GetSemanticModel(tree);
var root = tree.GetRoot();
var classSymbol = semanticModel.GetDeclaredSymbol(root.DescendantNodes().OfType<ClassDeclarationSyntax>().First());
Console.WriteLine(string.Join(", ", classSymbol.AllInterfaces));
This will display all the interfaces the class implements. Keep in mind however that this just refers to the current definition -- if you're also interested in base types you'll have to go through the hierarchy yourself.
In your scenario you should be able to just cast it to the right type (assuming you are checking a declaration node):
var objectSymbol = (INamedTypeSymbol) model.GetDeclaredSymbol(obj.Node);
I think this is what you are looking for:
var castedProperty = (IPropertySymbol) property;
var type = castedProperty.Type.Name;
The variable 'property' is an ISymbol instance.
I'm looking for a way to avoid FieldInfo.Get/SetValue overhead, and access memory directly for a few select, known ahead of time, primitive types. (Most specifically, I'm looking to avoid any memory allocations in our custom serializer)
Basically, here's what the official way allows me to do:
System.Object o = someobject;
int inOut = 0;
var type = o.GetType();
var fieldInfos = type.GetFields(BindingFlags.Public | BindingFlags.Instance);
foreach (var fi in fieldInfos) {
fi.SetValue(o, inOut);
inOut = (int)fi.GetValue(o);
}
And here's roughly what I'd like to do:
foreach (var fi in fieldInfos) {
fixed(int* ip = o.basePointer + fi.fieldOffset) {
*p = inOut;
inOut = *p;
}
}
I would use this only for Int32, Single, and possibly bools. I'm primarily interested in getting this working on Mono, so if there's anything Mono specific available, that'd be fine.
Note: I'm well aware of the "you shouldn't be doing this", and "have you profiled it" etc. I know, and I have, which is why I'm looking into this. We have a very specific case, where we control all variables (and all code), but we would like it to work on any 'normal' class without requiring additional markup or explicit struct layout.
EDIT: I should point out that I'm not able to emit dynamic code to solve this. I'm ok with a solution requiring me to write and assemble IL up-front though.
I'm well aware of the "you shouldn't be doing this"
That is good - I'll skip this part of the explanation then, and go straight to a way of accessing fields that avoids memory allocation, while staying within the limits of managed code.
You can use LINQ expressions to construct a Func<ObjType,int> for a getter and Action<ObjType,int> for a setter. Calling these functors would let you get or set int fields as if you were accessing their methods directly.
Here is how you can make a wrapper-free getter:
public class Test
{
public int myfield;
public static void Main()
{
// Make a parameter expression to represent the object
var argExpr = Expression.Parameter(typeof(Test), "a");
// Get the field of your object (the same way as in your first example)
var field = typeof(Test).GetField("myfield", BindingFlags.Public | BindingFlags.Instance);
// Make an expression accessing the field from the parameter
var fieldExpr = Expression.Field(argExpr, field);
// Compile the expression into a functor
var getter = (Func<Test,int>)Expression.Lambda(fieldExpr, argExpr).Compile();
// Construct a test object
var tmp = new Test {myfield = 123};
// Use a wrapper to avoid "boxing"/"unboxing" of "GetValue"
int res = getter(tmp);
Console.WriteLine("Res={0}", res);
}
}
Demo on ideone.
Construct the setter in a similar way, using one more parameter of type int, and Expression.Assign. The resultant lambda will compile into an Action<Test,int> rather than Func<Test,int>, because setters do not return value.
You say, that you can't use dynamic code generation. Here are some other ideas:
If you can work with properties instead of fields, create a delegate to the property getter (https://stackoverflow.com/a/724427).
Generate IL code for your serializer at build time. Compile that into an assembly that you can load at runtime. Just generate accessor code for each and every field. I think you can access private members in IL when FullTrust and SkipVerification permissions are present.
If I have this string list:
string myObjectString = "MyObject, SetWidth, int, 10, 0, 1";
in which:
- MyObject: the object class
- SetWidth: the property of the object
- int: type of the SetWidth is int
- 10: default value
- 0: object order
- 1: property order
Then how can I construct an object like this:
[ObjectOrder(0)]
public class MyObject:
{
private int _SetWidth = 10;
[PropertyOrder(1)]
public int SetWidth
{
set{_SetWidth=value;}
get{return _SetWidth;}
}
}
So, I would like have something like this:
Object myObject = ConstructAnObject(myObjectString);
and the myObject is an instance of MyObject. Could it be possible in C#?
Thanks in advance.
I think you better use the Object Serialization/Deserialization instead of creating a custom method that basically needs to do the same thing
more info at:
http://msdn.microsoft.com/en-us/library/ms233843.aspx
Here is some quick and dirty code to get you started:
string myObjectString = "MyObject, SetWidth, int, 10, 0, 1";
var info = myObjectString.Split(',');
string objectName = info[0].Trim();
string propertyName = info[1].Trim();
string defaultValue = info[3].Trim();
//find the type
Type objectType = Assembly.GetExecutingAssembly().GetTypes().Where(t=>t.Name.EndsWith(objectName)).Single();//might want to redirect to proper assembly
//create an instance
object theObject = Activator.CreateInstance(objectType);
//set the property
PropertyInfo pi = objectType.GetProperty(propertyName);
object valueToBeSet = Convert.ChangeType(defaultValue, pi.PropertyType);
pi.SetValue(theObject, valueToBeSet, null);
return theObject;
This will find the MyObject, create an object of the proper propertytype, and set the matching property.
If you use C# 4.0, you can use the new dynamic feature.
string myObjectString = "MyObject, SetWidth, int, 10, 0, 1";
String[] properties = myObjectString.Split(',');
dynamic myObj;
myObj.MyObject = (objtect)properties[0];
myObj.SetWidth = Int32.Parse(properties[1]);
// cast dynamic to your object. Exception may be thrown.
MyObject result = (MyObject)myObj;
I don't quite understand why do you need ObjectOrder and PropertyOrder... Once you have their names you probably don't need them, at least for "deserialization"...
Or please advice what is their role?
You definitely can simply do it via reflection:
Split the string by comma (using myString.Split)
Use reflection to find an object within your application:
Find the type with name = splittedString[0] (enumerate all the assemblies within the domain and all the types within each assembly);
Instantiate the type found (using Activator.CreateInstance)
Find the property by name (Using objectType.GetProperty)
Set the property value (using propertyInfo.SetValue)
Return the object
Assuming you need to generate new types there are two possible ways to do so:
Using Reflection Emit
Using CodeDom provider
I think the simpler solution is CodeDom provider. All needed is to generate the source as a string in memory, and then compile the code and instantiate a new instance with Activator. This is a nice example I just found.
The reason I think that CodeDom provider is simpler is that it has shorter setup - no need to generate dynamic module and assembly and then work with type builder and members builder. In addition, it doesn't require working with IL to generate the getter and setter bodies.
An advantage that reflection emit has is performance - dynamic module can add more types to itself even after one of the types was used. CodeDom provider requires creating all the types at once, otherwise it creates a new assembly each time.
There are many algorithms to evaluate expressions, for example:
By Recursive Descent
Shunting-yard algorithm
Reverse Polish notation
Is there any way to evaluate any mathematical expression using C# .net reflection or other modern .net technology?
Further to Thomas's answer, it's actually possible to access the (deprecated) JScript libraries directly from C#, which means you can use the equivalent of JScript's eval function.
using Microsoft.JScript; // needs a reference to Microsoft.JScript.dll
using Microsoft.JScript.Vsa; // needs a reference to Microsoft.Vsa.dll
// ...
string expr = "7 + (5 * 4)";
Console.WriteLine(JScriptEval(expr)); // displays 27
// ...
public static double JScriptEval(string expr)
{
// error checking etc removed for brevity
return double.Parse(Eval.JScriptEvaluate(expr, _engine).ToString());
}
private static readonly VsaEngine _engine = VsaEngine.CreateEngine();
It's certainly possible. The CodeSnippetCompileUnit class does basically this.
I wrote you some example usage code. You'll need to include these namespaces:
System.CodeDom.Compiler;
System.CodeDom;
Microsoft.CSharp;
System.Reflection;
Here's the code:
string source = #"
class MyType
{
public static int Evaluate(<!parameters!>)
{
return <!expression!>;
}
}
";
string parameters = "int a, int b, int c";
string expression = "a + b * c";
string finalSource = source.Replace("<!parameters!>", parameters).Replace("<!expression!>", expression);
CodeSnippetCompileUnit compileUnit = new CodeSnippetCompileUnit(finalSource);
CodeDomProvider provider = new CSharpCodeProvider();
CompilerParameters parameters = new CompilerParameters();
CompilerResults results = provider.CompileAssemblyFromDom(parameters, compileUnit);
Type type = results.CompiledAssembly.GetType("MyType");
MethodInfo method = type.GetMethod("Evaluate");
// The first parameter is the instance to invoke the method on. Because our Evaluate method is static, we pass null.
int result = (int)method.Invoke(null, new object[] { 4, -3, 2 });
Replace 'parameters' and 'expression' by whatever, and you've got yourself a general expression evaluator.
If you get a FileNotFoundException in results.CompiledAssembly, then the snippet failed to compile.
You might also want to take a look at the System.CodeDom.CodeSnippetExpression class. It's used for more specifically reading expressions, but an expression by itself can't be compiled, so you would need to use more CodeDom to build a working class and method around it. This is useful if you want to be able to programmatically manipulate what kind of class you're generating. CodeSnippetCompileUnit is nice to generate an entire working class at once (and simpler for an example) but to manipulate it you would have to do inconvenient string manipulations.
Although using compiler services is a simple and efficient solution, it raises serious security issues if the expression is entered by a user, because it could execute virtually anything.
There's another very simple solution that is much more secure : take advantage of the JScript Eval function. You just need to follow these steps :
Create a js file named JsMath.js :
class JsMath
{
static function Eval(expression : String) : double
{
return eval(expression);
};
}
Compile it into a class library :
jsc /t:library JsMath.js
Reference the JsMath library in your C# project, and use it like that :
double result = JsMath.Eval(expression);
For me Vici.Parser works extremely well: check it out here , it's the most flexible expression parser I've found so far.
(we've used it to set up 'human-readable' business rules, with data provided by an SQL server database)
Examples are available and there's a very good support by the developer (check the website's forum).
ncalc is the best. you can find it in codeplex also in nugget.
NCalc is a mathematical expressions evaluator in .NET. NCalc can parse any expression and evaluate the result, including static or dynamic parameters and custom functions.
I think this is the best way of all. Petar Repac's answer is amazing.
Using the 'expression' argument of the DataColumn object solves incredibly and easily the topic:
static double Evaluate(string expression)
{
var loDataTable = new DataTable();
var loDataColumn = new DataColumn("Eval", typeof(double), expression);
loDataTable.Columns.Add(loDataColumn);
loDataTable.Rows.Add(0);
return (double)(loDataTable.Rows[0]["Eval"]);
}
You can use Math-Expression-Evaluator library which implements Shunting Yard algorithm that I am author of. It supports simple expressions such as 2.5+5.9, 17.89-2.47+7.16, 5/2/2+1.5*3+4.58, expressions with parentheses (((9-6/2)*2-4)/2-6-1)/(2+24/(2+4)) and expressions with variables:
var a = 6;
var b = 4.32m;
var c = 24.15m;
var engine = new ExpressionEvaluator();
engine.Evaluate("(((9-a/2)*2-b)/2-a-1)/(2+c/(2+4))", new { a, b, c});
You can also pass parameters as named variables:
dynamic dynamicEngine = new ExpressionEvaluator();
var a = 6;
var b = 4.5m;
var c = 2.6m;
dynamicEngine.Evaluate("(c+b)*a", a: 6, b: 4.5, c: 2.6);
It supports .Net Standard 2.0 so can be used from .Net Core as well as .Net Full Framework projects and it doesn't have any external dependencies.
To dynamically compile code using the new Roslyn API's, and load the assembly in a .net core project;
string finalSource = ...;
IEnumerable<Assembly> references = ...;
var compilation = CSharpCompilation.Create("Dynamic",
new[] {
SyntaxFactory.ParseSyntaxTree(
finalSource,
CSharpParseOptions.Default
.WithLanguageVersion(LanguageVersion.Latest)
) },
references.Select(a => MetadataReference.CreateFromFile(a.Location)),
new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)
.WithAssemblyIdentityComparer(DesktopAssemblyIdentityComparer.Default)
);
using var ms = new MemoryStream();
var e = compilation.Emit(ms);
if (!e.Success)
throw new Exception("Compilation failed");
ms.Seek(0, SeekOrigin.Begin);
var context = new AssemblyLoadContext(null, true);
var assembly = context.LoadFromStream(ms);
Note that along with any other types required by the source you are compiling. In order to load the compiled assembly within the same process, references will need to include;
AppDomain.CurrentDomain.GetAssemblies().Where(a => a.GetName().Name == "netstandard").Single(),
typeof(object).Assembly