I'm trying to use Roslyn to execute C# code that is defined by the user at runtime, similar to this example:
public class Globals
{
public int X;
public int Y;
}
var globals = new Globals { X = 1, Y = 2 };
Console.WriteLine(await CSharpScript.EvaluateAsync<int>("X+Y", globals: globals));
Example copied from here
My problem is that the variable names used in the script are unknown at compile time. In other words, I don't know what member-names I should use for my globals class and how many members (script-parameters) there will be.
I tried to use ExpandoObject to solve the problem but couldn't get it to work. Is ExpandoObject supposed to work in this context? Are there other ways to solve the problem?
Update
For my use case, the best solution is probably to use System.Linq.Dynamic:
//Expression typed in by the user at runtime
string Exp = #"A + B > C";
//The number of variables and their names are given elsewhere,
//so the parameter-array doesn't need to be hardcoded like in this example.
var e = System.Linq.Dynamic.DynamicExpression.ParseLambda(new[]
{
Expression.Parameter(typeof(double), "A"),
Expression.Parameter(typeof(double), "B"),
Expression.Parameter(typeof(double), "C")
},
null, Exp);
var Lambda = e.Compile();
//Fake some updates...
foreach (var i in Enumerable.Range(0,10))
{
Console.WriteLine(Lambda.DynamicInvoke(i, 3, 10));
}
If you can retrieve at runtime all member names, their count and their values that were passed from input you can generate execution code at runtime and evaluate it. As a simple example of execution code you can generate variable declarations for all input values and then sum all of them:
// here you should put retrieved member names and their values. Just for example, currently here exist a couple of args
var variablesAndValues = new Dictionary<string, object> { ["arg_1"] = 5, ["arg_2"] = 6, ["arg_3"] = 7 };
// and you should create an execution code looks like this:
// var arg_1 = value_1;
// ..
// var arg_n = value_n;
// arg_1 + .. + arg_n
StringBuilder builder = new StringBuilder();
foreach (var item in variablesAndValues)
{
builder.Append("var ").Append(item.Key).Append(" = ").Append(item.Value).AppendLine(";");
}
var variablesCount = variablesAndValues.Count;
foreach (var item in variablesAndValues.Keys)
{
builder.Append(item);
if (--variablesCount > 0)
{
builder.Append(" + ");
}
}
var scriptState = CSharpScript.RunAsync(builder.ToString()).Result;
var result = scriptState.ReturnValue;
Be careful, this example assumes that the all value types has sum_operation and them are known by default script options, else you will receive compile error when try to execute the code.
Upd.
If your cases are performance critically you may create a script that will sum all input arguments and then run this script repeatedly when you need it.
public class Globals
{
public int[] Args;
}
...
// create script that sum all input arguments
var script = CSharpScript.Create(#"var result = 0;
foreach (var item in Args)
{
result += item;
}
return result; ", globalsType: typeof(Globals));
// run it at twice on the user values that were received before
// also you can reuse an array, but anyway
var res1 = script.RunAsync(new Globals { Args = new[] { 5, 6, 7 } }).Result.ReturnValue;
var res2 = script.RunAsync(new Globals { Args = new[] { 7, 8, 9, 10} }).Result.ReturnValue;
This approach ignore in the script code the input variable names from an user, and seems that it doesn't matter in your cases.
Related
I need your help
I just wrote the following code
var anynomousObject = new { Amount = 10, weight = 20 };
List<object> ListOfAnynomous = new List<object> { anynomousObject };
var productQuery =
from prod in ListOfAnynomous
select new { prod.Amount, prod.weight }; // here it object on 'prod.Amount, prod.weight' that the object defenetion does not contains the "Amount" and "weight" properties
foreach (var v in productQuery)
{
Console.WriteLine(v.Amount, v.weight);
}
so please could you help me to solve this problem.
You need to make a class of your object definition, or using the dynamic keywork instead of boxing in object :
var anynomousObject = new { Amount = 10, weight = 20 };
List<dynamic> ListOfAnynomous = new List<dynamic> { anynomousObject };
var productQuery =
from prod in ListOfAnynomous
select new { prod.Amount, prod.weight };
foreach (var v in productQuery)
{
Console.WriteLine(v.Amount, v.weight);
}
this is because, when you box as object, the compiler doesn't know the definition of your anonymous var. Dynamic make it evaluate at runtime instead of compile-time.
The other option is to create a class or struct.
Your List<object> has a list of objects. The Linq query looks this list, and all it sees are regular objects.
Either use a class or a structure to store your objects, or use List<dynamic>
I have a dictionary:
<string,List<string>>
The key is the product code say "product1" then the list is a list of properties:
"Brand","10.40","64","red","S"
Then I 'can' have a list of rules/filters e.g.
var tmpFilter = new customfilters();
tmpFilter.Field = "2";
tmpFilter.Expression = ">";
tmpFilter.Filter = "10";
So for the above example this would pass because at index 2 (tmpFilter.Field) it is more than 10; then I have another object which defines which fields within the list I want to write to file. For that dictionary item I just want to write the product brand and price where the filters match.
At the moment without the filter I have:
var tmp = new custom();
tmp.Columns = "0,1";
tmp.Delimiter = ",";
tmp.Extention = ".csv";
tmp.CustomFilters = new List<customfilters>() {new customfilters(){ Field = "2", Expression = ">", Filter = "10"} };
public static void Custom(custom custom)
{
foreach (var x in Settings.Prods)
{
//Get Current Product Code
var curprod = Settings.ProductInformation[x];// the dictionary value
foreach (var column in custom.Columns)
{
var curVal = curprod[Convert.ToInt32(column)];
tsw.Write(curVal + custom.Delimiter);
}
Settings.Lines++;
tsw.WriteLine();
}
tsw.Close();
}
I only want to write the curprod if all the filters pass for that list of strings.
How I can do this?
There's a really nice Nuget package based on an example published by Microsoft, that they have decided to make really hard to find for some reason, that allows dynamic linq queries:
https://www.nuget.org/packages/System.Linq.Dynamic/1.0.2
Source:
https://github.com/kahanu/System.Linq.Dynamic
Using that you can do stuff like this very easily (note: I used strings here because the OP states they have a List<string>):
List<string> stuff = new List<string> { "10.40", "64", "5", "56", "99", "2" };
var selected = stuff.Select(s => new { d = double.Parse(s) }).Where("d > 10");
Console.WriteLine(string.Join(", ", selected.Select(s => s.d.ToString()).ToArray()));
Outputs:
10.4, 64, 56, 99
That may give you a place to start. One thing you are going to have to tackle is identifying which of your fields are numeric and should be converted to a numeric type before trying to apply your filter. Otherwise you are going to comparing as strings.
I have been trying to solve the syntax for a dynamic linq query that is needed in my application.
I have a dynamic query that the where clause needs to be specified to either
GuidPrimaryKey is contained in a list of Guid OR
GuidPrimaryKey is equal to an item in a list of Guid (using some type of for-loop)
I have a Guid[] populated with over 5,000 keys. My Query is set up as
If I do this (as a test) it is successful
data = data.where("GuidPrimaryKey.Equals(#0)",array[0]);
as well as
data = data.where("GuidPrimaryKey.Equals(#0) OR GuidPrimaryKey.Equals(#1)",array[0], array[1]);
I have tried:data = data.where("GuidPrimaryKey.Contains(#0)",array); but that gives an error: No applicable method 'Contains' exists in type 'Guid'.
I also tried setting a loop to go through the elements in the array and set the where clause as a giant string, but that did not work either.
string s = "";
string p = ""
int counter = 0;
foreach(Guid g in Array)
{
s+= "GuidPrimaryKey.Equals(#" counter.ToString() + ") OR";
p += "Array[" counter.ToString() + "],";
counter++;
}
s = s.remove(s.length - 3, 3);
p = p.remove(p.length - 1, 1);
data = data.Where(s,p);
This gives me the error message: No Property or field '1' exists in type 'DynamicClass1'
Any ideas? I need to have the where clause build the query to check to see if the primary key (GuidPrimaryKey) exists in the list of keys (Guid[]).
I'm not sure if this works in the standard Dynamic Linq library, but I just tried this is my open-source version, and it works well:
var data = data.Where("GuidPrimaryKey in #0", array);
This also works:
var data = data.Where("#0.Contains(GuidPrimaryKey)", array);
Here is a full unit test I wrote to confirm this:
[TestMethod]
public void ExpressionTests_ContainsGuid()
{
//Arrange
//Generate some users with Id fields of type Guid
var userList = User.GenerateSampleModels(5, false);
var userQry = userList.AsQueryable();
//Generate a list of values that will fail.
var failValues = new List<Guid>() {
new Guid("{22222222-7651-4045-962A-3D44DEE71398}"),
new Guid("{33333333-8F80-4497-9125-C96DEE23037D}"),
new Guid("{44444444-E32D-4DE1-8F1C-A144C2B0424D}")
};
//Add a valid Guid so that this list will succeed.
var successValues = failValues.Concat(new[] { userList[0].Id }).ToArray();
//Act
var found1 = userQry.Where("Id in #0", successValues);
var found2 = userQry.Where("#0.Contains(Id)", successValues);
var notFound1 = userQry.Where("Id in #0", failValues);
var notFound2 = userQry.Where("#0.Contains(Id)", failValues);
//Assert
Assert.AreEqual(userList[0].Id, found1.Single().Id);
Assert.AreEqual(userList[0].Id, found2.Single().Id);
Assert.IsFalse(notFound1.Any());
Assert.IsFalse(notFound2.Any());
}
I'm working with scripting rules engine based on roslyn-ctp that will process IEnumerable<T> and returns results as IEnumerable<T>. To avoid creating of ScriptEngine, configuring and parsing again and again, I'd like to reuse one instance of ScriptEngine.
Here is a short sample (full sample at gist.github.com):
var engine = new ScriptEngine();
new[]
{
typeof (Math).Assembly,
this.GetType().Assembly
}.ToList().ForEach(assembly => engine.AddReference(assembly));
new[]
{
"System", "System.Math",
typeof(Model.ProcessingModel).Namespace
} .ToList().ForEach(#namespace => engine.ImportNamespace(#namespace));
IEnumerable<Model.ProcessingModel> models = new[]
{
new Model.ProcessingModel { InputA = 10M, InputB = 5M, Factor = 0.050M },
new Model.ProcessingModel { InputA = 20M, InputB = 2M, Factor = 0.020M },
new Model.ProcessingModel { InputA = 12M, InputB = 3M, Factor = 0.075M }
};
// no dynamic allowed
// anonymous class are duplicated in assembly
var script =
#"
Result = InputA + InputB * Factor;
Delta = Math.Abs((Result ?? 0M) - InputA);
Description = ""Some description"";
var result = new { Σ = Result, Δ = Delta, λ = Description };
result
";
// Here is ArgumentException `Duplicate type name within an assembly`
IEnumerable<dynamic> results =
models.Select(model => engine.CreateSession(model).Execute(script));
And here are several issues:
roslyn-ctp does not support dynamic keyword
While using anonymous types inside script I get an exception Duplicate type name within an assembly when rsolyn-ctp is creating assembly using System.Reflection.Emit
Question
Is there way to create ScriptEngine and reuse it many times when script contains anonymous type?
You're hitting a bug in Roslyn.
I'd recommend not compiling the script in a loop. Since the code doesn't change it is much more efficient to only update the data the script is working with. Such approach also avoids the bug.
var model = new Model();
var session = engine.CreateSession(model);
var submission = session.CompileSubmission<dynamic>(script);
foreach (Model data in models)
{
model.InputA = data.InputA;
model.InputB = data.InputB;
model.Factor = data.Factor;
dynamic result = submission.Execute();
Console.WriteLine("{0} {1} {2}", result.Σ, result.Δ, result.λ);
}
I've got a work around for this issue, but I'm trying to figure out why it works . Basically, I'm looping through a list of structs using foreach. If I include a LINQ statement that references the current struct before I call a method of the struct, the method is unable to modify the members of the struct. This happens regardless of whether the LINQ statement is even called. I was able to work around this by assigning the value I was looking for to a variable and using that in the LINQ, but I would like to know what is causing this. Here's an example I created.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace WeirdnessExample
{
public struct RawData
{
private int id;
public int ID
{
get{ return id;}
set { id = value; }
}
public void AssignID(int newID)
{
id = newID;
}
}
public class ProcessedData
{
public int ID { get; set; }
}
class Program
{
static void Main(string[] args)
{
List<ProcessedData> processedRecords = new List<ProcessedData>();
processedRecords.Add(new ProcessedData()
{
ID = 1
});
List<RawData> rawRecords = new List<RawData>();
rawRecords.Add(new RawData()
{
ID = 2
});
int i = 0;
foreach (RawData rawRec in rawRecords)
{
int id = rawRec.ID;
if (i < 0 || i > 20)
{
List<ProcessedData> matchingRecs = processedRecords.FindAll(mr => mr.ID == rawRec.ID);
}
Console.Write(String.Format("With LINQ: ID Before Assignment = {0}, ", rawRec.ID)); //2
rawRec.AssignID(id + 8);
Console.WriteLine(String.Format("ID After Assignment = {0}", rawRec.ID)); //2
i++;
}
rawRecords = new List<RawData>();
rawRecords.Add(new RawData()
{
ID = 2
});
i = 0;
foreach (RawData rawRec in rawRecords)
{
int id = rawRec.ID;
if (i < 0)
{
List<ProcessedData> matchingRecs = processedRecords.FindAll(mr => mr.ID == id);
}
Console.Write(String.Format("With LINQ: ID Before Assignment = {0}, ", rawRec.ID)); //2
rawRec.AssignID(id + 8);
Console.WriteLine(String.Format("ID After Assignment = {0}", rawRec.ID)); //10
i++;
}
Console.ReadLine();
}
}
}
Okay, I've managed to reproduce this with a rather simpler test program, as shown below, and I now understand it. Admittedly understanding it doesn't make me feel any less nauseous, but hey... Explanation after code.
using System;
using System.Collections.Generic;
struct MutableStruct
{
public int Value { get; set; }
public void AssignValue(int newValue)
{
Value = newValue;
}
}
class Test
{
static void Main()
{
var list = new List<MutableStruct>()
{
new MutableStruct { Value = 10 }
};
Console.WriteLine("Without loop variable capture");
foreach (MutableStruct item in list)
{
Console.WriteLine("Before: {0}", item.Value); // 10
item.AssignValue(30);
Console.WriteLine("After: {0}", item.Value); // 30
}
// Reset...
list[0] = new MutableStruct { Value = 10 };
Console.WriteLine("With loop variable capture");
foreach (MutableStruct item in list)
{
Action capture = () => Console.WriteLine(item.Value);
Console.WriteLine("Before: {0}", item.Value); // 10
item.AssignValue(30);
Console.WriteLine("After: {0}", item.Value); // Still 10!
}
}
}
The difference between the two loops is that in the second one, the loop variable is captured by a lambda expression. The second loop is effectively turned into something like this:
// Nested class, would actually have an unspeakable name
class CaptureHelper
{
public MutableStruct item;
public void Execute()
{
Console.WriteLine(item.Value);
}
}
...
// Second loop in main method
foreach (MutableStruct item in list)
{
CaptureHelper helper = new CaptureHelper();
helper.item = item;
Action capture = helper.Execute;
MutableStruct tmp = helper.item;
Console.WriteLine("Before: {0}", tmp.Value);
tmp = helper.item;
tmp.AssignValue(30);
tmp = helper.item;
Console.WriteLine("After: {0}", tmp.Value);
}
Now of course each time we copy the variable out of helper we get a fresh copy of the struct. This should normally be fine - the iteration variable is read-only, so we'd expect it not to change. However, you have a method which changes the contents of the struct, causing the unexpected behaviour.
Note that if you tried to change the property, you'd get a compile-time error:
Test.cs(37,13): error CS1654: Cannot modify members of 'item' because it is a
'foreach iteration variable'
Lessons:
Mutable structs are evil
Structs which are mutated by methods are doubly evil
Mutating a struct via a method call on an iteration variable which has been captured is triply evil to the extent of breakage
It's not 100% clear to me whether the C# compiler is behaving as per the spec here. I suspect it is. Even if it's not, I wouldn't want to suggest the team should put any effort into fixing it. Code like this is just begging to be broken in subtle ways.
Ok. We definitely have an issues here but I suspect that this issue not with closures per se but with foreach implementation instead.
C# 4.0 specification stated (8.8.4 The foreach statement) that "the iteration variable corresponds to a read-only local variable with a scope that extends over the embedded statement". That's why we can't change loop variable or increment it's property (as Jon already stated):
struct Mutable
{
public int X {get; set;}
public void ChangeX(int x) { X = x; }
}
var mutables = new List<Mutable>{new Mutable{ X = 1 }};
foreach(var item in mutables)
{
// Illegal!
item = new Mutable();
// Illegal as well!
item.X++;
}
In this regard read-only loop variables behave almost exactly the same as any readonly field (in terms of accessing this variable outside of the constructor):
We can't change readonly field outside of the constructor
We can't change property of the read-only field of value type
We're treating readonly fields as values that leads to using a temporary copy every time we accessing readonly field of value type.
.
class MutableReadonly
{
public readonly Mutable M = new Mutable {X = 1};
}
// Somewhere in the code
var mr = new MutableReadonly();
// Illegal!
mr.M = new Mutable();
// Illegal as well!
mr.M.X++;
// Legal but lead to undesired behavior
// becaues mr.M.X remains unchanged!
mr.M.ChangeX(10);
There is a plenty of issues related to mutable value types and one of them related to the last behavior: changing readonly struct via mutator method (like ChangeX) lead to obscure behavior because we'll modify a copy but not an readonly object itself:
mr.M.ChangeX(10);
Is equivalent to:
var tmp = mr.M;
tmp.ChangeX(10);
If loop variable treated by the C# compiler as a read-only local variable, than its seems reasonable to expect the same behavior for them as for read-only fields.
Right now loop variable in the simple loop (without any closures) behaves almost the same as a read-only field except copying it for every access. But if code changes and closure comes to play, loop variable starts behaving like pure read-only variable:
var mutables = new List<Mutable> { new Mutable { X = 1 } };
foreach (var m in mutables)
{
Console.WriteLine("Before change: {0}", m.X); // X = 1
// We'll change loop variable directly without temporary variable
m.ChangeX(10);
Console.WriteLine("After change: {0}", m.X); // X = 10
}
foreach (var m in mutables)
{
// We start treating m as a pure read-only variable!
Action a = () => Console.WriteLine(m.X));
Console.WriteLine("Before change: {0}", m.X); // X = 1
// We'll change a COPY instead of a m variable!
m.ChangeX(10);
Console.WriteLine("After change: {0}", m.X); // X = 1
}
Unfortunately I can't find strict rules how read-only local variables should behave but its clear that this behavior is different based on loop body: we're not copying to locals for every access in simple loop, but we DO this if the loop body closes over loop variable.
We all know that Closing over loop variable considered harmful and that loop implementation was changed in the C# 5.0. Simple way to fix that old issue in pre C# 5.0 era was introducing local variable, but interesting that introducing local variable in this our case will change behavior as well:
foreach (var mLoop in mutables)
{
// Introducing local variable!
var m = mLoop;
// We're capturing local variable instead of loop variable
Action a = () => Console.WriteLine(m.X));
Console.WriteLine("Before change: {0}", m.X); // X = 1
// We'll roll back this behavior and will change
// value type directly in the closure without making a copy!
m.ChangeX(10); // X = 10 !!
Console.WriteLine("After change: {0}", m.X); // X = 1
}
Actually this means that C# 5.0 has very subtle breaking change because no one will introduce a local variable any more (and even tools like ReSharper stops warning about it in VS2012 because its not an issue).
I'm OK with both behaviors but inconsistency seems strange.
I suspect this has to do with how lambda expressions are evaluated. See this question and its answer for more details.
Question:
When using lambda expressions or anonymous methods in C#, we have to be wary of the access to modified closure pitfall. For example:
foreach (var s in strings)
{
query = query.Where(i => i.Prop == s); // access to modified closure
Due to the modified closure, the above code will cause all of the Where clauses on the query to be based on the final value of s.
Answer:
This is one of the worst "gotchas" in C#, and we are going to take the breaking change to fix it. In C# 5 the foreach loop variable will be logically inside the body of the loop, and therefore closures will get a fresh copy every time.
Just to accomplish Sergey's post, I wanna to add following example with manual closure, that demonstrates compiler's behavior. Of course compiler might have any other implementation that satisfies readonly requirement of captured within foreach statement variable.
static void Main()
{
var list = new List<MutableStruct>()
{
new MutableStruct { Value = 10 }
};
foreach (MutableStruct item in list)
{
var c = new Closure(item);
Console.WriteLine(c.Item.Value);
Console.WriteLine("Before: {0}", c.Item.Value); // 10
c.Item.AssignValue(30);
Console.WriteLine("After: {0}", c.Item.Value); // Still 10!
}
}
class Closure
{
public Closure(MutableStruct item){
Item = item;
}
//readonly modifier is mandatory
public readonly MutableStruct Item;
public void Foo()
{
Console.WriteLine(Item.Value);
}
}
This might solve your issue. It swaps out foreach for a for and makes the struct immutable.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace WeirdnessExample
{
public struct RawData
{
private readonly int id;
public int ID
{
get{ return id;}
}
public RawData(int newID)
{
id = newID;
}
}
public class ProcessedData
{
private readonly int id;
public int ID
{
get{ return id;}
}
public ProcessedData(int newID)
{
id = newID;
}
}
class Program
{
static void Main(string[] args)
{
List<ProcessedData> processedRecords = new List<ProcessedData>();
processedRecords.Add(new ProcessedData(1));
List<RawData> rawRecords = new List<RawData>();
rawRecords.Add(new RawData(2));
for (int i = 0; i < rawRecords.Count; i++)
{
RawData rawRec = rawRecords[i];
int id = rawRec.ID;
if (i < 0 || i > 20)
{
RawData rawRec2 = rawRec;
List<ProcessedData> matchingRecs = processedRecords.FindAll(mr => mr.ID == rawRec2.ID);
}
Console.Write(String.Format("With LINQ: ID Before Assignment = {0}, ", rawRec.ID)); //2
rawRec = new RawData(rawRec.ID + 8);
Console.WriteLine(String.Format("ID After Assignment = {0}", rawRec.ID)); //2
i++;
}
rawRecords = new List<RawData>();
rawRecords.Add(new RawData(2));
for (int i = 0; i < rawRecords.Count; i++)
{
RawData rawRec = rawRecords[i];
int id = rawRec.ID;
if (i < 0)
{
List<ProcessedData> matchingRecs = processedRecords.FindAll(mr => mr.ID == id);
}
Console.Write(String.Format("With LINQ: ID Before Assignment = {0}, ", rawRec.ID)); //2
rawRec = new RawData(rawRec.ID + 8);
Console.WriteLine(String.Format("ID After Assignment = {0}", rawRec.ID)); //10
i++;
}
Console.ReadLine();
}
}
}