C# mutantion testing - change method runtime with il code - c#

You can skip to my approach if you don't mind what I'm actually trying to do.
What I'm trying to do
Hey I'm trying to make mutant testing,
inspired by the talk
http://www.infoq.com/presentations/kill-better-test
But I'm using c# and the best mutant libraries are made in java such as
http://pitest.org/
There are some frameworks for c# such as ninjaturtles and visualmutator,
but they both doesn't work in my computer for some reason(I get a weird error).
and also I thought it would be interesting creating my own.
About mutant testing
For those who doesn't know what is mutant testing,
it's a testing for the tests, most people use
code coverage to check that their test cover all the scenarios,
but it's not enough,
cause just because it gets to a piece of code doesn't mean it tests it.
It changes a piece of code, and if the tests still pass
it means you didn't tested the piece of code.
My approach
So I've tried starting with a simple code
that gets the il codes of a method.
var classType = typeof(MethodClass);
var methodInfo = classType.GetMethod("ExecuteMethod", BindingFlags.NonPublic | BindingFlags.Static);
byte[] ilCodes = methodInfo.GetMethodBody().GetILAsByteArray();
this is the MethodClass I'm trying to change:
public class MethodClass
{
private static int ExecuteMethod()
{
var i = 0;
i += 5;
if (i >= 5)
{
i = 2;
}
return i;
}
}
now I'm trying to replace the ils
for (int i = 0; i < ilCodes.Length; i++)
{
if (ilCodes[i] == OpCodes.Add.Value)
{
ilCodes[i] = (byte)OpCodes.Sub.Value;
}
}
but then I'm not sure how to update my function to work with the new il codes.
I've tried using
var dynamicFunction = new DynamicMethod("newmethod", typeof(int), null);
var ilGenerator = dynamicFunction.GetILGenerator();
and then the il generator has a function emit, that gets operator and value, so I could use this. but I don't have the value to put in the emit..
Does anybody know how to do it?

Related

How do I use Harmony to repeatedly replace an arbitrary method?

I need to replace a method with a call to a method with the same signature so that I can essentially replace the original method with a new method. Currently, I have the code below, which works, but when I try to patch the method again, it simply does nothing. I'm not sure if that's because Harmony doesn't like when I try to transpile it twice, or something else, either way it prevents me from repeatedly redirecting the original method.
// this is factored out of Transpiler() because yield return reasons
private static IEnumerable<CodeInstruction> TranspilerIterator(IEnumerable<CodeInstruction> instructions,
MethodBase original) {
var name = original.Name;
var par = original.GetParameters();
var method = newGuiType.GetMethod(name, (BindingFlags) FLAGS);
Console.WriteLine($"{name} == null == {method == null}");
if ((method.CallingConvention & CallingConventions.HasThis) != 0)
yield return new CodeInstruction(OpCodes.Ldarg_0);
for (var i = 0; i < par.Length; i++)
yield return new CodeInstruction(OpCodes.Ldarg_S, par[i].Position + 1);
yield return new CodeInstruction(OpCodes.Call, method);
yield return new CodeInstruction(OpCodes.Ret);
}
which is called by this:
private void DoPatches() {
Logger.Debug("Performing patches.");
var methods = oldGuiType.GetMethods((BindingFlags) FLAGS);
var t = this.GetType().GetMethod("Transpiler", BindingFlags.NonPublic | BindingFlags.Static);
for (var i = 0; i < methods.Length; i++) {
var name = methods[i].Name;
Logger.Debug($"Transpiling {name}");
harmony.Patch(methods[i], transpiler: new HarmonyMethod(t));
}
}
I can't use a prefix because I need to know the signature to get the args in a prefix, and I don't know the signature.
I know there are other libraries to make essentially this, but the game I'm modding ships with Harmony so I don't have to ship a whole lib with my very small mod.
For every change to an original method (like adding or removing a transpiler) Harmony will recalculate the replacement method by doing this (in pseudo code):
original_IL -> transpiler1 -> transpiler2 -> ... -> transpiler_n -> replacement_IL
where transpiler 1..n are all active transpilers. It then build the replacement by (roughly) structuring it like this:
// Note: the following is very simplified and only used to illustrate
// the difference between prefix/postfix and a transpiler.
REPLACEMENT()
{
if (Prefix_1() == false) return
// ...
if (Prefix_n() == false) return
// replacement_IL_here
Postfix_1()
// ...
Postfix_n()
}
Internally, Harmony has to therefore keep track of all patches and since those can be from different Assemblies it would need to serialize/deserialize state - which is in principle impossible if you have arbitrary state. Therefore, it only stores the simplest key for each patch possible: its MethodInfo which must be a static method.
As a result you cannot apply the same patch multiple times. It makes little sense since you could easily put all your code into one patch anyway.
For this application, ie updating a method to a newer method with an identical signature, Memory.DetourMethod(MethodBase original, MethodBase replacement) did what I needed.
// assuming newType is just a newer version of oldType
var newMethod = newType.getMethod("SomeMethod");
var oldMethod = oldType.getMethod("SomeMethod");
Memory.DetourMethod(oldMethod, newMethod);
This code effectively replaces the old method with the new method. The way it actually works is a bit more complicated.

How to call specific function depending on a variable?

I'm generating a random number from 1-1000. I have 200 functions named function1, function4, function 10, function 11, etc. What I would like to do is execute a specific function depending on if the number generated requires a function, and ignore it if not.
My first thought was to create an int[] containing all of the values that would trigger a function, and if the int[] contains the random number to use if statements to figure out what the number is. I'm concerned that it must be a really crude solution to an easy problem though.
I know the "best way" to do something is subjective, but is there a better way to accomplish this?
UPDATE: As per comments, I should probably have started out by pointing out that doing this for 200 functions is probably a good sign that there is some serious issue in your design. This is probably an XY question where you are trying to solve a problem in some crazy way and asking about your intended solution instead of asking about the problem itself.
That said I'll leave the original answer because it's still good advice when mapping a reasonable amount of function calls that can/will change during the life cylce of your app or dynamically as the code runs.
I won't get into why you are doing this, but I'll try to at least point you in the right direction so this doesn't become a complete nightmare when you need to modify/expand behavior:
You can map numbers to function calls using delegates and a dictionary. Assuming your functions take no arguments and return void you'd do:
var functionsMap = new Dictionary<int, Action>();
//map functions
var r = getSomeRandomNumber();
if (functions.TryGetValue(r), out var a)
a(); //invoke function
Mapping functions is simply adding keys and values:
functionsMap.Add(1, () => function1());
functionsMap.Add(3, () => function3());
//etc.
If your functions take arguments or return values, you'd use the adequate delegate: Action<T>, Func<T1, T2> etc.
You can use reflection to invoke appropriate method:
Type exampleType = exampleObject.GetType();
MethodInfo exampleMethod = exampleType.GetMethod(methodName);
exampleMethod.Invoke(this, null);
Where methodName can be created using your random number.
Without commenting on the wisdom of having 200 functions named the way yours are, you can use reflection to determine whether a given functionX() exists, like so:
public void ExecuteDynamicMethod(int number)
{
// Modify these two lines with your app's dll/exe and class type:
Assembly assembly = Assembly.LoadFile("...Assembly1.dll");
Type type = assembly.GetType("YourClassType");
if (type != null)
{
MethodInfo methodInfo = type.GetMethod("function" + number);
if (methodInfo != null)
{
object classInstance = Activator.CreateInstance(type, null);
methodInfo.Invoke(classInstance, null); // null = "no function arguments"
}
}
}
This can then be called for a given value like
ExecuteDynamicMethod(14);
See this SO answer for the inspiration behind this.
Reflection can be used for this purpose. I want to give and keep below example for not only the objective of the question but also for future reference. Also, of course that many function is not good but below code shows the approach that can work with many functions if they have similar name (like starting with "function" keyword).
Assume below is Methods.cs
using System;
using System.Reflection;
namespace YourMethodNamespace
{
public class YourMethodClass
{
public void function1()
{
Console.WriteLine("Function-1");
}
public void function2()
{
Console.WriteLine("Function-2");
}
...
public void function200()
{
Console.WriteLine("Function-200");
}
public static void invokeMethodsDynamically(int randomNumber){
Type yourClassType = typeof(YourMethodClass);
ConstructorInfo yourClassConstructorInfo = yourClassType.GetConstructor(Type.EmptyTypes);
object yourClassObject = yourClassConstructorInfo.Invoke(new object[]{});
//If the constructor has parameters, then we can pass them by this way. Like below;
/*ConstructorInfo yourClassConstructorInfo = yourClassType.GetConstructor(new[]{typeof(int)});
object yourClassObject = yourClassConstructorInfo.Invoke(new object[]{3});
*/
MethodInfo[] methodInfoArr = yourClassType.GetMethods();
foreach(MethodInfo methodInfo in methodInfoArr){
if(methodInfo.Name == "function" + randomNumber){
methodInfo.Invoke(yourClassObject, null);
}
}
}
}
}
Let's say below is Program.cs
using System;
using YourMethodNamespace;
namespace YourProgramNamespace
{
public class YourProgramClass
{
public static void Main()
{
Random random = new Random();
int randomNumber = random.Next(1, 201);
//If Methods.cs is in another Assembly
/*string pathToDllAssembly = #"Domain.dll";
Assembly dllAssembly = Assembly.LoadFrom(pathToDllAssembly);
Type methodsClassType = dllAssembly.GetType("YourMethodNamespace.YourMethodClass");
ConstructorInfo methodClassConstructorInfo = methodsClassType.GetConstructor(Type.EmptyTypes);
object methodsClassObject = methodClassConstructorInfo.Invoke(new object[]{});
MethodInfo methodInfo = methodsClassType.GetMethod("invokeMethodsDynamically");
methodInfo.Invoke(methodsClassObject, new object[]{randomNumber});
*/
YourMethodClass.invokeMethodsDynamically(randomNumber, null);
}
}
}
Also for testing and observing, below link can be used.
https://repl.it/#erdsavasci/ReflectionTest

C# function not accepting CLI array in C++

I'm currently writing a program in CLI/C++ using an imported C# package. I need to use one of the functions in one of the C# objects which takes in an array. Unfortunately it isn't allowing me to use a CLI array, defined like so:
array<float>^ knots = gcnew array<float>(nurbs->GetNumKnots());
(and then populated in a loop).
The object and the function are:
TRH::NURBSCurveKit^ nurbsKit = gcnew TRH::NURBSCurveKit();
nurbsKit->SetKnots(nurbs->GetNumKnots(), knots);
This returns an error basically saying that the cli::array type isn't compatible. Does anyone know of a way where I can cast the array, or possibly define it differently?
I'm quite new to CLI so I'm a little vague on the way it handles things at times.
Thanks
(I do something similar later on using an array of TRH::Points, but they're not defined as references or pointers, so I'm not sure if they'd work with any solutions or not).
I'm not sure if it's the same NURBSCurveKit, but according to an online API reference I found, the SetKnots method takes one parameter, not two. Since a managed array knows how long it is, you generally don't have to pass in a length with an array.
If this matches your API, just switch to pass a single parameter to SetKnots. (The reference I found uses a different namespace, so it may not be what you're using.)
array<float>^ knots = gcnew array<float>(nurbs->GetNumKnots());
TRH::NURBSCurveKit^ nurbsKit = gcnew TRH::NURBSCurveKit();
nurbsKit->SetKnots(knots);
This is my test case, seems everything is fine.
C#
namespace CS
{
public class Test
{
public int GetNum()
{
return 5;
}
public void ShowArray(int num, float[] arr)
{
for (int i = 0; i < num; i++)
{
Console.WriteLine("[{0}] = {1}",i,arr[i]);
}
}
}
}
C++/CLI
using namespace System;
using namespace CS;
int main(array<System::String ^> ^args)
{
Test^ test = gcnew Test();
array<float>^ arr = gcnew array<float>(test->GetNum());
for (int i = 0; i < test->GetNum(); i ++)
{
arr[i] = (float)i * i;
}
test->ShowArray(test->GetNum(), arr);
Console::ReadKey();
return 0;
}
Dose you code have something different form mine?

Method with just one line will hit performance?

In a code ValidateRequestmethod is defined
private bool ValidateRequest()
{
return _doc != null;
}
This method is called from everywhere I want to check if _doc is null. This method has been used 5 times in a cs file.
Performance point of view is it advisable to define a method with just a line? I think before calling this method everything from called will be pushed on stack and after it will be pulled from stack.
Any thoughts?
=== Edit ====
I am using .NET version 3.5
Don't bother with it. The compiler will probably inline the method as the corresponding IL is quite short.
If that method helps with code maintainability, as it communicates intention go on with it
It's highly unlikely that moving a single line into a method will have a significant impact on your application. It's actually quite possible that this will have no impact as the JIT could choose to inline such a function call. I would definitely opt for keeping the check in a separate method unless a profiler specifically showed it to be a problem.
Focus on writing code that is clear and well abstracted. Let the profiler guide you to the real performance problems.
As always: when you have doubts, benchmark!
And when you benchmark, do it in release mode, otherwise you're not benchmarking with compiler optimizations.
After that, if it does indeed impact performance, you can inline it with NGen.
This SO post talks about it.
ok, so this is just from LinqPad, and not I guess a definitive answer, but the following code produced a minuscule discrepancy:(00:00:00.7360736 vs 00:00:00.0740074)
void Main()
{
var starttime = DateTime.Now;
for (var i = 0; i < 1000000000; i++)
{
if (ValidateRequest()) continue;
}
var endtime = DateTime.Now;
Console.WriteLine(endtime.Subtract(starttime));
starttime = DateTime.Now;
for (var i = 0; i < 100000000; i++)
{
if (_doc != null) continue;
}
endtime = DateTime.Now;
Console.WriteLine(endtime.Subtract(starttime));
}
private object _doc = null;
private bool ValidateRequest()
{
return _doc != null;
}

Fluently setting C# properties and chaining methods

I'm using .NET 3.5. We have some complex third-party classes which are automatically generated and out of my control, but which we must work with for testing purposes. I see my team doing a lot of deeply-nested property getting/setting in our test code, and it's getting pretty cumbersome.
To remedy the problem, I'd like to make a fluent interface for setting properties on the various objects in the hierarchical tree. There are a large number of properties and classes in this third-party library, and it would be too tedious to map everything manually.
My initial thought was to just use object initializers. Red, Blue, and Green are properties, and Mix() is a method that sets a fourth property Color to the closest RGB-safe color with that mixed color. Paints must be homogenized with Stir() before they can be used.
Bucket b = new Bucket() {
Paint = new Paint() {
Red = 0.4;
Blue = 0.2;
Green = 0.1;
}
};
That works to initialize the Paint, but I need to chain Mix() and other methods to it. Next attempt:
Create<Bucket>(Create<Paint>()
.SetRed(0.4)
.SetBlue(0.2)
.SetGreen(0.1)
.Mix().Stir()
)
But that doesn't scale well, because I'd have to define a method for each property I want to set, and there are hundreds of different properties in all the classes. Also, C# doesn't have a way to dynamically define methods prior to C# 4, so I don't think I can hook into things to do this automatically in some way.
Third attempt:
Create<Bucket>(Create<Paint>().Set(p => {
p.Red = 0.4;
p.Blue = 0.2;
p.Green = 0.1;
}).Mix().Stir()
)
That doesn't look too bad, and seems like it'd be feasible. Is this an advisable approach? Is it possible to write a Set method that works this way? Or should I be pursuing an alternate strategy?
Does this work?
Bucket b = new Bucket() {
Paint = new Paint() {
Red = 0.4;
Blue = 0.2;
Green = 0.1;
}.Mix().Stir()
};
Assuming Mix() and Stir() are defined to return a Paint object.
To call methods that return void, you can use an extension method that will allow you to perform additional initialization on the object you pass in:
public static T Init<T>(this T #this, Action<T> initAction) {
if (initAction != null)
initAction(#this);
return #this;
}
Which could be used similar to Set() as described:
Bucket b = new Bucket() {
Paint = new Paint() {
Red = 0.4;
Blue = 0.2;
Green = 0.1;
}.Init(p => {
p.Mix().Stir();
})
};
I would think of it this way:
You essentially want your last method in the chain to return a Bucket. In your case, I think you want that method to be Mix(), as you can Stir() the bucket afterwards
public class BucketBuilder
{
private int _red = 0;
private int _green = 0;
private int _blue = 0;
public Bucket Mix()
{
Bucket bucket = new Bucket(_paint);
bucket.Mix();
return bucket;
}
}
So you need to set at least one colour before you call Mix(). Let's force that with some Syntax interfaces.
public interface IStillNeedsMixing : ICanAddColours
{
Bucket Mix();
}
public interface ICanAddColours
{
IStillNeedsMixing Red(int red);
IStillNeedsMixing Green(int green);
IStillNeedsMixing Blue(int blue);
}
And let's apply these to the BucketBuilder
public class BucketBuilder : IStillNeedsMixing, ICanAddColours
{
private int _red = 0;
private int _green = 0;
private int _blue = 0;
public IStillNeedsMixing Red(int red)
{
_red += red;
return this;
}
public IStillNeedsMixing Green(int green)
{
_green += green;
return this;
}
public IStillNeedsMixing Blue(int blue)
{
_blue += blue;
return this;
}
public Bucket Mix()
{
Bucket bucket = new Bucket(new Paint(_red, _green, _blue));
bucket.Mix();
return bucket;
}
}
Now you need an initial static property to kick off the chain
public static class CreateBucket
{
public static ICanAddColours UsingPaint
{
return new BucketBuilder();
}
}
And that's pretty much it, you now have a fluent interface with optional RGB parameters (as long as you enter at least one) as a bonus.
CreateBucket.UsingPaint.Red(0.4).Green(0.2).Mix().Stir();
The thing with Fluent Interfaces is that they're not that easy to put together, but they are easy for the developer to code against and they are very extensible. If you want to add a Matt/Gloss flag to this without changing all of your calling code, it's easy to do.
Also, if the provider of your API changes everything underneath you, you only have to rewrite this one piece of code; all the callin code can remain the same.
I would use the Init extension method because U can always play with the delegate.
Hell You can always declare extension methods that take up expressions and even play up with the expresions (store them for later, modify, whatever)
This way You can easily store default grups like:
Create<Paint>(() => new Paint{p.Red = 0.3, p.Blue = 0.2, p.Green = 0.1}).
Init(p => p.Mix().Stir())
This Way You can use all the actions (or funcs) and cache standard initializers as expression chains for later?
If you really want to be able to chain property settings without having to write a ton of code, one way to do this would be to use code generation (CodeDom). You can use Reflection to get a list of the mutable properties, the generate a fluent builder class with a final Build() method that returns the class you're actually trying to create.
I'm going to skip over all the boilerplate stuff about how to register the custom tool - that's fairly easy to find documentation on but still long-winded and I don't think I'd be adding much by including it. I will show you what I'm thinking of for the codegen though.
public static class PropertyBuilderGenerator
{
public static CodeTypeDeclaration GenerateBuilder(Type destType)
{
if (destType == null)
throw new ArgumentNullException("destType");
CodeTypeDeclaration builderType = new
CodeTypeDeclaration(destType.Name + "Builder");
builderType.TypeAttributes = TypeAttributes.Public;
CodeTypeReference destTypeRef = new CodeTypeReference(destType);
CodeExpression resultExpr = AddResultField(builderType, destTypeRef);
PropertyInfo[] builderProps = destType.GetProperties(
BindingFlags.Instance | BindingFlags.Public);
foreach (PropertyInfo prop in builderProps)
{
AddPropertyBuilder(builderType, resultExpr, prop);
}
AddBuildMethod(builderType, resultExpr, destTypeRef);
return builderType;
}
private static void AddBuildMethod(CodeTypeDeclaration builderType,
CodeExpression resultExpr, CodeTypeReference destTypeRef)
{
CodeMemberMethod method = new CodeMemberMethod();
method.Attributes = MemberAttributes.Public | MemberAttributes.Final;
method.Name = "Build";
method.ReturnType = destTypeRef;
method.Statements.Add(new MethodReturnStatement(resultExpr));
builderType.Members.Add(method);
}
private static void AddPropertyBuilder(CodeTypeDeclaration builderType,
CodeExpression resultExpr, PropertyInfo prop)
{
CodeMemberMethod method = new CodeMemberMethod();
method.Attributes = MemberAttributes.Public | MemberAttributes.Final;
method.Name = prop.Name;
method.ReturnType = new CodeTypeReference(builderType.Name);
method.Parameters.Add(new CodeParameterDeclarationExpression(prop.Type,
"value"));
method.Statements.Add(new CodeAssignStatement(
new CodePropertyReferenceExpression(resultExpr, prop.Name),
new CodeArgumentReferenceExpression("value")));
method.Statements.Add(new MethodReturnStatement(
new CodeThisExpression()));
builderType.Members.Add(method);
}
private static CodeFieldReferenceExpression AddResultField(
CodeTypeDeclaration builderType, CodeTypeReference destTypeRef)
{
const string fieldName = "_result";
CodeMemberField resultField = new CodeMemberField(destTypeRef, fieldName);
resultField.Attributes = MemberAttributes.Private;
builderType.Members.Add(resultField);
return new CodeFieldReferenceExpression(
new CodeThisReferenceExpression(), fieldName);
}
}
I think this should just about do it - it's obviously untested, but where you go from here is that you create a codegen (inheriting from BaseCodeGeneratorWithSite) that compiles a CodeCompileUnit populated with a list of types. That list comes from the file type you register with the tool - in this case I'd probably just make it a text file with a line-delimited list of types that you want to generate builder code for. Have the tool scan this, load the types (might have to load the assemblies first), and generate bytecode.
It's tough, but not as tough as it sounds, and when you're done you'll be able to write code like this:
Paint p = new PaintBuilder().Red(0.4).Blue(0.2).Green(0.1).Build().Mix.Stir();
Which I believe is almost exactly what you want. All you have to do to invoke the code generation is register the tool with a custom extension (let's say .buildertypes), put a file with that extension in your project, and put a list of types in it:
MyCompany.MyProject.Paint
MyCompany.MyProject.Foo
MyCompany.MyLibrary.Bar
And so on. When you save, it will automatically generate the code file you need that supports writing statements like the one above.
I've used this approach before for a highly convoluted messaging system with several hundred different message types. It was taking too long to always construct the message, set a bunch of properties, send it through the channel, receive from the channel, serialize the response, etc... using a codegen greatly simplified the work as it enabled me to generate a single messaging class that took all of the individual properties as arguments and spit back a response of the correct type. It's not something I would recommend to everyone, but when you're dealing with very large projects, sometimes you need to start inventing your own syntax!

Categories