How to benchmark methods from different classes that require arguments? - c#

I want to use BenchmarkDotNet to run benchmarks on all three Run() methods of the following classes. But it's not clear what the syntax should be.
class Class1
{
[Benchmark]
public bool Run(ref string[] columns)
{
// ...
}
}
class Class2
{
[Benchmark]
public bool Run(ref string[] columns)
{
// ...
}
}
class Class3
{
[Benchmark]
public bool Run(ref string[] columns)
{
// ...
}
}
I tried syntax like this.
BenchmarkRunner.Run<Class1>();
But this gives me an error.
Benchmark method ReadRow has incorrect signature.
Method shouldn't have any arguments.
Questions:
To compare the performance of these three methods:
How do I satisfy the argument requirements and eliminate this error?
How do I compare the performance of the methods? Do I simply call BenchmarkRunner.Run<Class1>() for each class, sequentially?

How do I satisfy the argument requirements and eliminate this error?
Create class containing parameterless benchmark method and invoke the becnhmarked method there passing parameters. Either create the param in-place or use some precreated ones:
class MyBenchmark
{
[Benchmark]
public bool RunClass1()
{
var columns = // create string[]; maybe use field to store it
new Class1().BenchmarkedMethod(ref columns);
}
}
Also check the docs articles on:
Parameterization
Setup (and clean up if needed)
If needed approaches specified there will allow to move instance and parameter creation out of the benchmarked methods.
How do I compare the performance of the methods
Move all method inside one benchmark class.
class MyBenchmark
{
[Benchmark]
public bool RunClass1()
{
var columns = // create string[];
return new Class1().BenchmarkedMethod(ref columns);
}
[Benchmark]
public bool RunClass2()
{
var columns = // create string[];
return new Class2().BenchmarkedMethod(ref columns);
}
}

Related

Call a test method twice with same parameter of different datatype

I'm trying to reduce the duplicate code by combining two tests validation methods. Both test methods pass three parameters (actualResponse, expectedReponse, filterParams) but the problem is that even if the two methods have define the same name for parameters their datatype are different.
Here is the summary of the script:
TestSteps.cs class where these methods are called:
public class TestSteps : BaseTest
{
// _result holds the actual results coming from context get cal
private Context _result = new();
// _SolutionExpectedResponse have expected response from json file.
private readonly Context _SolutionExpectedResponse = new();
public TestSteps()
{
string jsonstring = File.ReadAllText(#Path of response.json file);
_SolutionExpectedResponse = JsonConvert.DeserializeObject<Context>(jsonstring);
}
[When(#Call context api)]
public void WhenCallContextAPI()
{
_result = Context.GetAsync(token,resource).Result;
}
[Then(#Verify the response Values)]
public void ThenVerifyTheResponseValues()
{
ValidateDataValues(_result.Data, _SolutionExpectedResponse.Data, new string[] {"data"})
ValidateSensitiveDataValues(_result.SensitiveData, _SolutionExpectedResponse.SensitiveData, new string[] {"sensitiveData"})
}
}
BaseTest.cs class where validate methods were wrote:
Method #1:
Protected void ValidateDataValues(List<DataReadable> actualResponse, List <DataReadable> expectedResponse, string[] filterParams)
{
if (filterParams.contains("data"))
{
if(actualResponse !=null)
{
for(int i=0; i < expectedResponse.Count; i++)
{
if((actualResponse[i].Key !=null)
{
actualResponse[i].Key.Value.ToString().Should.BeEquivalentTo(expectedResponse[i].Key.Value.ToString());
}
}
return;
}
else
{
actualResponse.Should().BeNull();
}
}
}
Method #2:
Protected void ValidateSensitiveDataValues(List<SensitiveDataReadable> actualResponse, List <SensitiveDataReadable> expectedResponse, string[] filterParams)
{
if (filterParams.contains("sensitiveData"))
{
if(actualResponse !=null)
{
for(int i=0; i < expectedResponse.Count; i++)
{
if((actualResponse[i].Key !=null)
{
actualResponse[i].Key.Value.ToString().Should.BeEquivalentTo(expectedResponse[i].Key.Value.ToString());
}
}
return;
}
else
{
actualResponse.Should().BeNull();
}
}
}
Is it possible for me to reduce the duplicate code by adding method#2 parameters in method#1 and eliminating method #2?
OR
By calling method #1 from the method #2 by converting the datatype?
A typical approach would be to use generics, this lets you define a method that can take any type, or some subset of types as parameters:
public bool ValidateSensitiveDataValues<T>(IEnumerable<T> actual, IEnumerable<T> expected, IEqualityComparer<T> comparer){
return actual.SequenceEqual(expected, comparer);
}
Note that if T is a class you either need to give the method a EqualityComparer-object, or restrict the method to only work if T implements IEquatable: ValidateSensitiveDataValues<T>(IEnumerable<T> actual, IEnumerable<T> expected) where T : IEquatable<T>. I tend to prefer the former since it is more flexible. If you do not do this any reference objects will use reference equality, and that is probably not what you want.
You might also want to use a unit test library. NUnit is a poppular one that has Assert-methods like CollectionAssert.AreEqual and CollectionAssert.AreEquivalent. Where only the former care about ordering of the items.

C# Type GetMethod() where parameter order is random

I am trying to get a method of an object, and it works fine if the parameters of the method match the order of the list of parameters I provide. I am trying to avoid this, so I do not have to worry about the order of parameter types in methods across different files. Here is what I have
MethodInfo mi = stateType.GetMethod("Entrance", typesToUse.ToArray());
In my test case, typesToUse only contains two instances of unique interfaces,
IClass1 and IClass2 in that order.
If the Entrance method is : Entrance(IClass1 c1, IClass2 c2), it picks this method up. Although, if its Entrance(IClass2 c2, IClass1 c1), it will not and mi will then be null.
Is there a way around this? Perhaps a way to tell GetMethod to ignore parameter order?
Any help is appreciated and thank you.
It is not sensible to implement a method that will ignore parameter order. Parameter order is critical to determining that you have found the correct method.
Consider this simple class:
public class A
{
public void Foo(int a, string b)
{
PrintAString();
}
public void Foo(string b, int a)
{
FormatHardDrive();
}
}
If you're method ignored the parameter order...bad things might happen!
Having said all that, it is possible of course. Simply get all the methods with a given name, eliminate all those that do not contain parameters for all the types in typesToUse, and then ensure you only have one.
The following code demonstrates this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
public class Program
{
public static void Main()
{
var typesToUse = new Type[] { typeof(int), typeof(string) };
var methods = typeof(A).GetMethods().Where(m => m.Name == "Foo");
var matchingMethods = methods.Where(m => ContainsAllParameters(m, typesToUse));
Console.WriteLine(matchingMethods.Single());
}
private static bool ContainsAllParameters(MethodInfo method, Type[] typesToUse)
{
var methodTypes = method.GetParameters().Select(p => p.ParameterType).ToList();
foreach(var typeToUse in typesToUse)
{
if (methodTypes.Contains(typeToUse))
{
methodTypes.Remove(typeToUse);
}
else
{
return false;
}
}
return !methodTypes.Any();
}
}
public class A
{
public void Foo(string a, int b)
{
Console.WriteLine("Hello World");
}
}
You could create an overload of the method, that takes the parameters in different order.
The recommended solution would probably just be to make sure they always are passed in the correct order.
If you can't ensure this, you might solve the problem by creating an overload of the method, or by performing some checks as the first thing in the Entrance-method.

C# methods with variable paramterlist as parameter

Hi I'm currently trying to pass methods (with no return value) as parameters to another method (so that they can be called from within the methods).
The problem I'm currently having is, that I'm using Action in the parameterlist and thus need to exactly define which parameters this method takes.
The question thus is: Is there any way to omit this? Thus that I don't have to define which parameters exactly the method has in the parameterdeclaration?
Codeexample:
public void A(int myint)
{
Console.WriteLine(myint.ToString());
}
public void B(int myint1, int myint2)
{
Console.WriteLine((myint1 + myint2).ToString());
}
public void myQuestionMethod(Action<int> parameterMethod)
{
//....Dosomething special by creating the parameters within and calling the given methods
}
myQuestionMethod(A);
myQuestionMethod(B);
Thus Aciton parameterMethod can that be replaced by something else that allows me to give methods as parameters who have differing parameters?
Edit:
I forgot to mention that the TYPE of the parameters is also not fixated.
Thus a function C could exist with (int param1, String param2)
No. There is no way to do this with the Action delegate (that's why there are 16 overloads).
You could opt, if the variables are all of the same type and have the same meaning, to create an array of integers:
public void A(params int[] myint)
{
}
public void myQuestionMethod(Action<int[]> parameterMethod)
{
//....Dosomething special by creating the parameters within and calling the given methods
}
Depending on how big your methods are, you could go for just Action and use anonymous methods rather than explicitly defining the functions
public void myQuestionMethod(Action parameterMethod)
{
//
}
...
myQuestionMethod(() => Console.WriteLine(myInt.ToString()));
myQuestionMethod(() => Console.WriteLine((myInt1 + myInt2).ToString()));
One solution would be to use reflection. Of course don't use it unless you do not have any other choice (specifying a method using its name should be avoided if possible):
public class Foo
{
public void A(int myint)
{
Console.WriteLine(myint.ToString());
}
public void B(int myint1, int myint2)
{
Console.WriteLine((myint1 + myint2).ToString());
}
public void myQuestionMethod(string parameterMethodName, params object[] parameters)
{
var method = this.GetType().GetMethod(parameterMethodName, BindingFlags.Instance | BindingFlags.Public);
method.Invoke(this, parameters);
}
}
public class Test
{
public static void Main()
{
var foo = new Foo();
foo.myQuestionMethod("B", 1, 2);
Console.Read();
}
}

Extension method on string[] to lowercase all elements?

Consider the following method example:
public static string[] ParseOptions()
{
return Environment.GetCommandLineArgs();
}
What would I have to do to create an extension that would make ParseOptions() return all command line arguments in lower case?
I would like to be able to use the extension as follows:
var argArray = ParseOptions().MyExtToLower();
Note: I'm asking this to better understand how to create an extension for a method. I'm not actually interested in getting lower case command line arguments this way.
public static string[] MyExtToLower(this string[] source)
{
for (int i = 0; i < source.Length; i++)
{
source[i] = source[i].ToLower();
}
return source;
}
Notice the this keyword in the parameter list. That is what makes it possible to call the method like this:
var argArray = ParseOptions().MyExtToLower();
To be clear, you're not actually adding an extension to a method here. What you are doing is adding an extension to the type that the method returns.
You seem to be talking about Fluent Interfaces. Look at this example - http://blog.raffaeu.com/archive/2010/06/26/how-to-write-fluent-interface-with-c-and-lambda.aspx
Or, you can create extension methods on the type you are returning ( in your case, string[] ) to get the method chaining - http://msdn.microsoft.com/en-us/library/bb383977.aspx
For the syntax you describe you would have to extend String[] or possibly IEnumerable<String> the following way:
public static class MyExtensions {
public static String[] MyExtToLower(this String[] strings) {
return strings.Select(s => s.toLower()).ToArray();
}
public static IEnumerable<String> MyExtToLower(this IEnumerable<String> strings) {
return strings.Select(s => s.toLower());
}
}
You don't create extension of methods, you create methods which extend objects' capabilities. Those methods must be static and part of a static class. They must have one parameter marked with the this keyword to indicate which object you want to extend. In your case, you must write something like:
// the class must be static, I usually declare a class reserved for extension method.
// I mark it as partial so that I can put every method in the same file where I use it.
public static partial class Extension {
// This is the extension method; it must be static. Note the 'this' keyword before
// the first parameter: it tells the compiler extends the string[] type.
public static MyExtToLower( this string[ ] args ) {
// your code
}
}
Note that you cannot override instance method. Altough you can have a method with the same signature as an instance method that method will be never called due to the way the compiler binds to.

C# Using Activator.CreateInstance

I asked a question yesterday regarding using either reflection or Strategy Pattern for dynamically calling methods.
However, since then I have decided to change the methods into individual classes that implement a common interface. The reason being, each class, whilst bearing some similarities also perform certain methods unique to that class.
I had been using a strategy as such:
switch (method)
{
case "Pivot":
return new Pivot(originalData);
case "GroupBy":
return new GroupBy(originalData);
case "Standard deviation":
return new StandardDeviation(originalData);
case "% phospho PRAS Protein":
return new PhosphoPRASPercentage(originalData);
case "AveragePPPperTreatment":
return new AveragePPPperTreatment(originalData);
case "AvgPPPNControl":
return new AvgPPPNControl(originalData);
case "PercentageInhibition":
return new PercentageInhibition(originalData);
default:
throw new Exception("ERROR: Method " + method + " does not exist.");
}
However, as the number of potential classes grow, I will need to keep adding new ones, thus breaking the closed for modification rule.
Instead, I have used a solution as such:
var test = Activator.CreateInstance(null, "MBDDXDataViews."+ _class);
ICalculation instance = (ICalculation)test.Unwrap();
return instance;
Effectively, the _class parameter is the name of the class passed in at runtime.
Is this a common way to do this, will there be any performance issues with this?
I am fairly new to reflection, so your advice would be welcome.
When using reflection you should ask yourself a couple of questions first, because you may end up in an over-the-top complex solution that's hard to maintain:
Is there a way to solve the problem using genericity or class/interface inheritance?
Can I solve the problem using dynamic invocations (only .NET 4.0 and above)?
Is performance important, i.e. will my reflected method or instantiation call be called once, twice or a million times?
Can I combine technologies to get to a smart but workable/understandable solution?
Am I ok with losing compile time type safety?
Genericity / dynamic
From your description I assume you do not know the types at compile time, you only know they share the interface ICalculation. If this is correct, then number (1) and (2) above are likely not possible in your scenario.
Performance
This is an important question to ask. The overhead of using reflection can impede a more than 400-fold penalty: that slows down even a moderate amount of calls.
The resolution is relatively easy: instead of using Activator.CreateInstance, use a factory method (you already have that), look up the MethodInfo create a delegate, cache it and use the delegate from then on. This yields only a penalty on the first invocation, subsequent invocations have near-native performance.
Combine technologies
A lot is possible here, but I'd really need to know more of your situation to assist in this direction. Often, I end up combining dynamic with generics, with cached reflection. When using information hiding (as is normal in OOP), you may end up with a fast, stable and still well-extensible solution.
Losing compile time type safety
Of the five questions, this is perhaps the most important one to worry about. It is very important to create your own exceptions that give clear information about reflection mistakes. That means: every call to a method, constructor or property based on an input string or otherwise unchecked information must be wrapped in a try/catch. Catch only specific exceptions (as always, I mean: never catch Exception itself).
Focus on TargetException (method does not exist), TargetInvocationException (method exists, but rose an exc. when invoked), TargetParameterCountException, MethodAccessException (not the right privileges, happens a lot in ASP.NET), InvalidOperationException (happens with generic types). You don't always need to try to catch all of them, it depends on the expected input and expected target objects.
To sum it up
Get rid of your Activator.CreateInstance and use MethodInfo to find the factory-create method, and use Delegate.CreateDelegate to create and cache the delegate. Simply store it in a static Dictionary where the key is equal to the class-string in your example code. Below is a quick but not-so-dirty way of doing this safely and without losing too much type safety.
Sample code
public class TestDynamicFactory
{
// static storage
private static Dictionary<string, Func<ICalculate>> InstanceCreateCache = new Dictionary<string, Func<ICalculate>>();
// how to invoke it
static int Main()
{
// invoke it, this is lightning fast and the first-time cache will be arranged
// also, no need to give the full method anymore, just the classname, as we
// use an interface for the rest. Almost full type safety!
ICalculate instanceOfCalculator = this.CreateCachableICalculate("RandomNumber");
int result = instanceOfCalculator.ExecuteCalculation();
}
// searches for the class, initiates it (calls factory method) and returns the instance
// TODO: add a lot of error handling!
ICalculate CreateCachableICalculate(string className)
{
if(!InstanceCreateCache.ContainsKey(className))
{
// get the type (several ways exist, this is an eays one)
Type type = TypeDelegator.GetType("TestDynamicFactory." + className);
// NOTE: this can be tempting, but do NOT use the following, because you cannot
// create a delegate from a ctor and will loose many performance benefits
//ConstructorInfo constructorInfo = type.GetConstructor(Type.EmptyTypes);
// works with public instance/static methods
MethodInfo mi = type.GetMethod("Create");
// the "magic", turn it into a delegate
var createInstanceDelegate = (Func<ICalculate>) Delegate.CreateDelegate(typeof (Func<ICalculate>), mi);
// store for future reference
InstanceCreateCache.Add(className, createInstanceDelegate);
}
return InstanceCreateCache[className].Invoke();
}
}
// example of your ICalculate interface
public interface ICalculate
{
void Initialize();
int ExecuteCalculation();
}
// example of an ICalculate class
public class RandomNumber : ICalculate
{
private static Random _random;
public static RandomNumber Create()
{
var random = new RandomNumber();
random.Initialize();
return random;
}
public void Initialize()
{
_random = new Random(DateTime.Now.Millisecond);
}
public int ExecuteCalculation()
{
return _random.Next();
}
}
I suggest you give your factory implementation a method RegisterImplementation. So every new class is just a call to that method and you are not changing your factories code.
UPDATE:
What I mean is something like this:
Create an interface that defines a calculation. According to your code, you already did this. For the sake of being complete, I am going to use the following interface in the rest of my answer:
public interface ICalculation
{
void Initialize(string originalData);
void DoWork();
}
Your factory will look something like this:
public class CalculationFactory
{
private readonly Dictionary<string, Func<string, ICalculation>> _calculations =
new Dictionary<string, Func<string, ICalculation>>();
public void RegisterCalculation<T>(string method)
where T : ICalculation, new()
{
_calculations.Add(method, originalData =>
{
var calculation = new T();
calculation.Initialize(originalData);
return calculation;
});
}
public ICalculation CreateInstance(string method, string originalData)
{
return _calculations[method](originalData);
}
}
This simple factory class is lacking error checking for the reason of simplicity.
UPDATE 2:
You would initialize it like this somewhere in your applications initialization routine:
CalculationFactory _factory = new CalculationFactory();
public void RegisterCalculations()
{
_factory.RegisterCalculation<Pivot>("Pivot");
_factory.RegisterCalculation<GroupBy>("GroupBy");
_factory.RegisterCalculation<StandardDeviation>("Standard deviation");
_factory.RegisterCalculation<PhosphoPRASPercentage>("% phospho PRAS Protein");
_factory.RegisterCalculation<AveragePPPperTreatment>("AveragePPPperTreatment");
_factory.RegisterCalculation<AvgPPPNControl>("AvgPPPNControl");
_factory.RegisterCalculation<PercentageInhibition>("PercentageInhibition");
}
Just as an example how to add initialization in the constructor:
Something similar to: Activator.CreateInstance(Type.GetType("ConsoleApplication1.Operation1"), initializationData);
but written with Linq Expression, part of code is taken here:
public class Operation1
{
public Operation1(object data)
{
}
}
public class Operation2
{
public Operation2(object data)
{
}
}
public class ActivatorsStorage
{
public delegate object ObjectActivator(params object[] args);
private readonly Dictionary<string, ObjectActivator> activators = new Dictionary<string,ObjectActivator>();
private ObjectActivator CreateActivator(ConstructorInfo ctor)
{
Type type = ctor.DeclaringType;
ParameterInfo[] paramsInfo = ctor.GetParameters();
ParameterExpression param = Expression.Parameter(typeof(object[]), "args");
Expression[] argsExp = new Expression[paramsInfo.Length];
for (int i = 0; i < paramsInfo.Length; i++)
{
Expression index = Expression.Constant(i);
Type paramType = paramsInfo[i].ParameterType;
Expression paramAccessorExp = Expression.ArrayIndex(param, index);
Expression paramCastExp = Expression.Convert(paramAccessorExp, paramType);
argsExp[i] = paramCastExp;
}
NewExpression newExp = Expression.New(ctor, argsExp);
LambdaExpression lambda = Expression.Lambda(typeof(ObjectActivator), newExp, param);
return (ObjectActivator)lambda.Compile();
}
private ObjectActivator CreateActivator(string className)
{
Type type = Type.GetType(className);
if (type == null)
throw new ArgumentException("Incorrect class name", "className");
// Get contructor with one parameter
ConstructorInfo ctor = type.GetConstructors()
.SingleOrDefault(w => w.GetParameters().Length == 1
&& w.GetParameters()[0].ParameterType == typeof(object));
if (ctor == null)
throw new Exception("There is no any constructor with 1 object parameter.");
return CreateActivator(ctor);
}
public ObjectActivator GetActivator(string className)
{
ObjectActivator activator;
if (activators.TryGetValue(className, out activator))
{
return activator;
}
activator = CreateActivator(className);
activators[className] = activator;
return activator;
}
}
The usage is following:
ActivatorsStorage ast = new ActivatorsStorage();
var a = ast.GetActivator("ConsoleApplication1.Operation1")(initializationData);
var b = ast.GetActivator("ConsoleApplication1.Operation2")(initializationData);
The same can be implemented with DynamicMethods.
Also, the classes are not required to be inherited from the same interface or base class.
Thanks, Vitaliy
One strategy that I use in cases like this is to flag my various implementations with a special attribute to indicate its key, and scan the active assemblies for types with that key:
[AttributeUsage(AttributeTargets.Class)]
public class OperationAttribute : System.Attribute
{
public OperationAttribute(string opKey)
{
_opKey = opKey;
}
private string _opKey;
public string OpKey {get {return _opKey;}}
}
[Operation("Standard deviation")]
public class StandardDeviation : IOperation
{
public void Initialize(object originalData)
{
//...
}
}
public interface IOperation
{
void Initialize(object originalData);
}
public class OperationFactory
{
static OperationFactory()
{
_opTypesByKey =
(from a in AppDomain.CurrentDomain.GetAssemblies()
from t in a.GetTypes()
let att = t.GetCustomAttributes(typeof(OperationAttribute), false).FirstOrDefault()
where att != null
select new { ((OperationAttribute)att).OpKey, t})
.ToDictionary(e => e.OpKey, e => e.t);
}
private static IDictionary<string, Type> _opTypesByKey;
public IOperation GetOperation(string opKey, object originalData)
{
var op = (IOperation)Activator.CreateInstance(_opTypesByKey[opKey]);
op.Initialize(originalData);
return op;
}
}
That way, just by creating a new class with a new key string, you can automatically "plug in" to the factory, without having to modify the factory code at all.
You'll also notice that rather than depending on each implementation to provide a specific constructor, I've created an Initialize method on the interface I expect the classes to implement. As long as they implement the interface, I'll be able to send the "originalData" to them without any reflection weirdness.
I'd also suggest using a dependency injection framework like Ninject instead of using Activator.CreateInstance. That way, your operation implementations can use constructor injection for their various dependencies.
Essentially, it sounds like you want the factory pattern. In this situation, you define a mapping of input to output types and then instantiate the type at runtime like you are doing.
Example:
You have X number of classes, and they all share a common interface of IDoSomething.
public interface IDoSomething
{
void DoSomething();
}
public class Foo : IDoSomething
{
public void DoSomething()
{
// Does Something specific to Foo
}
}
public class Bar : IDoSomething
{
public void DoSomething()
{
// Does something specific to Bar
}
}
public class MyClassFactory
{
private static Dictionary<string, Type> _mapping = new Dictionary<string, Type>();
static MyClassFactory()
{
_mapping.Add("Foo", typeof(Foo));
_mapping.Add("Bar", typeof(Bar));
}
public static void AddMapping(string query, Type concreteType)
{
// Omitting key checking code, etc. Basically, you can register new types at runtime as well.
_mapping.Add(query, concreteType);
}
public IDoSomething GetMySomething(string desiredThing)
{
if(!_mapping.ContainsKey(desiredThing))
throw new ApplicationException("No mapping is defined for: " + desiredThing);
return Activator.CreateInstance(_mapping[desiredThing]) as IDoSomething;
}
}
There's no error checking here. Are you absolutely sure that _class will resolve to a valid class? Are you controlling all the possible values or does this string somehow get populated by an end-user?
Reflection is generally most costly than avoiding it. Performance issues are proportionate to the number of objects you plan to instantiate this way.
Before you run off and use a dependency injection framework read the criticisms of it. =)

Categories