We Keep Coding

Can I write tests for custom attributes without defining x^n classes?

I'm building a class library that includes several custom attributes that apply to properties. Then I have methods that do certain things based on the placement of the attributes.
Now I want to build some unit tests, but how to I make the tests without creating something on the order of x^(number of attributes) classes just for testing purposes? Can I leverage metadata classes or something?
Basically I'd love for there to be a way for me to apply attributes to properties at runtime (i.e. inside the "Arrange" part of my test method), but I'm pretty sure that's impossible.
Edit
This is the reflection code I'm using to test attributes, since apparently how I'm reading them may make a difference:
bool skip = false, iip = false;
string lt = null;
SerializeAsOptions sa = SerializeAsOptions.Ids;
object[] attrs = prop.GetCustomAttributes(true);
foreach (object attr in attrs)
{
Type attrType = attr.GetType();
if (typeof(JsonIgnoreAttribute).IsAssignableFrom(attrType))
{
skip = true;
continue;
}
if (typeof(IncludeInPayload).IsAssignableFrom(attrType))
iip = ((IncludeInPayload)attr).Include;
if (typeof(SerializeAs).IsAssignableFrom(attrType))
sa = ((SerializeAs)attr).How;
if (typeof(LinkTemplate).IsAssignableFrom(attrType))
lt = ((LinkTemplate)attr).LinkTemplate;
}
if (skip) continue;

I'm adding another answer, because since you now provided some code, the old one is too broad. It's now (mostly) obvious that:
you control the attribute-reading code
you are reading the code via reflection (PropertyInfo.GetCustomAttributes)
So. Since you are using Reflection, TypeDescriptors will not help. You'd need to:
either read the attrs differently so TypeDescr can be used
dynamically generate assemblies at runtime to generate classes with properties on the fly during tests
It can be very interesting/entertaining, but it can also turn into nice amount of work. But, since you control both sides of the code, none of these two is actually needed.
First, let's trim the code to significant parts:
somemethod(PropertyInfo prop)
{
// ...
object[] attrs = prop.GetCustomAttributes(true); // read ATTRs from PROP
foreach (object attr in attrs) // scan the PROP's ATTRs
{
// check attr type, do something
}
// ...
}
The core of your problem is not:
adding/removing attributes during Arrange/Teardown part
but
forcing the loop over PROP's ATTRs to see attributes that your test specifies
Looking at the problem like this, the answer is almost obvious: your loop has to abstract from the "Read attributes" part.
object[] attributeReader(PropertyInfo prop)
{
return prop.GetCustomAttributes(true);
}
somemethod(PropertyInfo prop)
{
// ...
object[] attrs = attributeReader(prop); // read ATTRs from PROP
foreach (object attr in attrs) // scan the PROP's ATTRs
{
// check attr type, do something
}
// ...
}
Now, your processing code is independent of the way the attributes are read. Sure, in the example above that way is hardcoded. But it does not have to be. Depending on how you want/like to organize your tests, you can use many ways to replace the attributeReader method with other mechanisms.
For example, just add 'virtual' to the attributeReader and use inheritance to create a class that will enable AttributeFaking:
// original class:
virtual object[] attributeReader(PropertyInfo prop)
{
return prop.GetCustomAttributes(true);
}
// derived class:
object[] AttributesOverrides {get;set;}
override object[] attributeReader(PropertyInfo prop)
{
if(prop.Name = "ShoeSize") return AttributesOverrides; // return what I say!
return base.attributeReader(prop);
}
// your test setup
var t = ... // that DERIVED object
t.AttributesOverrides = new [] { ... } ; // attributes to use
For example, use delegates/lambdas, no inheritace
// note the attributeReader is now a field, not function
Func<PropertyInfo, object[]> attributeReader = defaultAttributeReader;
static object[] defaultAttributeReader(PropertyInfo prop)
{
return prop.GetCustomAttributes(true);
}
// and your test setup
var t = ... // that ORIGNAL object
t.attributeReader = customReaderForTheTest; // change the reader on the fly
// that's the reader-function to use in THIS TEST setup
static object[] customReaderForTheTest(PropertyInfo prop)
{
if(prop.Name = "ShoeSize") return null; // crash when I say so! muhaHAHAhaa!
return prop.GetCustomAttributes(true);
}
Both of those two examples end up with one class that is enables faking the attributes in some way, but that's not the only ways to do that. You can use IoC to inject the correct attributeReader. You can do that in any way you like - you just need to abstract from reading part and leave it 'open'.

It is not possible to really apply the attribute at runtime to an existing class, but there are at least two ways you could do something similar to it - it depends on how exactly are you reading those attributes later.
The options focus on the 'really' and 'existing class' part:
1) don't do that, just fake adding them
2) apply them on a class that does not exist yet! ;)
First option is a CustomTypeDescriptor. In its implementations, you will be able to dynamically answer to any queries about Attributes for some class that uses it (-> see virtual GetAttributes method).
This leads to first way:
Create AttributableTestObject that i.e. inherits from your ClassCompatibleWithThatAttribute etc
Create something like DynamicallyAttributedClass : CustomTypeProvider that exposes a static property similar to IEnumerable<Attribute>
override the GetAttributes and return whatever was provided by that static property
on your AttributableTestObject class set a TypeDecriptorProvider attribute pointing to provider (you've got to implement it, again) that returns DynamicallyAttributedClass
Now, using that static property you can change what the GetAttributes returns, and therefore you can dynamically change the setof attributes that are visible through typedescriptor.
And here's the catch: Not all engines/observers/readers/(..) actually care about TypeDescriptors. Some simply read the metadata right away from the Reflection. Reflection will not check the typedescriptors. It will simply return an information that the AttributableTestObject class has a TypeDecriptorProvider property. But whe nusing used the ComponentModel mechanisms, the custom list of attribues will be visible.
That reminds me that the reading mechanisms simply sit at TypeDescriptor class, see its methods. There's also AddAttribute, so maybe you can even get what you want without actually implementing the stuff I said above - Try using AddAttribute and then GetAttribute on your target Type and check the results. It may "just work". But again, it will not fool the reflection.
So, there's a second, "more hardcore" approach - dynamic classes. With System.Reflection.Emit you can dynamically generate an assembly that will contain whatever IL code you wish, and then load the assembly to the current runtime.
This gives you a unique option to simply generate a new fresh class(es) at runtime, with any contents and any attributes you like. You can use inheritance, so you can inherit from any ClassCompatibleWithThatAttributes. It's not that easy to do manually, but there are some libraries that make using the IL emitter easier.
Note that the generated types will be generated at runtime. This means that at compile-time you will not have them checked, you must generate them perfectly or face some really rarely seen Exceptions. Also, note that as the types are not known at compile-time, you cannot simply new the objects. You will have to create the objects basing on their freshly-generated Type through i.e. Activator.CreateInstance(Type).
Also, even though you can generate as much new classes as you want - if you overdo it, you probably will eventually hit some CLR limit, or at leat OutOfMemory, since the generated and loaded assemblies actually occupy space. You can overcome it with AppDomains - you can create some extra AppDomains and generate and load the assemblies inside them, and then, finally, you can unload that extra domain to release all memory and also unload any assemblies that were loaded there. But, I suppose you will not generate that much types to really need that. I'm writing about it just-in-case. Dont worry until you hit the OutOfMemory or similar.

Could a security breach happen through Reflection?

Assume you have a function in C# which allows you to print out the properties of a model like this:
public void Export<T>(List<T> list)
{
// get properties of Model
Type modelType = typeof(T);
var modelInstance = Activator.CreateInstance(modelType);
PropertyInfo[] properties = modelInstance.GetType().GetProperties();
foreach (var property in properties)
{
if (property.CanRead)
PRINTOUT(property.Name);
}
foreach (var entity in list)
{
foreach (var property in properties)
{
if (property.CanRead)
PRINTOUT(property.GetValue(entity, null));
}
}
}
Now, this function is in a controller so, as I've read, is a publicly exposed HTTP endpoint! Is it possible for a malicious user to ask to PRINTOUT properties he should not see?
To avoid this, I'd like to call this function from another controller which pass in a very specific type.
For example, PrintoutCustomers in the controller Customers calls that function while also the controller Products (with the action PrintoutProducts) can call the same exact function but pass in a list of products (instead of customers).
The user will then have a list of specifics publicly accessible HTTP endpoints and no one of them will allows him to do anything harmful.
I'd like that function to be private and shared between all my controllers: is it possible? Can I do that? Or do I really have to copypaste the same method everywhere I need it (and wasted my time tried to make it generic using typeof and reflections)?

Why would you put that method in a controller? It belongs into a static helper class or something like this. You could even make it an extension method, if you want to go fancy.
public static class OutputExtensions
{
public static void Export<T>(this List<T> list)
{
// get properties of Model
PropertyInfo[] properties = typeof(T).GetProperties();
foreach (var property in properties)
{
if (property.CanRead)
PRINTOUT(property.Name);
}
foreach (var entity in list)
{
foreach (var property in properties)
{
if (property.CanRead)
PRINTOUT(property.GetValue(entity, null));
}
}
}
}
(This version also contains the fix suggested by Trevor Pilley in his comment)
You could use it like this in every controller:
listOfModels.Export();

In this case, I think Shannon's maxim applies:
The enemy knows the system.
This is normally cited in the context of cryptographic algorithms, but it works here too. Yes, this method provides malicious users with lists of stuff they might not otherwise know about, but in theory, they could also just query your server for all possible combinations of words and see which ones work; that would also give them a list of open endpoints.
When you leave your house, you don't put camouflage over your door - you lock it. The same principle works here: if there's something a user shouldn't be able to do, make sure your authentication and authorisation code works, and properly denies access to anything you want to keep them out of. Simply hiding things is known as security through obscurity, and in short, it doesn't work.
When you make software that can connect to the web, it's ok (and often a good idea) to hide certain things to reduce the attention they receive from attackers, but even then you should assume that at some point a malicious user will find out that they exist, and you should secure them against malicious use.
In your particular case, I'd say the following:
Can you do without this particular functionality?
Can you do it another way that exposes less of your system?
Can a malicious user use this function to do something you don't want them to be able to do?
If the answer to any of these is "yes", then you should probably not do this. If and only if all of them are "no", then you're fine. Personally, I think you're forgetting a simple rule: if it doesn't need to be accessible from the web, don't put it in a controller! Consider moving this method to an internal class in your logic layer, somewhere that only your trusted code can get at it.

Asserting JsonResult Containing Anonymous Type

I was trying to unit test a method in one of my Controllers returning a JsonResult. To my surprise the following code didn't work:
[HttpPost]
public JsonResult Test() {
return Json(new {Id = 123});
}
This is how I test it (also note that the test code resides in another assembly):
// Act
dynamic jsonResult = testController.Test().Data;
// Assert
Assert.AreEqual(123, jsonResult.Id);
The Assert throws an exception:
'object' does not contain a definition for 'Id'
I've since resolved it by using the following:
[HttpPost]
public JsonResult Test() {
dynamic data = new ExpandoObject();
data.Id = 123;
return Json(data);
}
I'm trying to understand why isn't the first one working ? It also seems to be working with basically anything BUT an anonymous type.

To be clear, the specific problem you are encountering is that C# dynamic does not work with non-public members. This is by design, presumably to discourage that sort of thing. Since as LukLed stated, anonymous types are public only within the same assembly (or to be more precise, anonymous types are simply marked internal, not public), you are running into this barrier.
Probably the cleanest solution would be for you to use InternalsVisibleTo. It allows you to name another assembly that can access its non-public members. Using it for tests is one of the primary reasons for its existance. In your example, you would place in your primary project's AssemblyInfo.cs the following line:
[assembly: InternalsVisibleTo("AssemblyNameOfYourTestProject")]
Once you do that, the error will go away (I just tried it myself).
Alternatively, you could have just used brute force reflection:
Assert.AreEqual(123, jsonResult.GetType().GetProperty("Id").GetValue(jsonResult, null));

Having read the responses here and then looking further afield I found a 2009 msdn blog post with a different approach again. But.. in the comments was a very simple and very elegant solution by Kieran ... to use .ToString().
In your original case:
[HttpPost]
public JsonResult Test()
{
return Json(new {Id = 123});
}
You could test by doing:
var jsonResult = controller.Test();
Assert.AreEqual("{Id = 123}", jsonResult.Data.ToString());
I much prefer this solution as it:
avoids changing the original code (InternalsVisibleTo, ExpandoObject),
avoids using MvcContrib and RhinoMocks (no issue with either of these but why add just to be able to test JsonResult?), and,
avoids using Reflection (adds complexity to the tests).

Anonymous types are internal, so you can't expose them to another library, the one with tests. If you placed testing code in the same library as controller, it will work.

Code Contracts and failure on private static readonly fields

I have a private static readonly field in my class:
public class MyClass
{
// ISSUE #1 -- requires unproven: path != null
private static readonly DirectoryInfo MyDirectory =
new DirectoryInfo(Settings.Default.MyDirectoryPath);
protected virtual void SomeMethod()
{
if (MyDirectory.Exists)
{
// ISSUE #2 -- requires unproven: !string.IsNullOrEmpty(path)
var catalog = new DirectoryCatalog(MyDirectory.FullName);
}
}
}
For issue #1 I used a null coalescing operator to default to some magic string and that fixed it, but I don't really like that solution. I was hoping there was a better solution.
For issue #2 the only thing I can think of is using a Contract.Assumes because if I attempt to use Contract.Requires(MyDirectory.Exists || !String.IsNullOrEmpty(MyDirectory.FullName)); it complains about visibility (private field used in a requires on a protected method).

Issue #1 is a result of Settings.Default.MyDirectoryPath being code generated by Visual Studio without any contracts on the property. This issue is not limited to null strings. Many API's now have contracts that require say a TimeSpan to be non-negative but using a setting directly in the API will generate a Code Contracts warning.
A way to solve this issue is to wrap the setting in a method that has a contract. E.g.:
String GetMyDirectoryPath() {
Contract.Ensures(Contract.Result<String>() != null);
var myDirectoryPath = Settings.Default.MyDirectoryPath;
Contract.Assume(myDirectoryPath != null);
return myDirectoryPath;
}
Notice how the Contract.Assume really performs validation of your setting (which can't be verified by Code Contracts because it is controlled by an external configuration file). Had it been a TimeSpan that is expected to be non-negative you can either use Contract.Assume to do the validation resulting in a ContractException or some other method using your own exception instead.
Adding this extra layer is somewhat tedious but because the setting is defined outside the application it needs to be run-time validated at some point just as you have to validate interactive user input.
Issue #2 is probably because DirectoryInfo doesn't have any contracts defined. The easist way is to use Contract.Assume. This will make a statement about what you believe is the expected behavior of DirectoryInfo but a run-time check will still be in place to ensure that your belief is correct (provided that you keep the checks in your code).
var path = MyDirectory.FullName;
Contract.Assume(!string.IsNullOrEmpty(path));
var catalog = new DirectoryCatalog(path);

After having used Code Contracts in a current project for a while now I have found that it does force you to rewrite your code at times to correct for issues. You really have two options here.
You can add the setting to your project settings to output what the correct attributes to apply are to ignore certain warnings. This is done by adding the "-outputwarnmasks" flag to the "Extra Static Checker Options" under the Advanced section in the Code Contracts tab of the Project file settings. This will add information to the Build Output window giving you the correct attributes to add to ignore the individual cases. (very useful when dealing with Entity Framework).
You can rewrite your code to add the proper Requires and Ensures to your code so that the warnings don't appear.
If you want to rewrite the code:
To solve Issue #1 you would have to wrap the Settings class and expose a new MyDirectoryPath as a property that isn't code generated so that you can add a check in it and return an empty string and add the Contract.Ensures(Contract.Result<string>() != null) at the top of the Getter for the property.
To solve Issue #2 you would have to wrap you access to the class field inside a private static property that adds the proper Ensures and Requires.
I have usually gone with rewriting the code wherever possible except with Entity Framework/LINQ where you need to add the attributes, especially with complex queries.
** Disclaimer ** These are just the ways I have found to solve the issues as there isn't a great deal of information on other ways of working around these types of items.

Well, for Issue#2, I think you might want to use && not ||. But beyond that, perhaps for Issue#1 you can put those checks in the static constructor? Another option for Issue#2 is to have the method to take the directory as a parameter:
private static readonly DirectoryInfo MyDirectory;
static MyClass()
{
Contract.Requires(Settings.Default.MyDirectoryPath != null);
MyDirectory = new DirectoryInfo(Settings.Default.MyDirectoryPath);
}
protected void SomeMethod()
{
SomeOtherMethod(MyDirectory);
}
protected virtual void SomeOtherMethod(DirectoryInfo directory)
{
Contract.Requires(directory.Exists && !String.IsNullOrEmpty(directory.FullName));
var catalog = new DirectoryCatalog(directory.FullName);
}
I don't have much experience working with the Contract API, so I might be off my rocker with all this. :)

Contract.Requires(MyDirectory.Exists || !String.IsNullOrEmpty(MyDirectory.FullName));
Don't do this! MyDirectory.Exists can change at any time and the caller cannot guarantee it. Just throw an exception if the directory doesn't exist - this is what exceptions are for.

Is it Possible to have Dynamic - On-The-Fly DeploymentItems?

I want to write tests to check the existance (and other stuff) of certain files that will be shipped with our project.
This is what I have right now:
[DeploymentItem("1.pdf")]
[DeploymentItem("2.pdf")]
public class DoFilesExist
{
List<string> _Files;
public DoFilesExist()
{
_Files = new List<string>();
_Files.Add("1.pdf");
_Files.Add("2.pdf");
}
delegate void fileTest(string fileName);
void Map(fileTest test)
{
foreach (string file in _Files)
{
test(file);
}
}
[TestMethod]
public void TestExists()
{
Map( x => Assert.IsTrue(File.Exists(x), x + " doesn't exist") );
}
}
As you can see, when I want to add another file to test, I have to add to the [DeploymentItem] and the _Files List
Is there a way to Dynamically Change the DeploymentItems? Or to grab from them during run time. I will probably end up having over 30 files here, and I do not want two lists.
Thanks in advance!

It looks like you're mainly testing whether [DeploymentItem] works... after all - it isn't [DeploymentItem] that defines your actual deployment.
Personally, this is something I despise about MSTest; .NET projects already have a way to define deployment data - the project! By introducing a second method, it introduces both duplication and risk. One of the reasons I use NUnit instead of MSTest, even though I have a VSTS license </rant>.
Re the question; you could use reflection to look at the DeploymentItem markers, but I'm really not sure what this is testing, other than the test framework itself...

One possibility is to put your files into a subdirectory and then you can have the deployment item reference that e.g.
[DeploymentItem("files\"]
public class Test
{
}
You might need to mark each file with the CopyAlways as well - see the main question for details.