Related
I've read various articles about mocking vs stubbing in testing, including Martin Fowler's Mocks Aren't Stubs, but still don't understand the difference.
Foreword
There are several definitions of objects, that are not real. The general term is test double. This term encompasses: dummy, fake, stub, mock.
Reference
According to Martin Fowler's article:
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
Style
Mocks vs Stubs = Behavioral testing vs State testing
Principle
According to the principle of Test only one thing per test, there may be several stubs in one test, but generally there is only one mock.
Lifecycle
Test lifecycle with stubs:
Setup - Prepare object that is being tested and its stubs collaborators.
Exercise - Test the functionality.
Verify state - Use asserts to check object's state.
Teardown - Clean up resources.
Test lifecycle with mocks:
Setup data - Prepare object that is being tested.
Setup expectations - Prepare expectations in mock that is being used by primary object.
Exercise - Test the functionality.
Verify expectations - Verify that correct methods has been invoked in mock.
Verify state - Use asserts to check object's state.
Teardown - Clean up resources.
Summary
Both mocks and stubs testing give an answer for the question: What is the result?
Testing with mocks are also interested in: How the result has been achieved?
Stub
I believe the biggest distinction is that a stub you have already written with predetermined behavior. So you would have a class that implements the dependency (abstract class or interface most likely) you are faking for testing purposes and the methods would just be stubbed out with set responses. They would not do anything fancy and you would have already written the stubbed code for it outside of your test.
Mock
A mock is something that as part of your test you have to setup with your expectations. A mock is not setup in a predetermined way so you have code that does it in your test. Mocks in a way are determined at runtime since the code that sets the expectations has to run before they do anything.
Difference between Mocks and Stubs
Tests written with mocks usually follow an initialize -> set expectations -> exercise -> verify pattern to testing. While the pre-written stub would follow an initialize -> exercise -> verify.
Similarity between Mocks and Stubs
The purpose of both is to eliminate testing all the dependencies of a class or function so your tests are more focused and simpler in what they are trying to prove.
A stub is a simple fake object. It just makes sure test runs smoothly.
A mock is a smarter stub. You verify your test passes through it.
Here's a description of each one followed by with real world sample.
Dummy - just bogus values to satisfy the API.
Example: If you're testing a method of a class which requires many mandatory parameters in a constructor which have no effect on your test, then you may create dummy objects for the purpose of creating new instances of a class.
Fake - create a test implementation of a class which may have a dependency on some external infrastructure. (It's good practice that your unit test does NOT actually interact with external infrastructure.)
Example: Create fake implementation for accessing a database, replace it with in-memory collection.
Stub - override methods to return hard-coded values, also referred to as state-based.
Example: Your test class depends on a method Calculate() taking 5 minutes to complete. Rather than wait for 5 minutes you can replace its real implementation with stub that returns hard-coded values; taking only a small fraction of the time.
Mock - very similar to Stub but interaction-based rather than state-based. This means you don't expect from Mock to return some value, but to assume that specific order of method calls are made.
Example: You're testing a user registration class. After calling Save, it should call SendConfirmationEmail.
Stubs and Mocks are actually sub types of Mock, both swap real implementation with test implementation, but for different, specific reasons.
In the codeschool.com course, Rails Testing for Zombies, they give this definition of the terms:
Stub
For replacing a method with code that returns a specified result.
Mock
A stub with an assertion that the method gets called.
So as Sean Copenhaver described in his answer, the difference is that mocks set expectations (i.e. make assertions, about whether or how they get called).
Stubs don't fail your tests, mock can.
Reading all the explanations above, let me try to condense:
Stub: a dummy piece of code that lets the test run, but you don't care what happens to it. Substitutes for real working code.
Mock: a dummy piece of code that you verify is called correctly as part of the test. Substitutes for real working code.
Spy: a dummy piece of code that intercepts and verifies some calls to real working code, avoiding the need to substitute all the real code.
I think the simplest and clearer answer about this question is given from Roy Osherove in his book The art of Unit Testing (page 85)
The easiest way to tell we’re dealing with a stub is to notice that the stub can never fail the test. The asserts the test uses are always against
the class under test.
On the other hand, the test will use a mock object to verify whether the
test failed or not. [...]
Again, the mock object is the object we use to see if the test failed or not.
Stub and mock are both fakes.
If you are making assertions against the fake it means you are using the fake as a mock, if you are using the fake only to run the test without assertion over it you are using the fake as a stub.
A Mock is just testing behaviour, making sure certain methods are called.
A Stub is a testable version (per se) of a particular object.
What do you mean an Apple way?
If you compare it to debugging:
Stub is like making sure a method returns the correct value
Mock is like actually stepping into the method and making sure everything inside is correct before returning the correct value.
This slide explain the main differences very good.
*From CSE 403 Lecture 16 , University of Washington (slide created by "Marty Stepp")
To be very clear and practical:
Stub: A class or object that implements the methods of the class/object to be faked and returns always what you want.
Example in JavaScript:
var Stub = {
method_a: function(param_a, param_b){
return 'This is an static result';
}
}
Mock: The same of stub, but it adds some logic that "verifies" when a method is called so you can be sure some implementation is calling that method.
As #mLevan says imagine as an example that you're testing a user registration class. After calling Save, it should call SendConfirmationEmail.
A very stupid code Example:
var Mock = {
calls: {
method_a: 0
}
method_a: function(param_a, param_b){
this.method_a++;
console.log('Mock.method_a its been called!');
}
}
let see Test Doubles:
Fake: Fakes are objects that have working implementations, but not the same as production one. Such as: in-memory implementation of Data Access Object or Repository.
Stub: Stub is an object that holds predefined data and uses it to answer calls during tests. Such as: an object that needs to grab some data from the database to respond to a method call.
Mocks: Mocks are objects that register calls they receive.
In test assertion, we can verify on Mocks that all expected actions were performed. Such as: a functionality that calls e-mail sending service.
for more just check this.
Using a mental model really helped me understand this, rather than all of the explanations and articles, that didn't quite "sink in".
Imagine your kid has a glass plate on the table and he starts playing with it. Now, you're afraid it will break. So, you give him a plastic plate instead. That would be a Mock (same behavior, same interface, "softer" implementation).
Now, say you don't have the plastic replacement, so you explain "If you continue playing with it, it will break!". That's a Stub, you provided a predefined state in advance.
A Dummy would be the fork he didn't even use... and a Spy could be something like providing the same explanation you already used that worked.
I think the most important difference between them is their intentions.
Let me try to explain it in WHY stub vs. WHY mock
Suppose I'm writing test code for my mac twitter client's public timeline controller
Here is test sample code
twitter_api.stub(:public_timeline).and_return(public_timeline_array)
client_ui.should_receive(:insert_timeline_above).with(public_timeline_array)
controller.refresh_public_timeline
STUB: The network connection to twitter API is very slow, which make my test slow. I know it will return timelines, so I made a stub simulating HTTP twitter API, so that my test will run it very fast, and I can running the test even I'm offline.
MOCK: I haven't written any of my UI methods yet, and I'm not sure what methods I need to write for my ui object. I hope to know how my controller will collaborate with my ui object by writing the test code.
By writing mock, you discover the objects collaboration relationship by verifying the expectation are met, while stub only simulate the object's behavior.
I suggest to read this article if you're trying to know more about mocks: http://jmock.org/oopsla2004.pdf
I like the explanantion put out by Roy Osherove [video link].
Every class or object created is a Fake. It is a Mock if you verify
calls against it. Otherwise its a stub.
Stub
A stub is an object used to fake a method that has pre-programmed behavior. You may want to use this instead of an existing method in order to avoid unwanted side-effects (e.g. a stub could make a fake fetch call that returns a pre-programmed response without actually making a request to a server).
Mock
A mock is an object used to fake a method that has pre-programmed behavior as well as pre-programmed expectations. If these expectations are not met then the mock will cause the test to fail (e.g. a mock could make a fake fetch call that returns a pre-programmed response without actually making a request to a server which would expect e.g. the first argument to be "http://localhost:3008/" otherwise the test would fail.)
Difference
Unlike mocks, stubs do not have pre-programmed expectations that could fail your test.
Stubs vs. Mocks
Stubs
provide specific answers to methods calls
ex: myStubbedService.getValues() just return a String needed by the code under test
used by code under test to isolate it
cannot fail test
ex: myStubbedService.getValues() just returns the stubbed value
often implement abstract methods
Mocks
"superset" of stubs; can assert that certain methods are called
ex: verify that myMockedService.getValues() is called only once
used to test behaviour of code under test
can fail test
ex: verify that myMockedService.getValues() was called once; verification fails, because myMockedService.getValues() was not called by my tested code
often mocks interfaces
I was reading The Art of Unit Testing, and stumbled upon the following definition:
A fake is a generic term that can be used to describe either a stub or a mock object (handwritten or otherwise), because they both look like the real object. Whether a fake is a stub or a mock depends on how it's used in the current test. if it's used to check an interaction (asserted against), it's a mock object. Otherwise, it's a stub.
The generic term he uses is a Test Double (think stunt double). Test Double is a generic term for any case where you replace a production object for testing purposes. There are various kinds of double that Gerard lists:
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test.
Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent(also called Partial Mock).
Mocks are pre-programmed with expectations which form a specification of the calls they are expected to receive. They can throw an exception if they receive a call they don't expect and are checked during verification to ensure they got all the calls they were expecting.
Source
A fake is a generic term that can be used to describe either a stub
or a mock object (handwritten or otherwise), because they both look like the
real object. Whether a fake is a stub or a mock depends on how it’s used in
the current test. If it’s used to check an interaction (asserted against), it’s a
mock object. Otherwise, it’s a stub.
Fakes makes sure test runs smoothly. It means that reader of your future test will understand what will be the behavior of the fake object, without needing to read its source code (without needing to depend on external resource).
What does test run smoothly mean?
Forexample in below code:
public void Analyze(string filename)
{
if(filename.Length<8)
{
try
{
errorService.LogError("long file entered named:" + filename);
}
catch (Exception e)
{
mailService.SendEMail("admin#hotmail.com", "ErrorOnWebService", "someerror");
}
}
}
You want to test mailService.SendEMail() method, to do that you need to simulate an Exception in you test method, so you just need to create a Fake Stub errorService class to simulate that result, then your test code will be able to test mailService.SendEMail() method. As you see you need to simulate a result which is from an another External Dependency ErrorService class.
Mocks: help to emulate and examine outcoming interactions. These interactions
are calls the SUT makes to its dependencies to change their state.
Stubs: help to emulate incoming interactions. These interactions are calls the
SUT makes to its dependencies to get input data.
source : Unit Testing Principles, Practices, and Patterns - Manning
Right from the paper Mock Roles, not Objects, by the developers of jMock :
Stubs are dummy implementations of production code that return canned
results. Mock Objects act as stubs, but also include assertions to
instrument the interactions of the target object with its neighbours.
So, the main differences are:
expectations set on stubs are usually generic, while expectations set on mocks can be more "clever" (e.g. return this on the first call, this on the second etc.).
stubs are mainly used to setup indirect inputs of the SUT, while mocks can be used to test both indirect inputs and indirect outputs of the SUT.
To sum up, while also trying to disperse the confusion from Fowler's article title: mocks are stubs, but they are not only stubs.
I came across this interesting article by UncleBob The Little Mocker. It explains all the terminology in a very easy to understand manner, so its useful for beginners. Martin Fowlers article is a hard read especially for beginners like me.
a lot of valid answers up there but I think worth to mention this form uncle bob:
https://8thlight.com/blog/uncle-bob/2014/05/14/TheLittleMocker.html
the best explanation ever with examples!
A mock is both a technical and a functional object.
The mock is technical. It is indeed created by a mocking library (EasyMock, JMockit and more recently Mockito are known for these) thanks to byte code generation.
The mock implementation is generated in a way where we could instrument it to return a specific value when a method is invoked but also some other things such as verifying that a mock method was invoked with some specific parameters (strict check) or whatever the parameters (no strict check).
Instantiating a mock :
#Mock Foo fooMock
Recording a behavior :
when(fooMock.hello()).thenReturn("hello you!");
Verifying an invocation :
verify(fooMock).hello()
These are clearly not the natural way to instantiate/override the Foo class/behavior. That's why I refer to a technical aspect.
But the mock is also functional because it is an instance of the class we need to isolate from the SUT. And with recorded behaviors on it, we could use it in the SUT in the same way than we would do with a stub.
The stub is just a functional object : that is an instance of the class we need to isolate from the SUT and that's all.
That means that both the stub class and all behaviors fixtures needed during our unit tests have to be defined explicitly.
For example to stub hello() would need to subclass the Foo class (or implements its interface it has it) and to override hello() :
public class HelloStub extends Hello{
public String hello {
return "hello you!";
}
}
If another test scenario requires another value return, we would probably need to define a generic way to set the return :
public class HelloStub extends Hello{
public HelloStub(String helloReturn){
this.helloReturn = helloReturn;
}
public String hello {
return helloReturn;
}
}
Other scenario : if I had a side effect method (no return) and I would check that that method was invoked, I should probably have added a boolean or a counter in the stub class to count how many times the method was invoked.
Conclusion
The stub requires often much overhead/code to write for your unit test. What mock prevents thanks to providing recording/verifying features out of the box.
That's why nowadays, the stub approach is rarely used in practice with the advent of excellent mock libraries.
About the Martin Fowler Article : I don't think to be a "mockist" programmer while I use mocks and I avoid stubs.
But I use mock when it is really required (annoying dependencies) and I favor test slicing and mini-integration tests when I test a class with dependencies which mocking would be an overhead.
Plus useful answers, One of the most powerful point of using Mocks than Subs
If the collaborator [which the main code depend on it] is not under our control (e.g. from a third-party library),
In this case, stub is more difficult to write rather than mock.
Stub
A stub is an object that holds predefined data and uses it to answer calls during tests. It is used when you can’t or don’t want to involve objects that would answer with real data or have undesirable side effects.
An example can be an object that needs to grab some data from the database to respond to a method call. Instead of the real object, we introduced a stub and defined what data should be returned.
example of Stub:
public class GradesService {
private final Gradebook gradebook;
public GradesService(Gradebook gradebook) {
this.gradebook = gradebook;
}
Double averageGrades(Student student) {
return average(gradebook.gradesFor(student));
}
}
Instead of calling database from Gradebook store to get real students grades, you preconfigure stub with grades that will be returned. You define just enough data to test average calculation algorithm.
public class GradesServiceTest {
private Student student;
private Gradebook gradebook;
#Before
public void setUp() throws Exception {
gradebook = mock(Gradebook.class);
student = new Student();
}
#Test
public void calculates_grades_average_for_student() {
//stubbing gradebook
when(gradebook.gradesFor(student)).thenReturn(grades(8, 6, 10));
double averageGrades = new GradesService(gradebook).averageGrades(student);
assertThat(averageGrades).isEqualTo(8.0);
}
}
Mock
Mocks are objects that register calls they receive. In test assertion you can verify on Mocks that all expected actions were performed. You use mocks when you don’t want to invoke production code or when there is no easy way to verify, that intended code was executed. There is no return value and no easy way to check system state change. An example can be a functionality that calls e-mail sending service.
You don’t want to send e-mails each time you run a test. Moreover, it is not easy to verify in tests that a right email was send. Only thing you can do is to verify the outputs of the functionality that is exercised in our test. In other worlds, verify that the e-mail sending service was called.
Example of Mock:
public class SecurityCentral {
private final Window window;
private final Door door;
public SecurityCentral(Window window, Door door) {
this.window = window;
this.door = door;
}
void securityOn() {
window.close();
door.close();
}
}
You don’t want to close real doors to test that security method is working, right? Instead, you place door and window mocks objects in the test code.
public class SecurityCentralTest {
Window windowMock = mock(Window.class);
Door doorMock = mock(Door.class);
#Test
public void enabling_security_locks_windows_and_doors() {
SecurityCentral securityCentral = new SecurityCentral(windowMock, doorMock);
securityCentral.securityOn();
verify(doorMock).close();
verify(windowMock).close();
}
}
Thanks a lot to Michał Lipski for his good article. For further reading:
Test Double – Martin Fowler https://martinfowler.com/bliki/TestDouble.html
Test Double – xUnit Patterns http://xunitpatterns.com/Test%20Double.html
Mocks Aren’t Stubs – Martin Fowler https://martinfowler.com/articles/mocksArentStubs.html
Command Query Separation – Martin Fowler https://martinfowler.com/bliki/CommandQuerySeparation.html
Stub helps us to run test. How? It gives values which helps to run test. These values are itself not real and we created these values just to run the test. For example we create a HashMap to give us values which are similar to values in database table. So instead of directly interacting with database we interact with Hashmap.
Mock is an fake object which runs the test. where we put assert.
See below example of mocks vs stubs using C# and Moq framework. Moq doesn't have a special keyword for Stub but you can use Mock object to create stubs too.
namespace UnitTestProject2
{
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
[TestClass]
public class UnitTest1
{
/// <summary>
/// Test using Mock to Verify that GetNameWithPrefix method calls Repository GetName method "once" when Id is greater than Zero
/// </summary>
[TestMethod]
public void GetNameWithPrefix_IdIsTwelve_GetNameCalledOnce()
{
// Arrange
var mockEntityRepository = new Mock<IEntityRepository>();
mockEntityRepository.Setup(m => m.GetName(It.IsAny<int>()));
var entity = new EntityClass(mockEntityRepository.Object);
// Act
var name = entity.GetNameWithPrefix(12);
// Assert
mockEntityRepository.Verify(m => m.GetName(It.IsAny<int>()), Times.Once);
}
/// <summary>
/// Test using Mock to Verify that GetNameWithPrefix method doesn't call Repository GetName method when Id is Zero
/// </summary>
[TestMethod]
public void GetNameWithPrefix_IdIsZero_GetNameNeverCalled()
{
// Arrange
var mockEntityRepository = new Mock<IEntityRepository>();
mockEntityRepository.Setup(m => m.GetName(It.IsAny<int>()));
var entity = new EntityClass(mockEntityRepository.Object);
// Act
var name = entity.GetNameWithPrefix(0);
// Assert
mockEntityRepository.Verify(m => m.GetName(It.IsAny<int>()), Times.Never);
}
/// <summary>
/// Test using Stub to Verify that GetNameWithPrefix method returns Name with a Prefix
/// </summary>
[TestMethod]
public void GetNameWithPrefix_IdIsTwelve_ReturnsNameWithPrefix()
{
// Arrange
var stubEntityRepository = new Mock<IEntityRepository>();
stubEntityRepository.Setup(m => m.GetName(It.IsAny<int>()))
.Returns("Stub");
const string EXPECTED_NAME_WITH_PREFIX = "Mr. Stub";
var entity = new EntityClass(stubEntityRepository.Object);
// Act
var name = entity.GetNameWithPrefix(12);
// Assert
Assert.AreEqual(EXPECTED_NAME_WITH_PREFIX, name);
}
}
public class EntityClass
{
private IEntityRepository _entityRepository;
public EntityClass(IEntityRepository entityRepository)
{
this._entityRepository = entityRepository;
}
public string Name { get; set; }
public string GetNameWithPrefix(int id)
{
string name = string.Empty;
if (id > 0)
{
name = this._entityRepository.GetName(id);
}
return "Mr. " + name;
}
}
public interface IEntityRepository
{
string GetName(int id);
}
public class EntityRepository:IEntityRepository
{
public string GetName(int id)
{
// Code to connect to DB and get name based on Id
return "NameFromDb";
}
}
}
I know how I use these terms, but I'm wondering if there are accepted definitions for faking, mocking, and stubbing for unit tests? How do you define these for your tests? Describe situations where you might use each.
Here is how I use them:
Fake: a class that implements an interface but contains fixed data and no logic. Simply returns "good" or "bad" data depending on the implementation.
Mock: a class that implements an interface and allows the ability to dynamically set the values to return/exceptions to throw from particular methods and provides the ability to check if particular methods have been called/not called.
Stub: Like a mock class, except that it doesn't provide the ability to verify that methods have been called/not called.
Mocks and stubs can be hand generated or generated by a mocking framework. Fake classes are generated by hand. I use mocks primarily to verify interactions between my class and dependent classes. I use stubs once I have verified the interactions and am testing alternate paths through my code. I use fake classes primarily to abstract out data dependencies or when mocks/stubs are too tedious to set up each time.
You can get some information :
From Martin Fowler about Mock and Stub
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
From xunitpattern:
Fake: We acquire or build a very lightweight implementation of the same functionality as provided by a component that the SUT depends on and instruct the SUT to use it instead of the real.
Stub : This implementation is configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested Code (see Production Bugs on page X) within the SUT. A key indication for using a Test Stub is having Untested Code caused by the inability to control the indirect inputs of the SUT
Mock Object that implements the same interface as an object on which the SUT (System Under Test) depends. We can use a Mock Object as an observation point when we need to do Behavior Verification to avoid having an Untested Requirement (see Production Bugs on page X) caused by an inability to observe side-effects of invoking methods on the SUT.
Personally
I try to simplify by using : Mock and Stub. I use Mock when it's an object that returns a value that is set to the tested class. I use Stub to mimic an Interface or Abstract class to be tested. In fact, it doesn't really matter what you call it, they are all classes that aren't used in production, and are used as utility classes for testing.
Stub - an object that provides predefined answers to method calls.
Mock - an object on which you set expectations.
Fake - an object with limited capabilities (for the purposes of testing), e.g. a fake web service.
Test Double is the general term for stubs, mocks and fakes. But informally, you'll often hear people simply call them mocks.
I am surprised that this question has been around for so long and nobody has as yet provided an answer based on Roy Osherove's "The Art of Unit Testing".
In "3.1 Introducing stubs" defines a stub as:
A stub is a controllable replacement for an existing dependency
(or collaborator) in the system. By using a stub, you can test your code without
dealing with the dependency directly.
And defines the difference between stubs and mocks as:
The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub.
Fake is just the name used for both stubs and mocks. For example when you don't care about the distinction between stubs and mocks.
The way Osherove's distinguishes between stubs and mocks, means that any class used as a fake for testing can be both a stub or a mock. Which it is for a specific test depends entirely on how you write the checks in your test.
When your test checks values in the class under test, or actually anywhere but the fake, the fake was used as a stub. It just provided values for the class under test to use, either directly through values returned by calls on it or indirectly through causing side effects (in some state) as a result of calls on it.
When your test checks values of the fake, it was used as a mock.
Example of a test where class FakeX is used as a stub:
const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);
cut.SquareIt;
Assert.AreEqual(25, cut.SomeProperty);
The fake instance is used as a stub because the Assert doesn't use fake at all.
Example of a test where test class X is used as a mock:
const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);
cut.SquareIt;
Assert.AreEqual(25, fake.SomeProperty);
In this case the Assert checks a value on fake, making that fake a mock.
Now, of course these examples are highly contrived, but I see great merit in this distinction. It makes you aware of how you are testing your stuff and where the dependencies of your test are.
I agree with Osherove's that
from a pure maintainability perspective, in my tests using mocks creates more trouble than not using them. That has been my experience, but I’m always learning something new.
Asserting against the fake is something you really want to avoid as it makes your tests highly dependent upon the implementation of a class that isn't the one under test at all. Which means that the tests for class ActualClassUnderTest can start breaking because the implementation for ClassUsedAsMock changed. And that sends up a foul smell to me. Tests for ActualClassUnderTest should preferably only break when ActualClassUnderTest is changed.
I realize that writing asserts against the fake is a common practice, especially when you are a mockist type of TDD subscriber. I guess I am firmly with Martin Fowler in the classicist camp (See Martin Fowler's "Mocks aren't Stubs") and like Osherove avoid interaction testing (which can only be done by asserting against the fake) as much as possible.
For fun reading on why you should avoid mocks as defined here, google for "fowler mockist classicist". You'll find a plethora of opinions.
As mentioned by the top-voted answer, Martin Fowler discusses these distinctions in Mocks Aren't Stubs, and in particular the subheading The Difference Between Mocks and Stubs, so make sure to read that article.
Rather than focusing on how these things are different, I think it's more enlightening to focus on why these are distinct concepts. Each exists for a different purpose.
Fakes
A fake is an implementation that behaves "naturally", but is not "real". These are fuzzy concepts and so different people have different understandings of what makes things a fake.
One example of a fake is an in-memory database (e.g. using sqlite with the :memory: store). You would never use this for production (since the data is not persisted), but it's perfectly adequate as a database to use in a testing environment. It's also much more lightweight than a "real" database.
As another example, perhaps you use some kind of object store (e.g. Amazon S3) in production, but in a test you can simply save objects to files on disk; then your "save to disk" implementation would be a fake. (Or you could even fake the "save to disk" operation by using an in-memory filesystem instead.)
As a third example, imagine an object that provides a cache API; an object that implements the correct interface but that simply performs no caching at all but always returns a cache miss would be a kind of fake.
The purpose of a fake is not to affect the behavior of the system under test, but rather to simplify the implementation of the test (by removing unnecessary or heavyweight dependencies).
Stubs
A stub is an implementation that behaves "unnaturally". It is preconfigured (usually by the test set-up) to respond to specific inputs with specific outputs.
The purpose of a stub is to get your system under test into a specific state. For example, if you are writing a test for some code that interacts with a REST API, you could stub out the REST API with an API that always returns a canned response, or that responds to an API request with a specific error. This way you could write tests that make assertions about how the system reacts to these states; for example, testing the response your users get if the API returns a 404 error.
A stub is usually implemented to only respond to the exact interactions you've told it to respond to. But the key feature that makes something a stub is its purpose: a stub is all about setting up your test case.
Mocks
A mock is similar to a stub, but with verification added in. The purpose of a mock is to make assertions about how your system under test interacted with the dependency.
For example, if you are writing a test for a system that uploads files to a website, you could build a mock that accepts a file and that you can use to assert that the uploaded file was correct. Or, on a smaller scale, it's common to use a mock of an object to verify that the system under test calls specific methods of the mocked object.
Mocks are tied to interaction testing, which is a specific testing methodology. People who prefer to test system state rather than system interactions will use mocks sparingly if at all.
Test doubles
Fakes, stubs, and mocks all belong to the category of test doubles. A test double is any object or system you use in a test instead of something else. Most automated software testing involves the use of test doubles of some kind or another. Some other kinds of test doubles include dummy values, spies, and I/O blackholes.
The thing that you assert on it is called a mock object.
Everything else that just helped the test run is a stub.
To illustrate the usage of stubs and mocks, I would like to also include an example based on Roy Osherove's "The Art of Unit Testing".
Imagine, we have a LogAnalyzer application which has the sole functionality of printing logs. It not only needs to talk to a web service, but if the web service throws an error, LogAnalyzer has to log the error to a different external dependency, sending it by email to the web service administrator.
Here’s the logic we’d like to test inside LogAnalyzer:
if(fileName.Length<8)
{
try
{
service.LogError("Filename too short:" + fileName);
}
catch (Exception e)
{
email.SendEmail("a","subject",e.Message);
}
}
How do you test that LogAnalyzer calls the email service correctly when the web service throws an exception?
Here are the questions we’re faced with:
How can we replace the web service?
How can we simulate an exception from the web service so that we can
test the call to the email service?
How will we know that the email service was called correctly or at
all?
We can deal with the first two questions by using a stub for the web service. To solve the third problem, we can use a mock object for the email service.
A fake is a generic term that can be used to describe either a stub or a mock.In our test, we’ll have two fakes. One will be the email service mock, which we’ll use to verify that the correct parameters were sent to the email service. The other will be a stub that we’ll use to simulate an exception thrown from the web service. It’s a stub because we won’t be using the web service fake to verify the test result, only to make sure the test runs correctly. The email service is a mock because we’ll assert against it that it was called correctly.
[TestFixture]
public class LogAnalyzer2Tests
{
[Test]
public void Analyze_WebServiceThrows_SendsEmail()
{
StubService stubService = new StubService();
stubService.ToThrow= new Exception("fake exception");
MockEmailService mockEmail = new MockEmailService();
LogAnalyzer2 log = new LogAnalyzer2();
log.Service = stubService
log.Email=mockEmail;
string tooShortFileName="abc.ext";
log.Analyze(tooShortFileName);
Assert.AreEqual("a",mockEmail.To); //MOCKING USED
Assert.AreEqual("fake exception",mockEmail.Body); //MOCKING USED
Assert.AreEqual("subject",mockEmail.Subject);
}
}
Unit testing - is an approach of testing where the unit(class, method) is under control.
Test double - is not a primary object(from OOP world). It is a realisation which is created temporary to test, check or during development. And they are created for closing dependencies of tested unit(method, class...)
Test doubles types:
fake object is a real implementation of interface(protocol) or an extend which is using an inheritance or other approaches which can be used to create - is dependency. Usually it is created by developer as a simplest solution to substitute some dependency
stub object is a bare object(0, nil and methods without logic) with extra state which is predefined(by developer) to define returned values. Usually it is created by framework
class StubA: A {
override func foo() -> String {
return "My Stub"
}
}
mock object is very similar to stub object but the extra state is changed during program execution to check if something happened(method was called, arguments, when, how often...).
class MockA: A {
var isFooCalled = false
override func foo() -> String {
isFooCalled = true
return "My Mock"
}
}
spy object is a real object with a "partial mocking". It means that you work with a non-double object except mocked behavior
dummy object is object which is necessary to run a test but no one variable or method of this object is not called.
stub vs mock
Martin Fowler said
There is a difference in that the stub uses state verification while the mock uses behavior verification.
[Mockito mock vs spy]
All of them are called Test Doubles and used to inject the dependencies that your test case needs.
Stub:
It already has a predefined behavior to set your expectation
for example, stub returns only the success case of your API response
A mock is a smarter stub. You verify your test passes through it.
so you could make amock that return either the success or failure success depending on the condition could be changed in your test case.
If you are familiar with Arrange-Act-Assert, then one way of explaining the difference between stub and mock that might be useful for you, is that stubs belong to the arrange section as they are for arranging input state, and mocks belong to the assert section as they are for asserting results against.
Dummies don't do anything. They are just for filling up parameter lists, so that you don't get undefined or null errors. They also exist to satisfy the type checker in statically typed languages, so that you can be allowed to compile and run.
Stub, Fakes and Mocks have different meanings across different sources. I suggest you to introduce your team internal terms and agree upon their meaning.
I think it is important to distinguish between two approaches:
- behaviour validation (implies behaviour substitution)
- end-state validation (implies behaviour emulation)
Consider email sending in case of error. When doing behaviour validation - you check that method Send of IEmailSender was executed once. And you need to emulate return result of this method, return Id of the sent message. So you say: "I expect that Send will be called. And I will just return dummy (or random) Id for any call". This is behaviour validation:
emailSender.Expect(es=>es.Send(anyThing)).Return((subject,body) => "dummyId")
When doing state validation you will need to create TestEmailSender that implements IEmailSender. And implement Send method - by saving input to some data structure that will be used for future state verification like array of some objects SentEmails and then it tests you will check that SentEmails contains expected email. This is state validation:
Assert.AreEqual(1, emailSender.SentEmails.Count)
From my readings I understood that Behaviour validation usually called Mocks.
And State validation usually called Stubs or Fakes.
It's a matter of making the tests expressive. I set expectations on a Mock if I want the test to describe a relationship between two objects. I stub return values if I'm setting up a supporting object to get me to the interesting behaviour in the test.
stub and fake are objects in that they can vary their response based on input parameters. the main difference between them is that a Fake is closer to a real-world implementation than a stub. Stubs contain basically hard-coded responses to an expected request. Let see an example:
public class MyUnitTest {
#Test
public void testConcatenate() {
StubDependency stubDependency = new StubDependency();
int result = stubDependency.toNumber("one", "two");
assertEquals("onetwo", result);
}
}
public class StubDependency() {
public int toNumber(string param) {
if (param == “one”) {
return 1;
}
if (param == “two”) {
return 2;
}
}
}
A mock is a step up from fakes and stubs. Mocks provide the same functionality as stubs but are more complex. They can have rules defined for them that dictate in what order methods on their API must be called. Most mocks can track how many times a method was called and can react based on that information. Mocks generally know the context of each call and can react differently in different situations. Because of this, mocks require some knowledge of the class they are mocking. a stub generally cannot track how many times a method was called or in what order a sequence of methods was called. A mock looks like:
public class MockADependency {
private int ShouldCallTwice;
private boolean ShouldCallAtEnd;
private boolean ShouldCallFirst;
public int StringToInteger(String s) {
if (s == "abc") {
return 1;
}
if (s == "xyz") {
return 2;
}
return 0;
}
public void ShouldCallFirst() {
if ((ShouldCallTwice > 0) || ShouldCallAtEnd)
throw new AssertionException("ShouldCallFirst not first thod called");
ShouldCallFirst = true;
}
public int ShouldCallTwice(string s) {
if (!ShouldCallFirst)
throw new AssertionException("ShouldCallTwice called before ShouldCallFirst");
if (ShouldCallAtEnd)
throw new AssertionException("ShouldCallTwice called after ShouldCallAtEnd");
if (ShouldCallTwice >= 2)
throw new AssertionException("ShouldCallTwice called more than twice");
ShouldCallTwice++;
return StringToInteger(s);
}
public void ShouldCallAtEnd() {
if (!ShouldCallFirst)
throw new AssertionException("ShouldCallAtEnd called before ShouldCallFirst");
if (ShouldCallTwice != 2) throw new AssertionException("ShouldCallTwice not called twice");
ShouldCallAtEnd = true;
}
}
According to the book "Unit Testing Principles, Practices, and Patterns by Vladimir Khorikov" :
Mocks: help to emulate and examine outcoming interactions. These interactions are calls the SUT makes to its dependencies to change their state. In other words it helps to examine the interaction (behaviour) of SUT and its dependencies. mocks could be :
Spy : created manually
Mocks : created using framework
Stubs: helps to emulate incoming interactions. These interactions are calls the SUT makes to its dependencies to get input data. IN other words it helps to test the data passed to SUT. It could be 3 types
Fake: is usually implemented to replace a dependency that doesn’t yet exist.
Dummy: is hard-coded value.
Stubs: Fledged dependency that you configure to return different values for different scenarios.
In xUnit Test Patterns book by Gerard Meszaros There is a nice table that gives a good insight about differences
I tend to use just 2 terms - Fake and Mock.
Mock only when using a mocking framework like Moq for example because it doesn't seem right to refer to it as a Fake when it's being created with new Mock<ISomething>() - while you can technically use a mocking framework to create Stubs or Fakes, it just seems kind of dumb to call it that in this situation - it has to be a Mock.
Fake for everything else. If a Fake can be summarised as an implementation with reduced capabilities, then I think a Stub could also be a Fake (and if not, who cares, everyone knows what I mean, and not once has anyone ever said "I think you'll find that's a Stub")
I've been trying to shift myself into a more test driven methodology when writing my .net MVC based app. I'm doing all my dependency injection using constructor-based injection. Thus far it's going well, but I've found myself doing something repeatedly and I'm wondering if there is a better practice out there.
Let's say I want to test a Controller. It has a dependency on a Unit Of Work (database) object. Simple enough... I write my controller to take that interface in its constructor, and my DI framework (Ninject) can inject it at runtime. Easy. Meanwhile in my unit test, I can manually construct my controller with a mocked-up database object. I like that I can write lots of individual self-contained tests that take care of all the object construction and testing.
Now I've moved on and started adding new features & functions to my Controller object. The Controller now has one or two more dependencies. I can write more tests using these 3 dependencies, but my older tests are all broken (won't compile), as the compiler throws a whole bunch of errors like this:
'MyProject.Web.Api.Controllers.MyExampleController' does not contain a constructor that takes 3 arguments
What I've been doing (which smells bad) is going back and updating all my unit tests, changing the construction code, and adding null parameters for all the new dependencies that my old tests don't care about, like this:
From this:
var controllerToTest = new MyExampleController(mockUOW.Object);
To this:
var controllerToTest = new MyExampleController(mockUOW.Object, null, null);
This gets everything to compile and gets my tests to run again, but I don't like the prospect of going back and editing tons of old tests just to change calls to my object constructors.
Is there a better way to write your unit tests (or a better way to write your classes and do DI?) so they don't all break when you add a new dependency?
You've run into a common issue when unit testing: duplication of code causing significant refactoring overhead. The solution is to try to restrict the instantiation of the object(s) under test to a single place. If you can, try to do it in the TestInitialize method of your tests:
[TestInitialize]
public void Init()
{
this.mockUOW = new Mock<ISomeDependency>();
this.mock2 = new Mock<IAnotherDependency>();
this.mock3 = new Mock<IYetAnotherDependency>();
// Do initial set-up on your mocks
this.controllerToTest = new MyExampleController(this.mockUOW, this.mock2, this.mock3);
}
Sometimes, this isn't practical: you need to do specific set-up in each test before you create an instance of the class under test. In this situation, move the code to create the object into a well-named method, and call that:
[TestMethod]
public void MyTestMethod()
{
// Do any required set-up on mocks, etc.
this.CreateController();
}
private void CreateController()
{
this.controllerToTest = new MyExampleController(this.mockUOW, this.mock2, this.mock3);
}
Now you (hopefully) have only a single place in your tests to update when dependencies get added to one of your controllers.
Note also that that the CreateController method isn't named CreateMyExampleController. As a best practice, I try to avoid method names in tests that are specific to certain classes or methods. Here, for example, including the class name in the method adds another refactoring dependency which is easily overlooked.
You just inline the mock initialization. Instead of supplying null just supply new Mock<IClassName>.Object().
You can try using automock (on codeplex) to automatically mock out your top-level objects, this reduces the amount of retyping that you need to fix objects.
Tests should not share the same context - you should be trying to test a stateless system. For the most part without refactoring tools enabled you're just going to have to deal with it.
Resharper is very powerful in this regard. When you add a new class, you can Ctrl+F6 and add a new parameter to a constructor. It will prompt you with how to fill caller locations - just type your mock in there and everywhere that it matters (if you are injecting all your dependencies the right way) will be filled in automatically.
If your DI container supports multiple instances (i.e. Unity does) you can use local container to resolve all references instead of calling constructor.
Pseudocode:
var container = new MyDependencyContainer();
container.RegisterInstance<IMyInterface>(mockedMyInterface);
var myClass = container.Reslove(myClass);
To simplify code in each test you can have container/mocks as field and configure container in method marked with [TestInitialize] attribute.
Note that such code will not break at compile time when you add dependency to constructor, but likely still require changes (i.e. adding new dependencies to container).
I currently have a repository that is using Entity Framework for my CRUD operations.
This is injected into my service that needs to use this repo.
Using AutoMapper, I project the entity Model onto a Poco model and the poco gets returned by the service.
If my objects have multiple properties, what is a correct way to set-up and then assert my properties?
If my service has multiple repo dependencies what is the correct way to setup all my mocks? * - A class [setup] where all the mocks and objects are configured for these test fixtures?*****
I want to avoid having 10 tests and each test has 50 asserts on properties and dozens on mocks set-up for each test. This makes maintainability and readability difficult.
I have read Art of Unit Testing and did not discover any suggestions how to handle this case.
The tooling I am using is Rhino Mocks and NUnit.
I also found this on SO but it doesn't answer my question: Correctly Unit Test Service / Repository Interaction
Here is a sample that expresses what I am describing:
public void Save_ReturnSavedDocument()
{
//Simulate DB object
var repoResult = new EntityModel.Document()
{
DocumentId = 2,
Message = "TestMessage1",
Name = "Name1",
Email = "Email1",
Comment = "Comment1"
};
//Create mocks of Repo Methods - Might have many dependencies
var documentRepository = MockRepository.GenerateStub<IDocumentRepository>();
documentRepository.Stub(m => m.Get()).IgnoreArguments().Return(new List<EntityModel.Document>()
{
repoResult
}.AsQueryable());
documentRepository.Stub(a => a.Save(null, null)).IgnoreArguments().Return(repoResult);
//instantiate service and inject repo
var documentService = new DocumentService(documentRepository);
var savedDocument = documentService.Save(new Models.Document()
{
ID = 0,
DocumentTypeId = 1,
Message = "TestMessage1"
});
//Assert that properties are correctly mapped after save
Assert.AreEqual(repoResult.Message, savedDocument.Message);
Assert.AreEqual(repoResult.DocumentId, savedDocument.DocumentId);
Assert.AreEqual(repoResult.Name, savedDocument.Name);
Assert.AreEqual(repoResult.Email, savedDocument.Email);
Assert.AreEqual(repoResult.Comment, savedDocument.Comment);
//Many More properties here
}
First of all, each test should only have one assertion (unless the other validates the real one) e.q. if you want to assert that all elements of a list are distinct, you may want to assert first that the list is not empty. Otherwise you may get a false positive. In other cases there should only be one assert for each test. Why? If the test fails, it's name tells you exactly what is wrong. If you have multiple asserts and the first one fails you don't know if the rest was ok. All you know than is "something went wrong".
You say you don't want to setup all mocks/stubs in 10 tests. This is why most frameworks offer you a Setup method which runs before each test. This is where you can put most of your mocks configuration in one place and reuse it. In NUnit you just create a method and decorate it with a [SetUp] attribute.
If you want to test a method with different values of a parameter you can use NUnit's [TestCase] attributes. This is very elegant and you don't have to create multiple identical tests.
Now lets talk about the useful tools.
AutoFixture this is an amazing and very powerful tool that allows you to create an object of a class which requires multiple dependencies. It setups the dependencies with dummy mocks automatically, and allows you to manually setup only the ones you need in a particular test. Say you need to create a mock for UnitOfWork which takes 10 repositories as dependencies. In your test you only need to setup one of them. Autofixture allows you to create that UnitOfWork, setup that one particular repository mock (or more if you need). The rest of the dependencies will be set up automatically with dummy mocks. This saves you a huge amount of useless code. It is a little bit like an IOC container for your test.
It can also generate fake objects with random data for you. So e.q. the whole initialization of EntityModel.Document would be just one line
var repoResult = _fixture.Create<EntityModel.Document>();
Especially take a look at:
Create
Freeze
AutoMockCustomization
Here you will find my answer explaining how to use AutoFixture.
SemanticComparison Tutorial This is what will help you to avoid multiple assertions while comparing properties of objects of different types. If the properties have the same names it will to it almost automatically. If not, you can define the mappings. It will also tell you exactly which properties do not match and show their values.
Fluent assertions This just provides you a nicer way to assert stuff.
Instead of
Assert.AreEqual(repoResult.Message, savedDocument.Message);
You can do
repoResult.Message.Should().Be(savedDocument.Message);
To sum up. These tools will help you create your test with much less code and will make them much more readable. It takes time to get to know them well. Especially AutoFixture, but when you do, they become first things you add to your test projects - believe me :). Btw, they are all available from Nuget.
One more tip. If you have problems with testing a class it usually indicates a bad architecture. The solution usually is to extract smaller classes from the problematic class. (Single Responsibility Principal) Than you can easily test the small classes for business logic. And easily test the original class for interactions with them.
Consider using anonymous types:
public void Save_ReturnSavedDocument()
{
// (unmodified code)...
//Assert that properties are correctly mapped after save
Assert.AreEqual(
new
{
repoResult.Message,
repoResult.DocumentId,
repoResult.Name,
repoResult.Email,
repoResult.Comment,
},
new
{
savedDocument.Message,
savedDocument.DocumentId,
savedDocument.Name,
savedDocument.Email,
savedDocument.Comment,
});
}
There is one thing to look-out for: nullable types (eg. int?) and properties that might have slightly different types (float vs double) - but you can workaround this by casting properties to specific types (eg. (int?)repoResult.DocumentId ).
Another option would be to create a custom assert class/method(s).
Basically, the trick is to push as much clutter as you can outside of the unittests, so that only the behaviour
that is to be tested remains.
Some ways to do that:
Don't declare instances of your model/poco classes inside each test, but rather use a static TestData class that
exposes these instances as properties. Usually these instances are useful for more than one test as well.
For added robustness, have the properties on the TestData class create and return a new object instance every time
they're accessed, so that one unittest cannot affect the next by modifying the testdata.
On your testclass, declare a helper method that accepts the (usually mocked) repositories and returns the
system-under-test (or "SUT", i.e. your service). This is mainly useful in situations where configuring the SUT
takes more than 2 or more statements, since it tidies up your test code.
As an alternative to 2, have your testclass expose properties for each of the mocked Repositories, so that you don't need to declare these in your unittests; you can even pre-initialize them with a default behaviour to reduce the configuration per unittest even further.
The helper method that returns the SUT then doesn't take the mocked Repositories as arguments, but rather contstructs the SUT using the properties. You might want to reinitialize each Repository property on each [TestInitialize].
To reduce the clutter for comparing each property of your Poco with the corresponding property on the Model object, declare a helper method on your test class that does this for you (i.e. void AssertPocoEqualsModel(Poco p, Model m)). Again, this removes some clutter and you get the reusability for free.
Or, as an alternative to 4, don't compare all properties in every unittest, but rather test the mapping code in only one place with a separate set of unittests. This has the added benefit that, should the mapping ever include new properties or change
in any other way, you don't have to update 100-odd unittests.
When not testing the property mappings, you should just verify that the SUT returns the correct object instances (i.e. based on Id or Name), and that just the properties that might be changed (by the business logic being currently tested)
contain the correct values (such as Order total).
Personally, I prefer 5 because of its maintainability, but this isn't always possible and then 4 is usually a viable alternative.
Your test code would then look like this (unverified, just for demonstration purposes):
[TestClass]
public class DocumentServiceTest
{
private IDocumentRepository DocumentRepositoryMock { get; set; }
[TestInitialize]
public void Initialize()
{
DocumentRepositoryMock = MockRepository.GenerateStub<IDocumentRepository>();
}
[TestMethod]
public void Save_ReturnSavedDocument()
{
//Arrange
var repoResult = TestData.AcmeDocumentEntity;
DocumentRepositoryMock
.Stub(m => m.Get())
.IgnoreArguments()
.Return(new List<EntityModel.Document>() { repoResult }.AsQueryable());
DocumentRepositoryMock
.Stub(a => a.Save(null, null))
.IgnoreArguments()
.Return(repoResult);
//Act
var documentService = CreateDocumentService();
var savedDocument = documentService.Save(TestData.AcmeDocumentModel);
//Assert that properties are correctly mapped after save
AssertEntityEqualsModel(repoResult, savedDocument);
}
//Helpers
private DocumentService CreateDocumentService()
{
return new DocumentService(DocumentRepositoryMock);
}
private void AssertEntityEqualsModel(EntityModel.Document entityDoc, Models.Document modelDoc)
{
Assert.AreEqual(entityDoc.Message, modelDoc.Message);
Assert.AreEqual(entityDoc.DocumentId, modelDoc.DocumentId);
//...
}
}
public static class TestData
{
public static EntityModel.Document AcmeDocumentEntity
{
get
{
//Note that a new instance is returned on each invocation:
return new EntityModel.Document()
{
DocumentId = 2,
Message = "TestMessage1",
//...
}
};
}
public static Models.Document AcmeDocumentModel
{
get { /* etc. */ }
}
}
In general, if your having a hard time creating a concise test, your testing the wrong thing or the code your testing has to many responsibilities. (In my experience)
In specific, it looks like your testing the wrong thing here. If your repo is using entity framework, your getting the same object back that your sending in. Ef just updates to Id for new objects and any time stamp fields you might have.
Also, if you can't get one of your asserts to fail without a second assert failing, then you don't need one of them. Is it really possible for "name" to come back ok but for "email" to fail? If so, they should be in separate tests.
Finally, trying to do some tdd might help. Comment out all the could in your service.save. Then, write a test that fails. Then un comment out only enough code to make your test pass. Them write your next failing test. Can't write a test that fails? Then your done.
I have a number of controllers that I am testing, each of which has a dependency on a repository. This is how I am supplying the mocked repository in the case of each test fixture:
[SetUp]
public void SetUp()
{
var repository = RepositoryMockHelper.MockRepository();
controller = new HomeController(repository.Object);
}
And here is the MockRepository helper method for good measure:
internal static Mock<IRepository> MockRepository()
{
var repository = new Mock<IRepository>();
var posts = new InMemoryDbSet<Post>()
{
new Post {
...
},
...
};
repository.Setup(db => db.Posts).Returns(posts);
return repository;
}
... = code removed for the sake of brevity.
My intention is to use a new instance of InMemoryDbSet for each test. I thought that using the SetUp attribute would achieve this but clearly not.
When I run all of the tests, the results are inconsistent as the tests do not appear to be isolated. One test will for example, remove an element from from the data store and assert that the count has been decremented but according to the whim of the test runner, another test might have incremented the count, causing both tests to fail.
Am I approaching these tests in the correct way? How can I resolve this issue?
The package you reference you are using for your InMemoryDataSet uses a static backing data structure, and so will persist across test runs. This is why you're seeing the inconsistent behaviors. You can get around this by using another package (as you mention), or pass in a new HashSet to the constructor in every test so it doesn't use a static member.
As to the rest of your question, I think you're approaching the testing well. The only thing is that since all of your tests have the same setup (in your SetUp method), they could influence each other. I.e., if you have a test that relies on having no Foo objects in the set and one that needs at least one, then you're either going to add one in SetUp and remove it in the test that doesn't need it, or vice versa. It could be more clear to have specific set up procedures as #BillSambrone mentioned.
As #PatrickQuirk pointed out, I think your problem is due to what InMemoryDbSet does under the covers.
Regarding the "Am I approaching this right ?" part :
If, as I suspect, your Repository exposes some kind of IDbSet, it's probably a leaky abstraction. The contract of an IDbSet is far too specific for what a typical Repository client wants to do with the data. Better to return an IEnumerable or some sort of read-only collection instead.
From what you describe, it seems that consumers of Posts will manipulate it as a read-write collection. This isn't a typical Repository implementation - you'd normally have separate methods such as Get(), Add(), etc. The actual internal data collection is never exposed, which means you can easily stub or mock out just the individual Repository operations you need and not fear the kind of issues you had with your test data.
Whatever is in your [SetUp] method will be called for each test. This is probably behavior that you don't want.
You can either place the code you have in the [SetUp] method inside each individual test, or you can create a separate private method within your unit test class that will spin up a freshly mocked DbSet for you to keep things DRY.