Test-After Development is not Test-Driven Development

Recently, I had a disagreement with a colleague over the correct way to do Test-Driven Development. This is an important disagreement because it affects the design of the ASP.NET MVC framework.

According to my co-worker (let’s call him Tad), there is no difference between Test-First Development and Test-After Development “except when you write your unit tests.” Tad is a practitioner and proponent of Test-After Development. When you practice Test-After Development, you write application code first and then you write a unit test that tests the application code.

From the perspective of someone who practices Test-Driven Development, this gets things backwards. I believe that it is an essential part of Test-Driven Development that you must write your unit test before writing any application code. Why does it matter?

Test-Driven Development is first and foremost an application design methodology. If you write your unit tests after you write your application code, then you are not driving the design of your application with your unit tests. In other words, Test-After Development ignores the Driven in Test-Driven Development.

In order to support Test-Driven Development, the ASP.NET MVC framework needs to support two things: testability and incremental design (what Martin Fowler calls Evolutionary Design). If you are only interested in Test-After Development, then you will ignore this second requirement to the detriment of those of us who are interested in true Test-Driven Development.

Let’s consider a concrete scenario: building a forums application.

Building a Forums Application with Test-Driven Development

Here are the steps that I would follow to build a forums application by using Test-Driven Development:

1. Write a list of user stories that describe what the forums application should do. These user stories should be non-technical (the type of thing that a customer would write).

2. Pick a user story and express the user story in a unit test.

3. Write just enough code to pass the unit test. In other words, do the simplest thing that could possibly work to pass the unit test.

4. Consider refactoring my code to improve the design of my application. I can fearlessly refactor because my code is covered by unit tests (see RefactorMercilessly).

5. Repeat steps 2 – 3 until I have completed the application (keeping in mind that the user stories might change over the course of the process of writing the application).

So, I might start with a list of user stories that look like this:

1. Can see all of the forum posts

2. Can create a new forum post

3. Can reply to a forum post

And, I would express the requirement embodied in the first user story with a unit test that looks like this:

[TestMethod]
public void CanListForumPosts()
{
    // Arrange
    var controller = new ForumController();

    // Act
    var result = (ViewResult)controller.Index();

    // Assert
    var forumPosts = (ICollection)result.ViewData.Model;
    CollectionAssert.AllItemsAreInstancesOfType(forumPosts, typeof(ForumPost));
}

This unit test verifies that invoking the Index() action on the Forum controller class returns a collection of forum posts. Currently, this unit test fails (I can’t even compile it) because I have not created a ForumController or ForumPost class.

Following good Test-Driven Development design methodology, at this point, I am only allowed to write enough code to make this unit test pass. And, I should make the test pass in the easiest and simplest way possible (I’m not allowed to go off and write a massive forums library however tempting that might be).

To make this test pass, I need to create a ForumsController class and a ForumPost class. Here’s the code for the ForumsController class:

using System.Collections.Generic;
using System.Web.Mvc;
using Forums.Models;

namespace Forums.Controllers
{
    public class ForumController : Controller
    {
        //
        // GET: /Forum/

        public ActionResult Index()
        {
            var forumPosts = new List<ForumPost>();
            return View(forumPosts);
        }

    }
}

Notice how simple the Index() method is. The Index() method simply creates a collection of forum posts and returns it.

From the perspective of good software design, this controller is horrible. I’m mixing responsibilities. My Data access code should go in a separate class. And, even worse, this controller doesn’t actually do anything useful at the moment.

However, from the perspective of Test-Driven Development, this is exactly the right way to initially create the Forums controller. Test-Driven Development enforces incremental design. I am only allowed to write enough code to pass my unit tests.

Test-Driven Development forces developers to focus on writing the code that they need right now instead of writing code that they might need in the future. Two of the important guiding principles behind Test-Driven Development are "Keep It Simple, Stupid" (KISS) and "You Ain't Gonna Need It" (YAGNI) (see Wikipedia and C2).

Eventually, after repeating the cycle of writing a unit test and writing just enough code to pass the test, you will start to notice duplication in your code. At that point, you can refactor your code to improve the design of your code. You will be able to refactor your code fearlessly because your code is covered by unit tests.

The important point here is that the design of your application should be driven by your unit tests. You don’t start with design principles and create an application. Instead, you incrementally improve the design of your application after each cycle of test and code.

Building a Forums Application with Test-After Development

A proponent of Test-After Development takes a very different approach to the process of building an application. Someone who practices Test-After Development starts by writing application code and then writes a unit test after the application code is written. More to the point, a proponent of Test-After Development makes all of their design decisions up front.

The crucial difference between Test-Driven Development and Test-After Development is a difference in belief about the importance of incremental design. Practitioners of Test-Driven Development take baby steps in improving the design of an application. Practitioners of Test-After Development attempt to implement good design from the very start.

Here are the steps that a practitioner of Test-After Development would take to building a forums application:

1. Create a list of user stories.

2. Consider the best design for the application (create separate controller and repository classes).

3. Write application code that follows the design.

4. Write unit tests for the code.

5. Repeat steps 2 – 4 until the forums application is completed.

Unlike someone who practices Test-Driven Development, a proponent of Test-After Development would start by creating separate Forums controller and repository classes.

For example, the Forums controller would look like this:

using System.Web.Mvc;
using TADApp.Models;

namespace TADApp.Controllers
{
    public class ForumsController : Controller
    {
        private IForumsRepository _repository;

        public ForumsController()
            :this(new ForumsRepository()){}

        public ForumsController(IForumsRepository repository)
        {
            _repository = repository;
        }


        public ActionResult Index()
        {
            var forumPosts = _repository.ListForumPosts();
            return View(forumPosts);
        }

    }
}

And, the repository class would look like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace TADApp.Models
{
    public interface IForumsRepository
    {
        IEnumerable<ForumPost> ListForumPosts();

    }

    public class ForumsRepository : IForumsRepository
    {

        private ForumsDBEntities _entities = new ForumsDBEntities(); 

        #region IForumsRepository Members

        public IEnumerable<ForumPost> ListForumPosts()
        {
            return _entities.ForumPostSet.ToList();
        }

        #endregion
    }
}

Next, the proponent of Test-After Development would create a unit test for the Forums controller that looks like this:

[TestMethod]
public void CanListForumPosts()
{
    // Arrange
    var mockRepository = new Mock<IForumsRepository>();
    mockRepository.Expect(r => r.ListForumPosts()).Returns(new List<ForumPost>());
    var controller = new ForumsController(mockRepository.Object);

    // Act
    var result = (ViewResult)controller.Index();

    // Assert
    var forumPosts = (ICollection)result.ViewData.Model;
    CollectionAssert.AllItemsAreInstancesOfType(forumPosts, typeof(ForumPost));
}

This unit test mocks the Forums repository (by mocking the IForumsRepository interface) and verifies that the Forums controller returns a set of forum posts.

Unit Tests versus TDD Tests

One place where the proponent of Test-Driven Development and the proponent of Test-After Development strongly differ is on the subject of unit tests. I disagree with Tad about the purpose -- and the correct way to write -- unit tests.

When I practice Test-Driven Development, I start with a test and then I write just enough code to pass the test. I use the tests as a safety net for change. In particular, I use the tests as a safety net so I can fearlessly refactor my application code to improve the design of my application.

When using Test-Driven Development to create the forums application, my first test verified that the Forums controller returns a list of forum posts. I would keep that test even after I refactor the design of my application to migrate my data access logic into a separate repository class. I need the original test to verify that I haven’t broken my original application code when refactoring my application to have a better design.

My unit tests flow directly from the user stories. After I add a unit test, I almost never remove it. I might refactor my unit tests to prevent code duplication in my tests. However, I don’t change what the unit tests are tests for.

A proponent of Test-After Development, in contrast, is constantly changing their tests. When Tad rewrites his application logic, Tad rewrites his unit tests. Tad’s unit tests are driven by his application design.

From the very beginning, Tad would create a separate Forums controller class and repository class. He would create distinct sets of unit tests for the Forums controller and the repository class. When Tad refactors his application to improve the design of his application, Tad rewrites his unit tests.

Suppose, for example, that both Tad and I decided to add support for validation to the Forums application. If someone submits a forum post with an empty Title, we both want to display a validation error message.

I would take the approach of testing whether or not the forums controller returns a validation error message in ModelState when I attempt to create an invalid forum post. My unit test would look something like this:

[TestMethod]
public void ForumPostSubjectIsRequired()
{
    // Arrange
    var controller = new ForumController();
    var postToCreate = new ForumPost { Subject = string.Empty };

    // Act
    var result = (ViewResult)controller.Create(postToCreate);

    // Assert
    var subjectError = result.ViewData.ModelState["Subject"].Errors[0];
    Assert.AreEqual("Subject is required!", subjectError.ErrorMessage);
}

This test verifies that a validation error message is included in model state when you attempt to create a new forum post without supplying a subject. Regardless of how I end up refactoring my application (for example, to use a separate validation service layer), I would keep this unit test to verify that my application continues to satisfy the requirement expressed by the user story.

Tad, on the other hand, would never create a test that verifies whether or not the Forums controller returns a validation error message. Tad would argue that it is not the responsibility of a controller to perform validation. The responsibility of a controller is to control application flow.

Tad would write a unit test for his validation logic. However, the nature of his unit tests would be dependent on the architectural design of his application. If Tad uses a separate service layer to contain his validation logic, then he would write unit tests that verify the behavior of the service layer. If Tad uses validator attributes to perform validation, then he would write unit tests that verify the presence of the expected validator attributes.

Tad would argue that my unit tests aren’t really unit tests at all. Over time, as the design of my application evolves, my unit tests start to resemble functional (or acceptance) tests. They are really verifying the outputs of the application given a certain input. My unit tests are independent of the application design.

I would agree with Tad, but I would argue that the tests that you write when performing Test-Driven Development have a different purpose than standard unit tests. A TDD test, in contrast to a unit test, does not necessarily test a separate unit of code. Instead, a TDD test is used to test “little areas of responsibility, and that could be a part of a class or it could be several classes together” (see Martin Fowler).

This is not to say that a TDD test is the same as an acceptance test. An acceptance test is used to test an application end-to-end (with the database and UI hooked up). A TDD test, on the other hand, is not an end-to-end test. A TDD test does not have external dependencies and it is designed to be executed very fast. A TDD test is used to test whether a particular requirement derived from a user story has been satisfied (see Uncle Bob).

From the perspective of Test-Driven Development, the purpose of unit tests is to drive the design of an application. A unit test tells me what application code I need to write next. For example, I don’t know how I will implement my validation logic when I create a unit test. I should not be making these design decisions up front. My test tells me what I am allowed to do and what I must do. The unit test provides me with the minimum and maximum criterion for success.

My primary objection to Tad’s approach to building applications is that it forces premature design decisions. Tad makes design decisions first and then creates his unit tests. I create my unit tests first and then create my design (see Jeff Langr). Tad’s approach does not allow for an Evolutionary Approach to design.

Conclusion

So why should any of this matter? The ASP.NET MVC framework was designed to be highly testable. Therefore, it should keep proponents of both Test-Driven Development and Test-After Development happy. Right?

The point of this blog entry is to claim that the ASP.NET MVC framework needs to support more than testability to support Test-Driven Development. To enable Test-Driven Development, the ASP.NET MVC framework was designed to support both testability and incremental design. From the perspective of a practitioner of Test-Driven Development, if a framework does not enable you to get from point A to point B by taking baby design steps, then there is no way to get to a well designed application at all.

If you liked this blog post then please Subscribe to this blog.
posted on Wednesday, April 08, 2009 12:52 PM | Filed Under [ ASP.NET ASP.NET MVC TDD ]

Comments

Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Mr Anonymous
on 4/8/2009 3:48 PM

Right in time. Just what i needed. :)
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Rob Conery
on 4/8/2009 5:22 PM

A good distinction between TAD and TDD - but I fail to see how MVC makes a choice here. From what I can tell, Phil and team devoted a massive amount of time to keeping this framework as testable as possible.

In terms of incremental design - you bring up validators:

>>> If someone submits a forum post with an empty Title, we both want to display a validation error message.

Good test. You then bring how you would test it:

>>>I would take the approach of testing whether or not the forums controller returns a validation error message in ModelState when I attempt to create an invalid forum post

... and that's not a good Unit Test. You're not actually testing that a forum post cannot have an empty title - you're testing whether the Controller and ModelState work.

Taking your query literally:

>>> If someone submits a forum post with an empty Title, we both want to display a validation error message

This is *not* the purview of Unit Testing, nor does it have anything to do with incremental design. This is something that's a bit tough to deal with (TAD or TDD) in ASP.NET but you can do it with a tool like WatiN or Watir.

To me the core of the issue isn't that ASP.NET MVC "doesn't support incremental design" - it's what you choose to incrementally design. In face I'll go so far as to say "what does ASP.NET MVC have to do with your unit tests right now anyway"?

Design your model - focus on your validations in whichever layer you choose. You can crank out ViewModel's even! And when you're ready - layer on the web app. All your testable stuff is handled and you won't need to worry about logic in the Controller or ModelState because, as you very well mention - they don't belong there anyway :).
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Stephen Walther
on 4/8/2009 5:43 PM

Hi Rob,

No one is suggesting that the MVC framework is not highly testable - that is why we all love it. The point of this blog post is to challenge Tad's assumption that there is no difference between test-driven development and Test-After Development except when you write your tests. Also, I believe that you should unit test any application functionality that could break (unless testing would take too long)
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Jack
on 4/8/2009 6:12 PM

Well, to be honest, I'm the one of the 'Test-After'. Thanks for your article, I'll come to the 'Test Driven'!
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Ben Scheirman
on 4/8/2009 6:25 PM

Test-after folks commonly make this mistake when comparing TDD/TAD. You hit the nail on the head, it's about driving design. However in your post you are writing code test *exactly* as they would look if you were doing TAD.

If you were doing TDD, you probably wouldn't have any controllers yet. You'd start with an entity. Then a repository. Then you might introduce another concept, such as validation.

Tying these to a UI is another aspect.

Good unit tests test things in isolation. This is why testing that your controller puts stuff in ModelState is not such a great idea. Sure we want to make sure this happens eventually, but this is not the stuff from which you drive good design.

Don't get me wrong, I applaud your effort in bring TDD to more widespread understanding and adoption, but this post seems to just be proving the TAD folks point.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Chad Myers
on 4/8/2009 7:28 PM

Stephen, thanks for the post. The TDD/TAD distinction can be nuanced at first, but once people get the hang of TDD, it yields far greater benefits beyond TAD. TAD is just about testing. TDD is testing, too, but also design, analysis, examination of design choices, experience from an API consumer perspective, and so much more.

Rob and Ben both bring up good points and it may seem like nitpicking. Both are experienced practitioners and have found flaws with your approach. Please do not take their criticisms as attacks, but rather them seeing you as someone they can have a higher level conversation with. Basically, you've reached Level 1, now let's talk about Level 2 and higher!

Keep up the effort and don't stop learning. I believe TDD really can make anyone a better developer, but it requires some discipline and rigor which is hard to learn on one's own without an experienced practitioner to help someone through the tough spots. It's hard to teach these practices via a blog post. Live evens and one-on-one pairing sessions are usually the most effective.

The trick is... how do you capture and share that with other people en masse? That's where we have the hardest time.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by PK
on 4/8/2009 8:05 PM

Intresting Post Stephen :) Looking forward to seeing all the responses and then blog posts that spawn from this.

@Ben: I'm not so sure i agree with everything in your reply. I'm 150% behind the 'driving design' comment, so we can put that aside. I want to talk about your suggestion that controllers shouldn't be handled yet, but entities and repository.

AFAIK, Stephen is engineering an asp.net mvc _website_. So far he hasn't really thought about his persistence (aka repository/ies). Nor how this will be modelled. He's thinking about a website. So he's generated his stories / scenario's and picks the first one and starts with that. Ok, lets show a list of posts. and off he goes, baby steps to get the first unit test going, driven by his design of his end goal -> a website.

Now I'm not necessarily saying that's the best or right way to do something. I'm still a bit undecided personally, on what to 'start with'. But I can understand his journey based upon his personal design-direction.

Traditionally, I would create the stories and scenarios, then delve deeper and say how this is going to be model'd and then even look at the persistence of this. Even more common or traditional is how people start with database design BEFORE any website code or design. I personally feel this way is flawed .. and I too used to do this years ago - I'm guessing a majority of us might have.

For myself, i'm still not sure what is the best approach to start with. Like, i have my website stories and scenarios. ie. my finite domain. Now do i start to model this domain? Do i go as far as modelling a repository, even if it's just a fake test repo (ie. in memory lists)? Or is the 'better/more practical' approach to wait to do that stuff a bit later and get to there, _when u need it_. So, if i was going to have a resource (i nearly said 'page' but that is sooooo 90's/early 00's/web forms and we've evolved beyond that) that lists blog posts, lets start with that -> and a controller is the key to that, right? We're trying to list blog posts on an asp.net mvc app. scenario is /forum/index or /forum/list . I don't see anything about a database or how posts are modelled.

I'm just trying to understand all this stuff - not personally attack you or anyone else, so please don't misread my words as anything mean or rude .. if anything it's the complete opposite. :)

At least we're all starting to think about unit testing, even if it's before or after .. at least it's part of the equation.

Thank. Gawd. For. ASP.NET MVC.

-PK-
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Todd
on 4/8/2009 8:59 PM

Ah, thanks for the clarification of the term "unit test". In our safety-critical software environment we call your TDD unit tests, "requirement-based tests" (RBT). These tests verify the required *functionality* or behavior of the app/function. DO-178B is the gov't standard that spells all this out as well as what integration test are and what they need to test.

I have struggled with my push for TDD in my division as when one hears one must write unit tests first, their jaws drop and give me the "are you insane" glare. To them, unit test are super, low-level test verifying the rigor and robustness of the implementation. Writing these types of tests first would me absurd...as you point out in your posting under the Tad scenario. As you point out, the big value is in thewriting of RBT ahead of code to give you the safety net and drive design.

Thanks again.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Gregory A. Beamer
on 4/8/2009 9:23 PM

I would love it if people would just have tests period so we had the opportunity to argue TAD versus TDD. ;-)

I am definitely on the side of TDD, especially if you have a good set of use cases to start with. TDD forces you to think of the behaviors before you actually code them. I am becomming a bit more a fan of BDD, as well, although I do not have a BDD framework right now to truly take it the full nine yards.

As for TAD. When combined with proper refactoring, you can end up with the same endpoint as TDD. TDD, at least in every case I can think of, will get you there faster, but the person who states "the only difference is when the tests are written" may have a point if he is, in fact, refactoring to a proper design. Note, however, that the endpoint is the same ... not the journey ... nor the time spent getting there.

Great article.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by rickj1
on 4/8/2009 9:26 PM

I've been playing with MVC I have Vs 2008 standard edition I installed XUNIT for testing
it's a nightmare are there any tutorials on how to make it work or on how to work with it in MVC
I keep getting this error
Error 1 The type or namespace name 'HttpContextBase' could not be found (are you missing a using directive or an assembly reference?)\Projects\NerdDinner\NerdDinner.Tests\Routes\RouteFacts.cs 101 29 NerdDinner.Tests
I dont think any of the learning material I've
came across is much use if you cant do the tests every thing seems to center around TDD
what would the work around be It would be nice to use VS for testing and not some open source bug infested softwear










Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Rob Conery
on 4/8/2009 10:18 PM

@PK where you're headed is something that DDD tries to tackle a bit (in my very small knowledge of it). FIrst - I agree that we overthink things. It's the tool - it lets us :).

At the same time in terms of "where to start" - start small! Know that you can change course - rename your tests - expand - whatever! The only thing you should "know" when doing TDD is what's in front of you in the form of requirements.

So - in Stephen's case - he laid out a set of User Stories that are interaction-focused:

1. Can see all of the forum posts
2. Can create a new forum post
3. Can reply to a forum post

The thing about these is that they don't describe the behavior of the application (more to the point - the "title is required" behavior under test isn't part of the story here... ). In that I think we have the conceptual break. What might be a better place to start (in getting to your question) is pushing your client to understand the behavior of the thing you're trying to build:

1) The Forum consists of Posts, written by Users who want to ask each other questions.
2) A Post has a Title and Date which must be filled out, and an Author with an Email that are also required.
3) Duplicate posts (same body and text) are not allowed
4) Authors cannot post more than 1 post every 4 minutes (to avoid spam).

In this we're focusing on behaviors and the responsibility of creating this list falls on you asking the right questions so you can model the right behaviors. In this small list we probably have mess of tests. Where to get started? Write the first sentence out as a test title, knowing you will change it.

Ideally you can keep the reviews to the conversation level and use something like Balsamiq to keep your client happy WRT visuals. Ideally you pull as much behavioral knowledge out and test it - with a growing Model that does what you client wants it to do (sans visuals). Once that's done - plugging in the UI is trivial.

PS - it's fine to think of your app from the outside-in. I pushed this idea in my MIX 09 talk (thinking like a scripter). I believe in this since you're trying to deliver an experience and not battleship gray, lifeless forms. That said - there is a point when you start that, and in my mind it's after you work up your model :).
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Dinia Oussama
on 4/9/2009 2:03 AM

Hello Stephen,

thanks for the post on this issue, which is very important in TDD projects.

I have had the chance to work with TDD. We had a team of developers who are new to the .NET Framework.
To explain to them to write the test before the code, was really wiered to them.

Here's how we introduced TDD to the team (critics are more than welcome):
Instead, we actually explained to them, to think of a test as a way to conceive the method, class or function we create.
It helps us think about how we want to use our code, it also helps us understand the code we write, and avoid too much untested copy and paste.

With this in mind, some team members started calling, asking for help on C#, but the nice thing, is they would ask for help on the unit test first, as they see, that, every technical lead and project manager, is in favor of this method.

One drawback though. We had an issue using Libraries like Entreprise Library from Microsoft. Alhtough Ent. Lib. is United Tested, team some team members would say it's too complex to use such a library with TDD.

In my opinion, writing and testing a data access layer, requires a lot of care, as simple as it might seem, consequences can be tuff, as it is a heavily used layer.

To stay away from conflicts, we kind of cheated. I took an open source data access library for .net 2.0 :), I had used it in many projects, wrote unit tests for the main methods we need, and started our Data Access layer based on the code we used. Everyone was happy, but no one actually asked me where the code comes from, because I understood it and explained it well to them. Then I actually mentionned where the code came from.
What about the cost, concerning the projet, and the timeline ?

Do we have to rewrite everytime, a data access layer, it will definitely push us towards copy and paste ?
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Gabriel Perez
on 4/9/2009 5:25 AM

Stephen, thanks for this post. It allows for a great discussion.
I'm a big believer in the incremental design and you are quite right when you say that TDD simplifies the process. I having done test after several times I had indeed the need to go back to the tests to change them in order to comply with changes in the architecture. If I had done higher level tests as you propose this wouldn't have happened.
Having said that I still find very hard to push TDD in a lot of companies, the initial time it takes to right the tests often scares managers as the can't see that this time will be far smaller than the time developers are going to spend of fixing bugs. What I try to do is convince people to do any amount of testing (even if it's TAD) in order for them to see the benefits of unit testing. I don't think that TAD is completely wrong, it's just a different thing than TDD.
I do agree with you that the MVC framework allow for TDD because that is where many people are heading, and if you are going to do TDD you should do it the right way or don't call it TDD at all :-)
Phil and the team are doing a great job and this kind of discussion only adds to the process. I can't tell you how good it is to see that you guys are having these kind of discussions at Microsoft. It shows how much you care about the future of MVC.
Good job guys!
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Stephen.Walther
on 4/9/2009 8:50 AM

@rick1 -- It sounds like you are missing the reference to the System.Web.Abstractions.dll in your unit test project. Select the Test project, select the menu option Project, Add Reference and add the reference from under the .NET tab. Make sure that you have downloaded and installed Service Pack 1 and the ASP.NET MVC framework. You can download the ASP.NET MVC framework from www.ASP.net/mvc.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Ryan McIlmoyl
on 4/9/2009 8:54 AM

I'm still working my way through what proper TDD is, but it seems to me the test from your example doesn't really satisfy the requirement. The requirement is that a user can list all the forum posts, yet you are only testing that a collection of items of the correct type are returned. You are not testing that, in fact, all of the forum posts are listed (Your test would pass, even if the collection was always returned as empty). To truly test this requirement, you would need some sort of pre-condition (ie. a know list of posts), and then verify the returned collection contained this list. At least, this is my understanding of how TDD should function.

Also, you mention that the TAD approach requires rewriting the tests as the application design changes. However, wouldn't you need to refactor the existing test as your design evolved (ie. when you added a repository to your application design, wouldn't you want to mock that repository in your existing test)? This is just my understanding of TDD and Unit Testing so far, someone please correct me if I'm way off ;)
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Stephen.Walther
on 4/9/2009 9:02 AM

@Ryan McIlmoyl -- The goal is to create one or more tests that express the intention behind the user story. I would start with the test that checks whether a collection of forum posts was returned. But, this single test might not capture everything that you intended with the original story of being able to see all of the forum posts. For example, you might flesh out the story by saying that the posts should be returned in a certain order, or that you should get at least one post back, and so on. In that case, you would write additional tests to capture these more refined intentions.

Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Ryan McIlmoyl
on 4/9/2009 9:49 AM

@Stephen
Thanks for clarifying that one. I think one of the problems in understanding TDD is that the examples used are often trivial and simple, making it hard to relate to the actual problems we as developers are trying to solve. (Although I understand the point of your post was not an in-depth description of what TDD is, but rather to contrast with the TAD approach).

Another issue I've seen with TAD is the same issue we often see with optimizations. They all get put on the pile of stuff to do 'after', and then never get done.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by rickj1
on 4/9/2009 10:30 AM

@Stephen
Thanks bingo that got rid of all the errors how simple I'm going through your sample chapters of MVC Unleashed when I have spare time I use ASP.NET 3.5 Unleashed every step of the way Great book I couldn't live without it
I'm realy looking forward to this new book
Thanks for the great work you guys are doing
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Dave T.
on 4/9/2009 1:05 PM

The pay now or pay later mindset of the TDD folks is rediculous. I know its passe at this point but business people prefer RAD with TAD over TDD. They want to be able to touch and feel something as soon as possible, so they can start making changes. I guess its just the industry you are in. Shooting Rockets into space - TDD. Writing a Business app with constantly changing requirements - TDD costs too much up front.

Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Kyle Nunery
on 4/9/2009 2:55 PM

The community needs to distinguish between Test First and Test After development better. In fact, I think TDD should stand for Test Driven Design. The power of Test First development is that a lot of coding decisions that are somewhat arbitrary become meaningful, hence your design is affected by your tests. For example, I could make a static helper class or use a fancy design pattern like the Command pattern to do the same thing. Which is better? From a testing viewpoint, it is more difficult to unit test static classes, so I go with the the code that makes my testing easier or even doable in some cases. This is a very contrived example, but the gist is that my code is written to be testable from the get go which produces a much different product than throwing tests on at the end. Test first development forces thought about the dependencies in your code to a higher level.

Test first and test after development isn't slower in the sense that you can rework your code at a much faster rate and it's in general better written. I wish the stigma that this process is slower would be quantified with some real data.

Rob Conery makes an excellent point and I kind of take it as people define unit tests differently and you have to test what makes sense for your app and your development process.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by EricTN
on 4/9/2009 4:21 PM

Thank you for another fine post. I'm very interested in the TDD approach and I think the comments to this post are a great conversation about it. I'd love to see more webcasts where various practitioners of this art (TDD) show their methodologies; quite a mind-opener. I want to design my own projects this way, but I must say it's close to impossible to do TDD in the office environments I've worked in. The leaders/managers will be like, "we need to encorporate these various external libraries and API's, and this other team has these services we need to use, and the UI has to completely change like so & so, and there's this stack of action items we need to plow through, and it needs to be done by end-of-month because so & so has already booked a flight to Florida to show the most important client how great it all is"... in the face of that, I'm not at liberty to open up Visual Studio and start typing [TEST] WhenUserTypesValueIntoCityComboBoxAndValueIsNotFoundInListAppropriateCommandsAndActionsBecomeDisabledAndEventFromIncomingSubscribedServiceIsBlockedAndNotAllowedToDisplayModalDialog(). I friggin' swallow more Mountain Dew and start slamming code down, then turn around, become the user for a few minutes while another developer invokes the external service, hit Ctrl-Shift-Alt-Elbow plus a bunch of additional keystrokes and see if I can break the application. (No, I don't believe that's an appropriate methodology, merely a way I've been able to stay employed in various scenarios I've found myself in.)
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Bob Saggett
on 4/10/2009 1:30 AM

I dislike these types of posts. They come across as evangelical rather than logical. There is a whole lot of hype around TDD at the moment but remember that it is just the current fad at not the magic bullet that everyone wants it to be. There will be another fad around the corner and another and another ad nauseum.

Personally I use TDD for some projects and for others I design up-front, write the code and add the tests. There are benefits to both approaches and there are drawbacks. You are correct that TDD drives the design and this is probably the most important part of TDD. However, sometimes the YAGNI principle that tends to go hand-in-hand with TDD (yes I know it is a generally good principle elsewhere too) causes the TDD practioner to go down the wrong path commercially. Remember too that you don't necessarily need TDD to drive design. You could perhaps... just do some design.

I worry that with many developers who don't fully understand the principles, you end up with worse quality code or with people stuck in the refactoring loop and becoming inefficient, ineefective and unproductive. Too often I have seen developers spend ten days developing a simple module that should have taken one day without TDD. I have been guilty of this myself having been trapped in the "must have 100% coverage" mentality. I have also seen people struggle for an hour to test a piece of code that took a minute to write and that they *know* works because it is incredibly simple but awkward to test because of the limitations of the testing tools.

I guess we are just using a fancy new hammer to bash in that screw.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Dan Sylvester
on 4/10/2009 7:10 AM

I'm really trying to get on this TDD boat somewhere.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Khaja Minhajuddin
on 4/10/2009 1:40 PM

Thanks for the great post, Steve.
Here is my 2 cents: I have been reading about TDD for more than 6 months. And whenever people say the most important thing in TDD is the *Driven* part, I used to say to myself ah.. whatever, I get it. But, I have started *practicing* TDD for a couple of days now and it seems somebody struck me with a sledge hammer, listening to other people talking about TDD just doesn't cut it, You really have to *experience* it. For anybody who really thinks they *get* TDD and yet don't practice it I suggest you actually practice TDD for a week, that's when you'll actually *get* it. And thanks for the good work Steve.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Bayram Çelik
on 4/10/2009 4:31 PM

@Bob Saggett
Totally agree with you.

@Stephen
Great article, thanks for pointing out the difference.

But as Bob said TDD is not the holy grail. Just a good pattern for the moment
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Gustavo Martinez
on 4/10/2009 6:54 PM

First of all,Stephen, thank you for your post. I go along with you on TAD is not TDD.

In fact, I'd go further. As an idea for the future, do you think that it could be possible to create a mechanism to force to develop a test before the application? A silly example, not being able to build the solution if there's no at least one test associated with every public method.

In my view, the point is that everybody know how to write code directly, and you can see the results immediately. But, writing tests before developing is annoying.

Therefore, Why not to force us to write the test before?
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Yiyi Sun
on 4/10/2009 10:09 PM

Thanks Stephen for the interesting comparison of TDD and TAD. I found it is very useful information to me.

I want to point out a mistake in your test case for TAD. Under //Assert section, there is no meaning to assert if the Forums controller returned a list of post, because you just did returned an (empty) list on line 6. Instead, you may want to verify the mock's expectations, mockRepository.VerifyAll(). Therefore, the TAD test case becomes meaningful. It actually confirms that controller.Index() should call into repository.ListForumPosts()). This is a white-box test that verifies behavior. On the other hand, the TDD test case is pretty much a black-box, which verifies the state only.

Will you share your thoughts on white-box vs black-box unit test?
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Colin Jack
on 4/13/2009 3:11 AM

Regardless of anything else it is funny that you disabled comment on the subsequent post.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Gregory A. Beamer
on 4/14/2009 6:50 AM

Stephen:

Thanks for the comment. I apologize if it sounds like I was criticizing your post here, as that was not the intent. I was trying to build on the idea based on a comment someone had made to me about your article, which I viewed as a misunderstanding of your point.

I really do see the industry obsession with UI as a primary reason for apps being tightly coupled with little Separation of Concern. I also see the focus on testing MVC from the UI down as an impedement to good design in many cases.

Continue to post on your TDD efforts in MVC. I believe it helps get the focus on proper application design.

Peace and Grace,
Greg
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Stephen.Walther
on 4/14/2009 9:19 AM

@Colin - Disabling comments on the other post was a difficult decision. I found the comments interesting and valuable. However, the last few comments had very little to do with the blog post and I didn't want the conversation to degenerate. My plan is to re-enable the original comments in a few days.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Daniel Root
on 4/16/2009 8:02 AM

I'm not sure I agree that a TDD approach means you won't have to rewrite tests if your application logic changes. I'm currently trying to keep a TDD approach going developing an MVC app, and have had to scrap tons of tests frequently due to both application logic changes and design/refactoring changes. This could well mean I'm not doing TDD correctly- I do suspect I'm 'cutting corners' when it comes to incremental design. For example, I write the test first, watch it fail, but then 'know' I'll need to use a repository and just go ahead with that approach to pass the test.

So I suspect my coverage isn't great, and tests aren't 100% kosher. But even with this number of tests I find myself worrying about refactoring, renaming, etc. "because I don't want to have to rewrite the tests." I usually go ahead with it anyway, but it is something I've noticed...
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Shiju Varghese
on 4/17/2009 12:06 AM

Nice post and good comments.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by MrDustpan
on 4/20/2009 1:24 PM

@Stephen:

Great post as usual!

I think I'm on board with the benefits of using the test first TDD approach. But what about when you begin to refactor, would you recommend writing tests before doing that?

For example: In the controller there is a Create() method that adds a post. I know this works because I have a test that covers this controller method, so with warm and fuzzies I begin to refactor.

I decide I want to move my validation logic into a service class. Would you:

a) Write a new TDD test for my service class method and then implement that method?

b) Simply move the validation logic from my controller class to my service class (no need to write a new test at all because my original test still passes)?

c) Move the validation logic from my controller class to my service class and then write a new unit test to cover the new service method?
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by James Waletzky
on 4/21/2009 5:18 PM

Nice post! It's nice to see people advocating the design benefits of TDD. This is one of the biggest challenges I have at Microsoft - people see TDD as purely a test technique and argue that tests before or after makes no difference. They are wrong, IMHO.

I would modify your steps to TDD a little bit. It is important to write just enough code to make your code compile AND fail the test. It is important to make the test FAIL as this is a test of your test. You want to see the red, in the red-green-refactor step as validation that the code you write is immplementing the right functionality (seeing the green).

Code coverage is actually another important distinction between test-after and test-first. My line (vs. path) coverage with TDD automatically approaches 100% as I never write any code without having corresponding tests. That is harder to achieve with test-after. I can also use coverage feedback (not the number, but covered lines) in a different way. Coverage is now interesting but not mandatory, but with test-after you really need code coverage to measure how well you are testing. Yet another advantage of writing tests first.

I have written a set of TDD-related blog posts that complement yours quite nicely here:
blogs.msdn.com/.../default.aspx
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Sean Feldman
on 4/24/2009 9:59 AM

Thank you for stating the difference!
This is exactly the argue I had with one of the "old school TDD" folks. I care about the design of my code, therefore my specifications are written first, not to have tests as guard dogs to make sure any deviation from the original intent throws exceptions just because an irresponsible team member changes code prior to make observation first.
10x
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Jones
on 4/27/2009 3:08 PM

Helpful post...
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Lloyd Phillips
on 4/30/2009 8:07 PM

Hi Stephen,

Firstly, great post, I'm new to TDD and still ultimately trying to fully comprehend what you should test and when so I hope no one takes my comments too strongly and slates me for it if I'm totally in the wrong. I've started a project and having read your post I've realised I'm heading down the route of TAD. This got me thinking a great deal and compared the arguments and having thought about TAD I'm not overly concerned that I might be going down the wrong path. It seems to me that it comes down to the preference of the developer and the project on where you start testing.
In my project I already know that I want to split out my layers into a repository and service layer. This is where I've started my coding rather from the website which seems to be your preference to start. I don't think there is anything wrong with either my starting point or yours. Please feel free to correct me if you disagree. In my case because I know where the project is heading and is likely to head/expand in the future it seemed good sense to split from the outset and save me some time rather than refactor out further down the line. That said, I'm still trying to take babysteps and create tests on the repository and then the service before testing my mvc controllers. This way I can ensure I have got adequate tests on each unit in each layer and my user story is covered by a test on the mvc controller to test my expected outcome. I read the Fowler white paper where he commented:

'If you want to build a doghouse, you can just get some wood together and get a rough shape. However if you want to build a skyscraper, you can't work that way - it'll just collapse before you even get half way up. So you begin with engineering drawings, done in an engineering office like the one my wife works at in downtown Boston. As she does the design she figures out all the issues, partly by mathematical analysis, but mostly by using building codes. Building codes are rules about how you design structures based on experience of what works (and some underlying math). Once the design is done, then her engineering company can hand the design off to another company that builds it.'

This is what got me thinking it is important to have some plan from the outset as to the design, especially if you have a developer who has a great deal of experience and is likely to see the benefits of using certain design patterns from the outset. It really doesn't matter if you start testing/developing on the service/repository layer as long as tests are in place to capture any future changes that are likely to unwittingly break your application in the future. That said, I think once I have my tests in place it's a good opportunity, and a safety net as you rightly point out, that I can go back and refactor to see if I can better my code. In this respect, am I not following a mixed path of TDD and TAD?
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Pikesville Paesano
on 5/1/2009 8:25 AM

Can't TAD & TDD work together? I'm building my first MVC app using guidance from the ASP.NET MVC Storefront Starter Kit and started down the TDD path. But once I've finallized how I'm going to do all of my CRUD screens, I hand off to TAD because the design is solidified and the only real change is what fields I'm validating. This allows me to keep moving forward and get my prototype done without getting bogged down updating hundreds of tests on nearly identical objects. If I had a junior developer on staff, then I'd have him/her banging out my tests. To use the engineering example from above, then TDD comes up with 'the Dog house' and then TOD can test out the variations of the dog houses.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Deven Kalra
on 5/17/2009 10:37 PM

Apropos one of the responses above, in my experience I have found that stories, no matter how well written, fail to capture all the detailed behavior of the features in the application. In particular, the writer of the stories will leave out the constraints, edge cases and negative cases. For example, subject of the blog post is required, body cannot be empty.

One thing I am experimenting with now is to have the testers start at the same time as the developers. After a few days of the sprint (I am following SCRUM), the testers need to have the test cases done. The testers have the mind-set to look at not only the positive "happy" cases but also on the constraints and out-of-the ordinary cases. The QA test cases for a feature, in my opinion, really capture everything the testers can think of how the feature will be used. I then get the developers to review each test case and make sure that their code and their tests capture these cases.

Another issue I am hoping to avoid is developers using the missing details in the story as a crutch for parts of features, especially outside the happy case not working.

My theory is that the test cases is what QA will test the feature with. If each test case is coded and working, there should really be practically zero bugs.

Let's see how this goes. Would love to hear your comments.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Panduan PTD
on 7/1/2009 1:36 PM

I think it is just a concept rather than approach. Test first and test after development is not slower in the sense that you can re-work your code at a much faster rate and it is in general should be better written. Looking at it in the different view make me laugh a bit. hehe. btw good article though
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by .net 3.5 development
on 7/1/2009 10:21 PM

Ya really i agree with you. Its very useful. Thanks.
Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Adrian Grigore
on 7/14/2009 5:53 AM

Interesting post, especially since I am not sure there really is a holy grail (TDD or TAD). I am somewhere between the TDD and the TAD.

My question is whether you believe that TDD ALWAYS yields the better results. I'm still not conviced it does.

Let's assume you are implementing a simple application based on ASP.NET MVC which always uses the same design principles (fetch data from repository, run a few business rules, perhaps write some data back to repository, return view).

About 90% of these action methods do not require any new design. So TDD would not have any advantage over TAD when implementing those.

Since I am usually coding slower when using TDD than when using TAD, I see no reason to use TDD in those instances. Doing it nevertheless seems like following the methodology for the sake of the methodology to me, the price being wasted development time.

Don't get me wrong, I am not saying TDD is worthless. When implementing something non-trivial (in other words, coding something where I am not sure upfront about the design), then TDD is definitely useful. But for implementing data-driven web pages with ASP.NET MVC, I found that this is rarely the case.

Gravatar
# re: Test-After Development is not Test-Driven Development
Posted by Neil M
on 9/21/2009 2:25 AM

A concern that pops up when I read about TDD, with the "do the minimum right now" code approach, is how do you get a team of developers to work together, solving common problems just once?

Without a guiding design up front, wouldn't developers solve comparable problems with new code? I've seen too many projects working as a collection of individual developers rather than a team, with the result of a huge amount of duplication. TDD seems to me like it steers you down that path, and it's a rocky path to hell.

What have I not 'got' about TDD?

Comments have been closed on this topic.