Setting Up for Success

In the interest of transparency, I ran into a few issues while writing this post that had me rethinking the structure of everything we’ve done so far. But I think that’s OK, because it was a learning experience, and the final result will be a better testing framework at the end of the day. I haven’t completely decided on exactly how we need to change things here, but in this post I’ll walk through the progress on our current implementation that lead to the realization that we need may need to backtrack a bit.

Last time, we got to a state where we were able to run individual test cases and verify that they ran. The next step is to enable users of our test type to specify setup functions before each case is run. This is an essential function of any good testing framework, and I’d like to discuss the reasons for it in the composite post that I put together once the Evergreen framework is feature complete, but a good overview is given in Kent Beck’s Test-Driven Development.

Get to Red

As usual, we start by writing a test. Right now, we have our TestTestRunner.go file’s main() function running our single test on its own. Now that we have a second test, let’s break them up into separate functions and have the main function call them. Our second test should verify that, after running a test, the test “setup” was performed as expected. The result is something like this:

package main

func main() {
	testWasRun()
	testSetUp()
}

func testWasRun() {
	test := NewTestCase("TestMethod")
	Assert(!test.wasRun)
	test.Run()
	Assert(test.wasRun)
}

func testSetUp() {
	test := NewTestCase("TestMethod")
	test.Run()
	Assert(test.wasSetUp)
}

This will not build, of course, because the wasSetUp field is undefined. As we’ve said, this means we’ve gotten to “red”; that is, a failing test. But we do want to see our test run and fail before we attempt to make it pass.

Let’s add the wasSetUp field.

type TestCase struct {
	name     string
	wasRun   bool
	wasSetUp bool
}

If we run this, we get the message, “Assertion failed.” This is what we expect, of course–a failing test. Ideally, I’d like to see a little more information here about which test failed and why, so we know our test is failing for the reason that we expect it’s failing. For now, however, our code is simple enough that we can add this functionality a bit later.

By the way, its worth asking ourselves whether the assertion fails because the wasSetUp member was false, or whether at that point it’s undefined (nil in go).

The answer here is the latter. In Go, there is no such thing as an unitizialized value–all members and variables get initialized automatically to the zero value for their type (false, in the case of a boolean). An argument could be made to explicitly set that value in the constructor for the sake of readability, but for now we’ll let Go do its thing.

Get to Green

Now, let’s define a SetUp method on the TestCase and give it the minimum functionality to get our test to pass.

func (t *TestCase) SetUp() {
	t.wasSetUp = true
}

Of course, we need to call our new method as the first step of our Run() function.

func (t *TestCase) Run() {
	t.SetUp()
	method := reflect.ValueOf(t).MethodByName(t.name)
	method.Call(nil)
}

Running this removes our failed assertion. We have a passing test!

In Test-Driven Development, once you have a failing test, the mandate is that you do the smallest amount of work to get to green as fast as possible. In the case of the above, the smallest amount of work would have been to simply set t.wasSetUp = true directly in the Run() method, and then break that call out into the SetUp() method as a part of the refactoring step (or, perhaps even better, to write a failing test that requires that assignment be performed by an existing SetUp() method, and pull out the assignment in the “get to gree” step).

It is important to be able to do things in the smallest steps possible. Once you understand how to make progress in tiny steps, you can more confidently make progress in larger steps, secure in the knowledge that if something doesn’t work how you expect it to, you can walk it back and then slow things down.

Refactor

This is where things start to break down.

First, we want to set the wasRun field to false in the SetUp method, rather than in the factory method.

func (t *TestCase) SetUp() {
	t.wasRun = false
	t.wasSetUp = true
}

I also decided to rename the file TestTestRunner.go to TestTestCase.go, since what this file is really doing is exercising the TestCase type, and we need to actually define a new type which we want to give the same name. In this newly renamed file, let’s use the functionality we just implemented to refactor our tests (refactoring applies to our tests as much as to the code that makes our tests pass).

type TestCaseTest struct {
	TestCase
}

We define a new struct TestCaseStruct, which is a TestCase. In object-oriented languages, the is a relationship is typically represented using inheritance. TestCaseTest inherits from or extends the TestCase type. Go does not support inheritance in the same way that languages like C++ or Java do. Instead, it uses a concept called composition through type embedding.

For our case, that would look like this:

type TestCaseTest struct {
	TestCase
	test TestCase
}

We want to define a type TestCaseTest which is responsible for holding all of the individual test cases as named methods. This above syntax specifies that TestCaseTest embeds the TestCase type (written as a property on the struct without a name). That gives TestCaseTest all of it’s own fields and methods in addition to all of the fields and methods of TestCase. This is how Go does code re-use without classical inheritance.

What’s the problem with doing things this way? Let’s look again at TestCase.Run().

func (t *TestCase) Run() {
	t.SetUp()
	method := reflect.ValueOf(t).MethodByName(t.name)
	method.Call(nil)
}

The above method can be called on instances of TestCaseTest even though it isn’t directly defined on the type, because it’s defined on a type that TestCaseTest embeds. Unfortunately, calling myTestCaseTest.Run() is really just a convenience feature of Go. Under the hood, the compiler is actually calling myTestCaseTest.TestCase.Run(). The method is actually being called on an instance of TestCase, rather than an instance of TestCaseTest. And TestCase doesn’t have access to methods defined on containing types, even via reflection, because it doesn’t know about the containing type in the first place.

This is a limitation of Go’s approach of not implementing classical inheritance. We can’t select TestCase methods to execute at runtime without re-defining the Run() method on the containing type. This wouldn’t be a problem for us in languages like Java or Python, but in Go, we’ll have to take a few steps back and re-think the structure at a high-level.

That’s exactly what we’ll do next time. I likely won’t have a post out for a few weeks (I’m going out of town for a little vacation), but will attack this problem with renewed vigor once I do. See you then and thanks for reading!

Alec