Accurate handler tests using httptest

In Go, testing HTTP handlers is more work than testing regular functions. Inputs must be wrapped in HTTP requests, and responses are written to http.ResponseWriter, rather than being returned directly.

To make matters worse, handlers expect server-side requests, while it’s easy to mistakenly construct client-side requests.

You’d almost give up on testing your handlers altogether.

Please don’t.

This article will discuss how we can use httptest to safely test HTTP handlers.

How to test an HTTP handler

As you’re probably aware, HTTP handlers implement the following interface:

type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}

So, to test a handler we need to provide both a response writer and a request.

Let’s begin by looking at the request.

Constructing a test request

The request needs to be provided as a pointer to a http.Request. Unfortunately, it’s easy to create this the wrong way by accident.

The net/http package in the standard library uses http.Request in two ways:

• As incoming requests, on the server-side. Provided to handlers by a http.Server.
• As outgoing requests, on the client-side. Provided as input to a http.Client.

If you check the docs, you will see that a lot of fields should be interpreted differently depending on whether the request is server-side or client-side.

When testing HTTP handlers you need to make sure that you’re using server-side requests as input.

However, the two main constructor functions in the net/http package provide client-side requests. This is mentioned in the docs, but it’s easy to glance over.

Using client-side requests can lead to inaccurate handler tests. When running as part of a http.Server your handlers will only ever be provided with server-side requests.

This is where the httptest comes in. It has two constructor functions that return server-side requests:

• httptest.NewRequest: Creates a request that uses context.Background as a context.
• httptest.NewRequestWithContext: Creates a request with a provided context, useful when you need to test handlers that require values from the context and/or might be cancelled. (Since Go 1.23).

In the example below the same request is constructed twice. Once using a server-side constructor and once using a client-side constructor.

As you will see, these requests will have different ContentLength values.

package main

import (
	"fmt"
	"log"
	"net/http"
	"net/http/httptest"
)

func main() {
	serverReq := httptest.NewRequest("POST", "https://example.com", customReader("Hello world!"))
	clientReq, err := http.NewRequest("POST", "https://example.com", customReader("Hello world!"))
	if err != nil {
		log.Fatalf("failed to construct request: %v", err)
	}

	fmt.Printf("server content length: %+v\n", serverReq.ContentLength)
	fmt.Printf("client content length: %+v\n", clientReq.ContentLength)
}

// customReader is a reader of which the http package can't determine the length.
// In real apps this might be a file or other data stream.
type customReader string

func (customReader) Read(b []byte) (n int, err error) {
	return 0, nil
}

So, always be sure to use the httptest constructors to create server-side requests when testing HTTP handlers.

Now, let’s look at responses.

Recording the response

As we saw, HTTP handlers accept a http.ResponseWriter.

This type is an interface, meaning that our tests will need to provide an implementation to our handlers.

We could write this implementation ourselves, but, why do extra work?

The httptest package provides a mock implementation that records mutations and does sanity checks.

This mock implementation can be created by calling httptest.NewRecorder(). It will return a httptest.ResponseRecorder.

Each recorder can only record a single response, a new recorder must be created for each call to the handler.

To get the recorded response from the recorder, call the Result method after the handler has run. This returns a regular http.Response.

For example:

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"
)

func TestExtractResponse(t *testing.T) {
	req := httptest.NewRequest("POST", "https://example.com", strings.NewReader("Hello world!"))
	rr := httptest.NewRecorder()

	Handler(rr, req)

	// extract the response
	res := rr.Result()
	if res == nil {
		t.Fatal("no response :(")
	}

	defer res.Body.Close()

	fmt.Printf("%+v\n", res)
}

func Handler(w http.ResponseWriter, r *http.Request) {
	bdy, err := ioutil.ReadAll(r.Body)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	w.WriteHeader(http.StatusOK)
	w.Write(bdy)
}

It’s now possible to verify elements of this response, see the examples in the next section.

Verification examples

Response status

The example below verifies that the status code of the response is 200.

package main

import (
	"io/ioutil"
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"
)

func TestStatusCode(t *testing.T) {
	req := httptest.NewRequest("POST", "https://example.com", strings.NewReader("Hello world!"))
	rr := httptest.NewRecorder()

	Handler(rr, req)

	// extract the response
	res := rr.Result()
	defer res.Body.Close()

	// verify the status code
	if res.StatusCode != http.StatusOK {
		t.Fatalf("got %v wanted %v", res.StatusCode, http.StatusOK)
	}
}

func Handler(w http.ResponseWriter, r *http.Request) {
	bdy, err := ioutil.ReadAll(r.Body)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	w.WriteHeader(http.StatusOK)
	w.Write(bdy)
}

Response body

The example below verifies that the response body is equal to "Hello world!".

Here we first need to read the response body before we can compare it.

package main

import (
	"io/ioutil"
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"
)

func TestResponseBody(t *testing.T) {
	req := httptest.NewRequest("POST", "https://example.com", strings.NewReader("Hello world!"))
	rr := httptest.NewRecorder()

	Handler(rr, req)

	// extract the response
	res := rr.Result()
	defer res.Body.Close()

	bdy, err := ioutil.ReadAll(res.Body)
	if err != nil {
		t.Fatalf("unexpected error: %v", err)
	}

	if string(bdy) != "Hello world!" {
		t.Fatalf("unexpected body: %s", bdy)
	}
}

func Handler(w http.ResponseWriter, r *http.Request) {
	bdy, err := ioutil.ReadAll(r.Body)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	w.WriteHeader(http.StatusOK)
	w.Write(bdy)
}

Response headers

The example below tests that the Content-Type header of the response is equal to "text/plain".

package main

import (
	"io/ioutil"
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"
)

func TestContentTypeHeader(t *testing.T) {
	req := httptest.NewRequest("POST", "https://example.com", strings.NewReader("Hello world!"))
	rr := httptest.NewRecorder()

	Handler(rr, req)

	// extract the response
	res := rr.Result()
	defer res.Body.Close()

	contentType := res.Header.Get("Content-Type")
	if contentType != "text/plain" {
		t.Fatalf("got %s want %s", contentType, "text/plain")
	}
}

func Handler(w http.ResponseWriter, r *http.Request) {
	bdy, err := ioutil.ReadAll(r.Body)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
	// header needs to be set before WriteHeader is called.
	w.Header().Set("Content-Type", "text/plain")
	w.WriteHeader(http.StatusOK)
	w.Write(bdy)
}

Summary

• Testing HTTP handlers requires some care, the httptest package provides essential tools.
• Avoid using client-side constructors in the net/http when generating test requests.
• Always use httptest.NewRequest or httptest.NewRequestWithContext to create server-side requests for accurate tests.
• Use httptest.ResponseRecorder to construct valid http.ResponseWriter implementations.
• Create a new response recorder for each call to your HTTP handler.
• Extract the response from the recorder using the Result method.

That’s it for today.

Happy coding.