How can I prevent a data race when adding handlers in a goroutine?

Issue

In my HTTP application written in golang I have a few routes that rely on 3rd party services (and vendored code) to do some work before I am actually able to register the route. This might fail or need to be retried, but I still want the application to respond to other requests while this is process is potentially ongoing.

This means I am registering handlers on http.DefaultServeMux in goroutines that I spawn from my main func. This works as expected, but I will find my tests complaining about data races now.

A minimal case to repro looks like this:

package main

import (
    "log"
    "net/http"
)

func main() {
    go func() {
        http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
            w.Write([]byte("hello"))
        })
    }()
    srv := http.Server{
        Addr: ":3000",
    }
    log.Fatal(srv.ListenAndServe())
}

With a test like:

package main

import (
    "io/ioutil"
    "net/http"
    "os"
    "testing"
    "time"
)

func TestMain(m *testing.M) {
    go main()
    time.Sleep(time.Second)
    os.Exit(m.Run())
}

func TestHello(t *testing.T) {
    t.Run("default", func(t *testing.T) {
        res, err := http.DefaultClient.Get("http://0.0.0.0:3000/hello")
        if err != nil {
            t.Fatalf("Calling /hello returned %v", err)
        }
        if res.StatusCode != http.StatusOK {
            b, _ := ioutil.ReadAll(res.Body)
            defer res.Body.Close()
            t.Errorf("Expected /hello to return 200 response, got %v with body %v", res.StatusCode, string(b))
        }
    })
}

will show me the following output:

==================
WARNING: DATA RACE
Read at 0x000000a337d8 by goroutine 14:
  net/http.(*ServeMux).shouldRedirect()
      /usr/local/go/src/net/http/server.go:2239 +0x162
  net/http.(*ServeMux).redirectToPathSlash()
      /usr/local/go/src/net/http/server.go:2224 +0x64
  net/http.(*ServeMux).Handler()
      /usr/local/go/src/net/http/server.go:2293 +0x184
  net/http.(*ServeMux).ServeHTTP()
      /usr/local/go/src/net/http/server.go:2336 +0x6d
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2694 +0xb9
  net/http.(*conn).serve()
      /usr/local/go/src/net/http/server.go:1830 +0x7dc

Previous write at 0x000000a337d8 by goroutine 8:
  net/http.(*ServeMux).Handle()
      /usr/local/go/src/net/http/server.go:2357 +0x216
  net/http.(*ServeMux).HandleFunc()
      /usr/local/go/src/net/http/server.go:2368 +0x62
  net/http.HandleFunc()
      /usr/local/go/src/net/http/server.go:2380 +0x68
  github.com/m90/test.main.func1()
      /home/frederik/projects/go/src/github.com/m90/test/main.go:10 +0x4f

Goroutine 14 (running) created at:
  net/http.(*Server).Serve()
      /usr/local/go/src/net/http/server.go:2795 +0x364
  net/http.(*Server).ListenAndServe()
      /usr/local/go/src/net/http/server.go:2711 +0xc4
  github.com/m90/test.main()
      /home/frederik/projects/go/src/github.com/m90/test/main.go:17 +0xb6

Goroutine 8 (finished) created at:
  github.com/m90/test.main()
      /home/frederik/projects/go/src/github.com/m90/test/main.go:9 +0x46
==================

From what I understand reading the code of package http net/http.(*ServeMux).Handler() in the stacktrace does not lock the mutex that protects the handler map, as it expects this to be done by net/http.(*ServeMux).handler() which in my scenario does not get called.

Am I doing something that is not supposed to be done? Is this an issue with the standard library? Am I doing something wrong in the way I attach the handlers in a goroutine?

Solution

This seems to be an issue in package http itself, that is resolved via this Pull Request.

As of April 2018 the patch is not included in go1.10.1, but it’s supposed to ship with go1.11

Answered By – m90

Answer Checked By – Terry (GoLangFix Volunteer)

Leave a Reply

Your email address will not be published.