goroutine leak in example of book The Go Programming Language

Issue

I am reading The Go Programming Language book, there is an example of the book which demonstrates goroutine leaking


func mirroredQuery() string {
    responses := make(chan string, 3)
    go func() { responses <- request("asia.gopl.io") }()
    go func() { responses <- request("europe.gopl.io") }()
    go func() { responses <- request("americas.gopl.io") }()
    return <-responses // return the quickest response
}
func request(hostname string) (response string) { /* ... */ }

And I have tried to solve the leak, and got the following code

func request(url string) string {
    res, err := http.Get(url)
    if err == nil {
        body, err := io.ReadAll(res.Body)
        if err == nil {
            return string(body)
        } else {
            return err.Error()
        }
    } else {
        return err.Error()
    }
}

func getany() string {
    rsp := make(chan string, 3)
    done := make(chan struct{}, 3)
    doRequest := func(url string) {
        select {
            case rsp <- request(url):
                fmt.Printf("get %s\n", url)
                done <- struct{}{}
            case <- done:
                fmt.Printf("stop %s\n", url)
                return
        }
    }
    go doRequest("http://google.com")
    go doRequest("http://qq.com")
    go doRequest("http://baidu.com")
    return <-rsp
}

but it seems does not solve the problem? any suggestions?

Solution

There is no goroutine leakage in the provided code. The mirroredQuery method uses a buffered channel to collect the result and return the first answer. And the currently buffer has enough space to collect all answers from all goroutines, even if the rest of the responses are never read. The situation will change if the buffer is smaller than N – 1, where N is the number of spawned goroutines. In this situation some of the goroutines spawned by mirroredQuery will get stuck trying to send a response to the responses channel. Repeating the call to mirroredQuery will cause increase of stucked goroutines which can be called goroutines leak.

Here is the code with the logs added and the output for both scenarios.

func mirroredQuery() string {
    responses := make(chan string, 2)
    go func() {
        responses <- request("asia.gopl.io")
        log.Printf("Finished goroutine asia.gopl.io\n")
    }()
    go func() {
        responses <- request("europe.gopl.io")
        log.Printf("Finished goroutine europe.gopl.io\n")
    }()
    go func() {
        responses <- request("americas.gopl.io")
        log.Printf("Finished goroutine americas.gopl.io\n")
    }()
    return <-responses // return the quickest response
}
func request(hostname string) (response string) {
    duration := time.Duration(rand.Int63n(5000)) * time.Millisecond
    time.Sleep(duration)
    return hostname
}

func main() {
    rand.Seed(time.Now().UnixNano())
    result := mirroredQuery()
    log.Printf("Fastest result for %s\n", result)
    time.Sleep(6*time.Second)
}

Output for buffer size >= N-1

2021/06/26 16:05:27 Finished europe.gopl.io
2021/06/26 16:05:27 Fastest result for europe.gopl.io
2021/06/26 16:05:28 Finished asia.gopl.io
2021/06/26 16:05:30 Finished americas.gopl.io

Process finished with the exit code 0

Output for buffer size < N-1

2021/06/26 15:47:54 Finished europe.gopl.io
2021/06/26 15:47:54 Fastest result for europe.gopl.io

Process finished with the exit code 0

Above implementation can be "improved" by introducing goroutines termination when the first response arrives. This can potentially lower the number of used resources. It strongly depends on what request method do. For computation heavy scenarios it makes sense, for cancelling http request may lead to a connection termination, so the next request must open new one. For highly loaded servers it may be less effective than waiting for a response even if response is not used.

Below is the improved implementation with context usage.

func mirroredQuery() string {
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    responses := make(chan string)
    f := func(hostname string) {
        response, err := request(ctx, hostname)
        if err != nil {
            log.Printf("Finished %s with error %s\n", hostname, err)
            return
        }
        responses <- response
        log.Printf("Finished %s\n", hostname)
    }
    go f("asia.gopl.io")
    go f("europe.gopl.io")
    go f("americas.gopl.io")
    return <-responses // return the quickest response
}

func request(ctx context.Context, hostname string) (string, error) {
    duration := time.Duration(rand.Int63n(5000)) * time.Millisecond
    after := time.After(duration)
    select {
    case <-ctx.Done():
        return "", ctx.Err()
    case <-after:
        return "response for "+hostname, nil
    }
}

func main() {
    rand.Seed(time.Now().UnixNano())
    result := mirroredQuery()
    log.Printf("Fastest result for %s\n", result)
    time.Sleep(6 * time.Second)
}

Answered By – Jaroslaw

Answer Checked By – Dawn Plyler (GoLangFix Volunteer)

Leave a Reply

Your email address will not be published.