An idiomatic way to stop / kill / cleanup useless producer go routines

Issue

I am learning Go and read some articles about goroutines and channels. This seems like a really nice feature of Go. But… I don’t understand a good/idiomatic way to ensure proper cleanup for what I would call a ‘useless producer goroutine’.

There’s an article that explains the problem quite well, and it inspired this question. While the article explains the problem well, I find the solutions it’s offering not very compelling because they lack a bit in generality (e.g. the ‘just use buffered channels’ doesn’t really work for an infinite producer).

Note: You may read the article for background/reference, but I’ll explain enough here to make my question self contained.

What do I mean with a ‘useless producer’? I mean two things:

  1. a producer is a goroutine who’s only purpose is to do some processing and send some values to a channel. A producer may produce any number of values, including potentially infinitely many (e.g. say a producer that produces all prime numbers).

  2. a useless producer means a producer that is sending values to a channel that is no longer accessible to anyone but itself.

Let me clarify with a concrete example. Below is an example of function that creates a ‘Prime number producer’ and returns a channel that allows a consumer to get any number of primes.

func isPrime(num int) bool { ... details are not relevant... }

func GetPrimes() chan int {
    primes := make(chan int)

    go func() {
        primes <- 2
        primes <- 3
        for candidate := 5; ; candidate += 2 {
            if isPrime(candidate) {
                primes <- candidate
            }
        }
    }()

    return primes
}

A consumer might use this to get some primes and print them out:

func PrintSomePrimes() {
    primes := GetPrimes()
    for i := 0; i < 10; i++ {
        fmt.Println(<-primes)
    }
}

The problem / question I’m having here is, I think that once this function exits, there is no more reference to the primes channel (except for the one from the producer itself). So that means it is now impossible for anyone to read any more primes. The primes producer is now ‘useless’, since it is literally impossible for any consumer to come along and try to use/read the primes it is producing.

Nevertheless this ‘useless’ goroutine will not be garbage collected. It will simply remain ‘blocked’ trying to send values to a channel that nobody will ever be able to read.

Now the question… Is there a good ‘idiomatic way’ to stop this useless producer goroutine and avoid a memory leak?

One thought/hope I had was that perhaps there might be some way that this kind of thing could be automatically ‘garbage collected’ because no more consumers reading from the channel exist. Maybe the GC could detect this and somehow trigger the cleanup. I don’t think that is possible, but I’m not really sure.

A second thought was that, assuming GC of channels can’t provide this kind of mechanic, a ‘second best’ solution might be that the consumer signals they are done by closing the channel explicitly: so something like:

func PrintSomePrimes() {
    primes := GetPrimes()
    defer func() { close(primes) }()
    for i := 0; i < 10; i++ {
        fmt.Println(<-primes)
    }
}

However, I read that

One general principle of using Go channels is don’t close a channel from the receiver side

So it seems that is not ‘idiomatic’ go.

Solution

The idiomatic way of doing this is notifying the generator goroutine using another channel, or a context, that its results are no longer needed.

func generator(ctx context.Context) chan int {
   ch:=make(chan int)
   go func() {
       for {
          // Generate some value
          select {
             case <-ctx.Done():
                 return
             case ch<-value:
          }
       }
   }()
   return ch
}

func consumer() {
   ctx,cancel:=context.WithCancel(context.Background())
   defer cancel()
   ch:=generator(ctx)
   // Consume values
}

The above context-based implementation uses a separate channel (the done channel) to notify the generator. A separate channel can also be used. Closing the done channel on the receiving end broadcasts all the generators that they can stop.

Answered By – Burak Serdar

Answer Checked By – Terry (GoLangFix Volunteer)

Leave a Reply

Your email address will not be published.