Daisy chain input,output channels together in golang

Issue

I have the following interface and struct

type PiplineStep interface {
    Do(ctx context.Context, in <-chan Message) (<-chan Message, <-chan error, error)
}

type Pipline struct {
    Steps []core.PiplineStep
}

Now I am trying to daisy the interfaces to create a pipeline like the following

    for _, step := range p.Steps {
        out, errc, err := step.Do(ctx, out)
        errcList = append(errcList, errc)

        if err != nil {
            errc <- err
            return
        }

        select {
        case outer <- msg:
        case <-ctx.Done():
            return
        }
    }

But the compiler says no is this possible?

I get the following Error ‘out declared and not used’ i have attempted following but it appears that all steps are receiving the same chan

    for _, step := range p.Steps {
        var tmpOut <-chan core.Message
        tmpOut = out
        tmpOut, errcTmp, err := step.Do(ctx, tmpOut)
        errcList = append(errcList, errcTmp)

        if err != nil {
            errc <- err
            return
        }
        select {
        case out <- msg:
        case <-ctx.Done():
            return
        }
    }

Solution

You have to declare your channel variable outside the loop if you want to re-use it in each iteration (errors and context omitted for brevity):

package main

import "fmt"

func main() {
    var pipeline Pipeline

    pipeline.Steps = append(pipeline.Steps,
            AddBang{},
            AddBang{},
            AddBang{},
    )

    src := make(chan Message)
    pipe := src

    for _, s := range pipeline.Steps {
            pipe = s.Do(pipe)
    }

    go func() {
            src <- "msg 1"
            src <- "msg 2"
            src <- "msg 3"
    }()

    fmt.Println(<-pipe)
    fmt.Println(<-pipe)
    fmt.Println(<-pipe)
}

type Message string

type Pipeline struct {
    Steps []PipelineStep
}

type PipelineStep interface {
    Do(in chan Message) chan Message
}

type AddBang struct{}

func (AddBang) Do(in chan Message) chan Message {
    out := make(chan Message)
    go func() {
            defer close(out)
            for m := range in {
                    out <- m + "!"
            }
    }()
    return out
}

Try it on the playground: https://play.golang.org/p/ItVLUBRpNA1

Answered By – Peter

Answer Checked By – Gilberto Lyons (GoLangFix Admin)

Leave a Reply

Your email address will not be published.