MongoDB driver pagination

Issue

Currently I am able to return all my products from the collection.
I however want to be able to return products that come after a specific product ID (which would be the last one on the client side so they could load more)

Current way (return all)

query := bson.M{}

var product ReturnedProdcut
    var products []ReturnedProduct
    cur, err := mg.Db.Collection("products").Find(c.Request().Context(), query)
    if err != nil {
        fmt.Println(err)
    }
    for cur.Next(c.Request().Context()) {
        err := cur.Decode(&product)
        if err != nil {
            fmt.Println(err)
        }
        products = append(products, product)

    }

    // return products list in JSON format
    return c.JSON(http.StatusOK, products)

New Way Attempt(return based on page)

afterID := c.QueryParam("afterID")
if afterID == "" {
   // get from start of collection based on latest date
}
// get 10 products after this ID, if no ID then get from start

query := bson.M{}

var product ReturnedProduct
    var products []Returnedproduct
    //.find(afterId).limit(10) - something like this?
    
    cur, err := mg.Db.Collection("products").Find(c.Request().Context(), query)
    if err != nil {
        fmt.Println(err)
    }
    for cur.Next(c.Request().Context()) {
        err := cur.Decode(&product)
        if err != nil {
            fmt.Println(err)
        }
        products = append(products, product)

    }

    // return products list in JSON format
    return c.JSON(http.StatusOK, products)

Solution

You may construct a query where _id is greater than afterID, in which case you should also specify sorting by _id. For sorting and for setting a limit, you may use options.FindOptions.

You also should use Cursor.All() to decode all results and not one-by-one.

This is how it could look like:

query := bson.M{"_id": bson.M{"$gt": afterID}}
opts := options.Find().
    SetSort(bson.M{"_id": 1}).
    SetLimit(10)
    
ctx := c.Request().Context()

curs, err := mg.Db.Collection("products").Find(ctx, query, opts)
if err != nil {
    // Handle error
}

var products []Returnedproduct
if err = curs.All(ctx, &products); err != nil {
    // Handle error
}

return c.JSON(http.StatusOK, products)

Answered By – icza

Answer Checked By – Katrina (GoLangFix Volunteer)

Leave a Reply

Your email address will not be published.