Explore the future of Web Scraping. Request a free invite to ScrapeCon 2024

Simple Pagination with Node.js, Express, and Mongoose

Keeping it simple with Pagination

Are you looking to limit the amount of the documents that comes back from MongoDB per request? We've all seen it before, pagination to the rescue! There are many ways to implement pagination but I will show you the simplest approach (in my opinion) to get it done.

This approach allows the client to supply two query parameters in the request: page and limit. Page refers to the current page you are requesting. Limit is the amount of documents you wish to retrieve.

Let's Get Started

Now, I am going to assume that you have basic knowledge working with Node.js, Express, and Mongoose. Below, we have a simple express server running on port 3000 with a mongoose connection. We also import a mongoose model Posts so we can use it in the route handler.

app.js

require("dotenv").config();
const express = require("express");
const mongoose = require("mongoose");
const Posts = require("./models/posts");

const app = express();
const port = 3000;

mongoose.connect(process.env.MONGO_URI, {
  useNewUrlParser: true,
  useUnifiedTopology: true,
});

app.get("/posts", async (req, res) => {
  const posts = await Posts.find().exec();

  res.json(posts);
});

app.listen(port, () => console.log(`Example app listening on port ${port}!`));

In the code above, when we make a GET request to "/posts", we will retrieve all the documents in the collection. Let's modify the route to incorporate pagination.

app.js

app.get("/posts", async (req, res) => {
  // destructure page and limit and set default values
  const { page = 1, limit = 10 } = req.query;

  try {
    // execute query with page and limit values
    const posts = await Posts.find()
      .limit(limit * 1)
      .skip((page - 1) * limit)
      .exec();

    // get total documents in the Posts collection
    const count = await Posts.countDocuments();

    // return response with posts, total pages, and current page
    res.json({
      posts,
      totalPages: Math.ceil(count / limit),
      currentPage: page,
    });
  } catch (err) {
    console.error(err.message);
  }
});

Let's break it down. When a request is made to the "/post" route, we are destructing page and limit from the request query. We are also providing default values for them if they are not provided. If they are provided, they will override the default values.

Now, let's look at the mongoose query. Limit is the number of documents we want to retrieve. Skip is the amount of documents we want to skip before retrieving our documents. For skip, we take the page and subtract one, then multiply it by the limit. On line 10, we grab the total documents and store it in a variable count so we can calculate total pages by dividing the count by the limit. Lastly, we return the posts, total pages, and current page. We wrap everything in a try/catch to catch any exceptions.

Testing our Simple Pagination Solution

To test our simple pagination solution, let's use Postman to make a GET request to the "/posts" with query parameters of page set to 1 and limit set to 2.

image

Viola! There you have it! Instead of retrieving all the documents, we are now retrieving page 1 of 50 with limit of 2.

Conclusion

I want to reemphasize that there are more than a few ways to implement pagination. If you are new to working with pagination, I found this to be the simplest solution. In the near future, I will cover cursor pagination, which provides a unique identify for a specific record and is more performant when working with very large data sets. I hope you found this article helpful and you are able to implement this into whatever project you may be working on.




Continue Learning