Reusable Node Code Modules, Part 11 - Pagination

Handling Large Data Sets: Creating Reusable Pagination Modules

Pagination is essential for efficiently handling large datasets by breaking them into manageable chunks. Implementing reusable pagination modules ensures consistent and efficient data handling across projects. This guide covers how to structure reusable pagination modules in Node.js using Express, manage pagination efficiently, and integrate common libraries.

Common Libraries and Tools

1. Mongoose-Paginate

Mongoose-Paginate is a simple pagination plugin for Mongoose

Key Features

• Simple Integration: Easy to integrate with Mongoose models

• Customizable: Allows customization of pagination options

• Aggregations: Supports pagination for aggregation queries

• Performance: Efficient pagination with minimal overhead

2. Sequelize-Paginate

Sequelize-Paginate is a plugin for paginating Sequelize queries

Key Features

• Seamless Integration: Integrates with Sequelize models

• Customizable: Allows flexible pagination options

• Associations: Supports pagination for queries with associations

• Query Optimization: Optimized for performance with large datasets

3. Knex-Paginate

Knex-Paginate is a pagination plugin for Knex.js

Key Features

• Versatile: Works with any SQL database supported by Knex.js

• Flexible: Provides flexible pagination options

• Customizable: Supports custom pagination queries

• Efficiency: Designed for efficient pagination with large datasets

4. Express-Paginate

Express-Paginate is a middleware for pagination in Express

Key Features

• Middleware Integration: Integrates easily with Express routes

• Customizable: Allows customization of pagination parameters

• Query Handling: Simplifies handling of pagination queries

• Compatibility: Compatible with various database query libraries

Comparison

• Mongoose-Paginate: Best for simple and efficient pagination with Mongoose models

• Sequelize-Paginate: Ideal for paginating Sequelize queries with support for associations

• Knex-Paginate: Suitable for versatile and flexible pagination with Knex.js

• Express-Paginate: Best for middleware-based pagination in Express applications

Examples and Wrapper Class

Example 1: Mongoose-Paginate

Setup:

$ npm install mongoose mongoose-paginate

Configuration:

import mongoose from 'mongoose';
import mongoosePaginate from 'mongoose-paginate';

const userSchema = new mongoose.Schema({
  username: String,
  email: String
});

userSchema.plugin(mongoosePaginate);

const User = mongoose.model('User', userSchema);

const paginateUsers = async (page, limit) => {
  const options = {
    page,
    limit
  };
  return await User.paginate({}, options);
};

export { paginateUsers };

Wrapper Class:

class UserService {
  constructor() {
    this.User = User;
  }

  async paginate(page, limit) {
    const options = {
      page,
      limit
    };
    return await this.User.paginate({}, options);
  }
}

export default new UserService();

Example 2: Sequelize-Paginate

Setup:

$ npm install sequelize sequelize-paginate

Configuration:

import { Sequelize, DataTypes } from 'sequelize';
import sequelizePaginate from 'sequelize-paginate';

const sequelize = new Sequelize('database', 'username', 'password', {
  host: 'localhost',
  dialect: 'mysql'
});

const User = sequelize.define('User', {
  username: DataTypes.STRING,
  email: DataTypes.STRING
});

sequelizePaginate.paginate(User);

const paginateUsers = async (page, limit) => {
  const options = {
    page,
    paginate: limit
  };
  return await User.paginate(options);
};

export { paginateUsers };

Wrapper Class:

class UserService {
  constructor() {
    this.User = User;
  }

  async paginate(page, limit) {
    const options = {
      page,
      paginate: limit
    };
    return await this.User.paginate(options);
  }
}

export default new UserService();

Example 3: Knex-Paginate

Setup:

$ npm install knex knex-paginate

Configuration:

import knex from 'knex';
import knexPaginate from 'knex-paginate';

const db = knex({
  client: 'mysql',
  connection: {
    host: 'localhost',
    user: 'username',
    password: 'password',
    database: 'database'
  }
});

knexPaginate(db);

const paginateUsers = async (page, limit) => {
  const { data, pagination } = await db('users').paginate({
    perPage: limit,
    currentPage: page
  });
  return { data, pagination };
};

export { paginateUsers };

Wrapper Class:

class UserService {
  constructor() {
    this.db = db;
  }

  async paginate(page, limit) {
    const { data, pagination } = await this.db('users').paginate({
      perPage: limit,
      currentPage: page
    });
    return { data, pagination };
  }
}

export default new UserService();

Example 4: Express-Paginate

Setup:

$ npm install express express-paginate

Configuration:

import express from 'express';
import paginate from 'express-paginate';
import db from './db'; // Your database setup

const app = express();

app.use(paginate.middleware(10, 50)); // Default limit and maximum limit

const getUsers = async (req, res) => {
  const [ results, itemCount ] = await Promise.all([
    db.select().from('users').limit(req.query.limit).offset(req.skip),
    db('users').count('* as count').first()
  ]);

  const pageCount = Math.ceil(itemCount.count / req.query.limit);

  res.json({
    object: 'list',
    has_more: paginate.hasNextPages(req)(pageCount),
    data: results
  });
};

app.get('/users', getUsers);

app.listen(3000, () => {
  console.log('Server started on http://localhost:3000');
});

Wrapper Class:

class UserService {
  constructor(db) {
    this.db = db;
  }

  async paginate(req) {
    const [ results, itemCount ] = await Promise.all([
      this.db.select().from('users').limit(req.query.limit).offset(req.skip),
      this.db('users').count('* as count').first()
    ]);

    const pageCount = Math.ceil(itemCount.count / req.query.limit);

    return {
      object: 'list',
      has_more: paginate.hasNextPages(req)(pageCount),
      data: results
    };
  }
}

export default new UserService(db);