Implementing Pagination, Search, and Filtering in a Angular & Node.js API

When developing APIs for modern web applications, it’s essential to implement features like pagination, search, and filtering to improve user experience and optimize performance. In this blog post, we’ll break down how to implement a GET API using Node.js, Express, and MongoDB that handles pagination, search (filtering), and sorting for a test series.

1. Setting Up the API Endpoint

We’re using an Express router to define an API route for fetching test series. Below is the basic structure of the route handler for the /api/testSeries endpoint.

router.get('/api/testSeries', async (req, res) => {
  try {
    // Pagination, filtering, and sorting logic will go here
  } catch (error) {
    res.status(500).json({ message: 'Internal Server Error' });
  }
});

2. Pagination

Pagination is used to break large datasets into smaller chunks, making it easier to load, display, and navigate through large collections of data. In the API, pagination is controlled by two query parameters:

• page: The current page number, starting from 1.
• limit: The number of items to be displayed on each page.

// Pagination
const page = parseInt(req.query.page) || 1;   // Default to page 1
const limit = parseInt(req.query.limit) || 10; // Default limit of 10 items

• If no values for page or limit are provided, the default values of page 1 and limit 10 are used.
• We calculate the startIndex and endIndex based on these values, which helps in slicing the data for pagination.

3. Search and Filtering

In most cases, users will want to search or filter items based on specific fields, such as the name, category, or sub-category of the test series. For this, we construct a filter object based on the query parameter searchTerm.

The filter is built using the MongoDB $regex operator to perform case-insensitive partial text matching. Here’s how we set it up:

// Filtering
const filter = {};
if (req.query.searchTerm) {
  filter.$or = [
    { name: { $regex: req.query.searchTerm, $options: 'i' } },
    { category: { $regex: req.query.searchTerm, $options: 'i' } },
    { subCategory: { $regex: req.query.searchTerm, $options: 'i' } }
  ];
}

• The $or operator allows us to search across multiple fields.
• The $regex operator performs a text search based on the provided searchTerm, with the i option for case-insensitivity.

4. Sorting

Sorting allows users to view data in a specific order, such as alphabetically or by date. Sorting is handled by the sortBy query parameter, which consists of two parts: the field to sort by and the sort order (ascending or descending).

// Sorting
const sort = {};
if (req.query.sortBy) {
  const [field, order] = req.query.sortBy.split(':');
  sort[field] = order === 'desc' ? -1 : 1;
}

• We split the sortBy query parameter using : to separate the field and the sort order.
• order === 'desc' ? -1 : 1 ensures that the sorting order is descending (-1) or ascending (1), based on the provided query parameter.

5. Fetching the Data

Once the pagination, filtering, and sorting configurations are set, we fetch the data from MongoDB using Mongoose’s .find() method. We apply the filter and sort conditions and then slice the data for pagination.

// Fetch all matching test series without pagination for global search
const allMatchingTestSeries = await TestSeries.find(filter).sort(sort);

// Apply pagination to the results
const totalTestSeries = allMatchingTestSeries.length;
const startIndex = (page - 1) * limit;
const endIndex = startIndex + limit;
const paginatedTestSeries = allMatchingTestSeries.slice(startIndex, endIndex);

• We first retrieve all matching documents from the database using the .find(filter) method.
• Then, we apply the sorting using .sort(sort).
• Finally, we slice the results using the slice(startIndex, endIndex) method to implement pagination.

6. Sending the Response

After retrieving the data, we send the response to the client in JSON format. The response includes the paginated test series, total number of pages, current page, and total number of items.

res.json({
  testSeries: paginatedTestSeries,
  totalPages: Math.ceil(totalTestSeries / limit),
  currentPage: page,
  totalItems: totalTestSeries
});

• totalPages: The total number of pages, calculated by dividing the total number of items by the page limit.
• currentPage: The current page number.
• totalItems: The total number of matching items in the database.

7. Handling Errors

To handle any potential errors that may arise during the execution of the API (such as database connectivity issues), we wrap the entire code inside a try-catch block. If an error occurs, we send a 500 status code and an error message.

catch (error) {
  console.error('Error fetching test series:', error);
  res.status(500).json({ message: 'Internal Server Error' });
}

SEE FULL API ROUTE

router.get('/api/testSeries', async (req, res) => {
  try {
    // Pagination
    const page = parseInt(req.query.page) || 1;
    const limit = parseInt(req.query.limit) || 10;

    // Filtering
    const filter = {};
    if (req.query.searchTerm) {
      filter.$or = [
        { name: { $regex: req.query.searchTerm, $options: 'i' } },
        // Add other fields you want to search here
        { category: { $regex: req.query.searchTerm, $options: 'i' } },
        { subCategory: { $regex: req.query.searchTerm, $options: 'i' } }
      ];
    }

    // Sorting
    const sort = {};
    if (req.query.sortBy) {
      const [field, order] = req.query.sortBy.split(':');
      sort[field] = order === 'desc' ? -1 : 1;
    }

    // Fetch all matching test series without pagination for global search
    const allMatchingTestSeries = await TestSeries.find(filter).sort(sort);

    // Apply pagination to the results
    const totalTestSeries = allMatchingTestSeries.length;
    const startIndex = (page - 1) * limit;
    const endIndex = startIndex + limit;
    const paginatedTestSeries = allMatchingTestSeries.slice(startIndex, endIndex);

    res.json({
      testSeries: paginatedTestSeries,
      totalPages: Math.ceil(totalTestSeries / limit),
      currentPage: page,
      totalItems: totalTestSeries
    });
  } catch (error) {
    console.error('Error fetching test series:', error);
    res.status(500).json({ message: 'Internal Server Error' });
  }
});

ALL SEE HTML & TS File

<p-toast></p-toast>

<div class="page">
  <div class="page-main">
    <div class="app-content">
      <div class="side-app">
        <div class="row row-deck">
          <div class="col-lg-12 col-md-12">
            <div class="card">
              <div class="card-header d-flex justify-content-between align-items-center">
                <form [formGroup]="searchForm" class="w-50">
                  <input type="text" formControlName="searchTerm" class="form-control" placeholder="Search MCQs globally...">
                </form>
                <a routerLink="/dashboard/mcq/form" class="btn btn-success" id="skip">Add Mcq</a>
              </div>
              <div class="table-responsive">
                <table class="table card-table table-bordered table-vcenter text-nowrap">
                  <thead>
                    <tr>
                      <th (click)="onSort('id')">Id <i class="fa" [ngClass]="{'fa-sort-up': sortBy === 'id:asc', 'fa-sort-down': sortBy === 'id:desc'}"></i></th>
                      <th (click)="onSort('name')">Strength Name <i class="fa" [ngClass]="{'fa-sort-up': sortBy === 'name:asc', 'fa-sort-down': sortBy === 'name:desc'}"></i></th>
                      <th>Image</th>
                      <th (click)="onSort('dateCreated')">Date <i class="fa" [ngClass]="{'fa-sort-up': sortBy === 'dateCreated:asc', 'fa-sort-down': sortBy === 'dateCreated:desc'}"></i></th>
                      <th>Edit</th>
                    </tr>
                  </thead>
                  <tbody>
                    <tr *ngFor="let items of getMcqList; let i=index">
                      <td>{{ (currentPage - 1) * itemsPerPage + i + 1 }}</td>
                      <td class="">{{items.name}}</td>
                      <td class=""><img [src]="items.image" alt="" style="width: 40%"></td>
                      <td class="text-nowrap">{{ items.dateCreated | date: 'short' }}</td>
                      <td class="text-nowrap">
                        <a class="btn btn-outline-primary" (click)="updateMcq(items.id)">Edit</a>
                      </td>
                    </tr>
                  </tbody>
                </table>
              </div>
              <div *ngIf="isLoading" class="text-center mt-3">
                <div class="spinner-border" role="status">
                  <span class="sr-only">Loading...</span>
                </div>
              </div>
              <div *ngIf="error" class="alert alert-danger mt-3" role="alert">
                {{ error }}
              </div>
              <div *ngIf="getMcqList.length === 0 && !isLoading" class="alert alert-info mt-3" role="alert">
                No MCQs found matching your search criteria.
              </div>
              <div class="card-footer d-flex justify-content-between align-items-center">
                <div>
                  Showing {{ getDisplayRange().start }} to {{ getDisplayRange().end }} of {{ totalItems }} entries
                </div>
                <nav aria-label="Page navigation">
                  <ul class="pagination mb-0">
                    <li class="page-item" [class.disabled]="currentPage === 1">
                      <a class="page-link" (click)="onPreviousPage()" aria-label="Previous">
                        <span aria-hidden="true">&laquo;</span>
                      </a>
                    </li>
                    <li class="page-item" *ngFor="let page of getPageNumbers()" [class.active]="page === currentPage">
                      <a class="page-link" (click)="onPageChange(page)">{{ page }}</a>
                    </li>
                    <li class="page-item" [class.disabled]="currentPage === totalPages">
                      <a class="page-link" (click)="onNextPage()" aria-label="Next">
                        <span aria-hidden="true">&raquo;</span>
                      </a>
                    </li>
                  </ul>
                </nav>
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

TS FILE

import { Component, OnInit } from '@angular/core';
import { FormBuilder, FormGroup } from '@angular/forms';
import { ActivatedRoute, Router } from '@angular/router';
import { TestSeriesService } from 'src/app/service/testSeries.service';
import { debounceTime, distinctUntilChanged, finalize } from 'rxjs/operators';

@Component({
  selector: 'app-mcq-list',
  templateUrl: './mcq-list.component.html',
  styleUrls: ['./mcq-list.component.css']
})
export class McqListComponent implements OnInit {
  getMcqList: any[] = [];
  totalItems: number = 0;
  currentPage: number = 1;
  itemsPerPage: number = 10;
  searchForm: FormGroup;
  totalPages: number = 1;
  isLoading: boolean = false;
  error: string | null = null;
  sortBy: string = '';

  constructor(
    private formBuilder: FormBuilder,
    private testSeriesService: TestSeriesService,
    private router: Router,
    private route: ActivatedRoute
  ) {
    this.searchForm = this.formBuilder.group({
      searchTerm: ['']
    });
  }

  ngOnInit() {
    this.setupSearchListener();
    this.getAllMcqsList();
  }

  setupSearchListener() {
    this.searchForm.get('searchTerm')?.valueChanges
      .pipe(
        debounceTime(300),
        distinctUntilChanged()
      )
      .subscribe(() => {
        this.currentPage = 1;
        this.getAllMcqsList();
      });
  }

  getAllMcqsList() {
    this.isLoading = true;
    this.error = null;
    const searchTerm = this.searchForm.get('searchTerm')?.value;
    this.testSeriesService.getMcqs(this.currentPage, this.itemsPerPage, searchTerm, this.sortBy)
      .pipe(
        finalize(() => this.isLoading = false)
      )
      .subscribe(
        (res: any) => {
          this.getMcqList = res.testSeries;
          this.totalItems = res.totalItems;
          this.totalPages = res.totalPages;
        },
        (error) => {
          this.error = 'An error occurred while fetching data. Please try again.';
          console.error('Error fetching MCQs:', error);
        }
      );
  }

  updateMcq(id: string) {
    this.router.navigateByUrl(`dashboard/mcq/form/${id}`);
  }

  onPageChange(page: number) {
    this.currentPage = page;
    this.getAllMcqsList();
  }

  onPreviousPage() {
    if (this.currentPage > 1) {
      this.currentPage--;
      this.getAllMcqsList();
    }
  }

  onNextPage() {
    if (this.currentPage < this.totalPages) {
      this.currentPage++;
      this.getAllMcqsList();
    }
  }

  getPageNumbers(): number[] {
    const pageNumbers = [];
    const startPage = Math.max(1, this.currentPage - 2);
    const endPage = Math.min(this.totalPages, startPage + 4);
    for (let i = startPage; i <= endPage; i++) {
      pageNumbers.push(i);
    }
    return pageNumbers;
  }

  getDisplayRange(): { start: number; end: number } {
    const start = (this.currentPage - 1) * this.itemsPerPage + 1;
    const end = Math.min(this.currentPage * this.itemsPerPage, this.totalItems);
    return { start, end };
  }

  onSort(field: string) {
    this.sortBy = this.sortBy === `${field}:asc` ? `${field}:desc` : `${field}:asc`;
    this.getAllMcqsList();
  }
}

Service

getMcqs(page: number = 1, limit: number = 10, searchTerm: string = '', sortBy: string = ''): Observable<any> {
    const params: any = {
      page: page.toString(),
      limit: limit.toString()
    };

    if (searchTerm) {
      params.searchTerm = searchTerm;
    }

    if (sortBy) {
      params.sortBy = sortBy;
    }

    return this.http.get(`${this.apiURLTestSeries}/api/testSeries`, { params });
  }

Conclusion

This API implementation provides a comprehensive way to handle pagination, search, filtering, and sorting in a Node.js application using MongoDB. These features are crucial for improving performance and user experience, especially when dealing with large datasets.

With these techniques, you can provide users with a more seamless and responsive browsing experience in your application.