Automated Performance Testing in CI/CD Pipelines

Manual performance testing doesn't scale. By the time someone notices the site is slow, it's been slow for weeks. Automated performance testing in CI/CD catches regressions the moment they're introduced.

Why Automate Performance Testing

Without automation:

1. Developer adds a 200KB library → nobody notices
2. Deploy goes to production → site gets slower
3. Weeks later → "why is our bounce rate up?"
4. Investigation → hours spent finding the cause
5. Revert/fix → damage already done

With automation:

1. Developer adds a 200KB library → CI fails
2. PR gets performance feedback → developer optimizes
3. Deploy goes to production → performance maintained

Lighthouse CI Setup

Basic Setup

npm install -g @lhci/cli

Configuration File

// lighthouserc.json
{
  "ci": {
    "collect": {
      "url": [
        "http://localhost:3000/",
        "http://localhost:3000/pricing",
        "http://localhost:3000/blog"
      ],
      "startServerCommand": "npm run start",
      "numberOfRuns": 3
    },
    "assert": {
      "assertions": {
        "categories:performance": ["error", { "minScore": 0.7 }],
        "categories:accessibility": ["warn", { "minScore": 0.9 }],
        "largest-contentful-paint": ["error", { "maxNumericValue": 2500 }],
        "cumulative-layout-shift": ["error", { "maxNumericValue": 0.1 }],
        "total-blocking-time": ["warn", { "maxNumericValue": 300 }],
        "total-byte-weight": ["error", { "maxNumericValue": 1500000 }]
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

GitHub Actions Workflow

name: Performance CI
on:
  pull_request:
    branches: [main]

jobs:
  lighthouse:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Build
        run: npm run build
      
      - name: Run Lighthouse CI
        run: |
          npm install -g @lhci/cli
          lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_TOKEN }}

Bundle Size Checking

Using bundlesize

// package.json
{
  "scripts": {
    "check-size": "bundlesize"
  },
  "bundlesize": [
    {
      "path": ".next/static/chunks/**/*.js",
      "maxSize": "200 kB",
      "compression": "gzip"
    },
    {
      "path": ".next/static/css/**/*.css",
      "maxSize": "50 kB",
      "compression": "gzip"
    }
  ]
}

Using size-limit

// package.json
{
  "size-limit": [
    {
      "path": "dist/index.js",
      "limit": "15 kB"
    },
    {
      "path": "dist/vendor.js",
      "limit": "100 kB"
    }
  ]
}

# GitHub Action
- name: Check bundle size
  uses: andresz1/size-limit-action@v1
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}

Visual Regression + Performance

Combine visual testing with performance metrics:

jobs:
  performance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build
      
      # Start the app
      - run: npm start &
      - run: npx wait-on http://localhost:3000
      
      # Run Lighthouse
      - name: Lighthouse
        run: |
          lhci autorun
      
      # Capture screenshots for visual comparison
      - name: Visual snapshot
        run: npx playwright test --project=screenshot

PR Comments with Results

Lighthouse CI GitHub App

Install the Lighthouse CI GitHub App to get automatic PR comments:

📊 Lighthouse Results for PR #234

| Page | Performance | LCP | CLS | TBT |
|------|-------------|-----|-----|-----|
| / | 78 (↓2) | 2.1s | 0.02 | 180ms |
| /pricing | 82 (→) | 1.8s | 0.01 | 150ms |
| /blog | 75 (↑3) | 2.4s | 0.03 | 200ms |

⚠️ Homepage performance decreased by 2 points

Custom PR Comment

- name: Comment PR
  uses: actions/github-script@v7
  with:
    script: |
      const results = JSON.parse(fs.readFileSync('lhci-results.json'));
      const comment = formatResults(results);
      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: comment
      });

Best Practices

1. Run Multiple Times

Lighthouse results vary between runs. Run 3-5 times and use the median:

{
  "collect": {
    "numberOfRuns": 5
  },
  "assert": {
    "aggregationMethod": "median-run"
  }
}

2. Test Production Build

Always test the production build, not development mode:

- run: npm run build
- run: npm start &  # Start production server

3. Use Consistent Hardware

CI runner performance varies. Use the same runner type for comparable results:

runs-on: ubuntu-latest  # Keep consistent

4. Set Appropriate Thresholds

Start with warnings, then tighten to errors:

• Week 1-4: Warnings only (learn your baseline)
• Month 2: Errors for critical metrics (LCP, CLS)
• Month 3: Errors for all budgets

5. Don't Block All PRs

• Block on: LCP > 4s, CLS > 0.25, bundle > 2x budget
• Warn on: Score decrease > 5 points, minor budget excess

Combine CI/CD with Monitoring

CI catches regressions before deploy. Monitoring catches issues in production. Use both.

Set up production monitoring →