Synthetic monitoring in Next.js with SST, Pulumi and AWS

ยท8 minute read

When building web applications, monitoring uptime and functionality is crucial for maintaining a great user experience. In this post, we'll explore how to leverage Amazon CloudWatch Synthetics to monitor a Next.js application.

Canary

What are canaries?

CloudWatch Synthetics allows us to create canaries, automated tests that run continuously to monitor your endpoints and APIs. They can simulate user interactions, check for specific elements on your pages, and alert you when something goes wrong - all before your real users encounter issues.

The term "canary" originates from the historical practice of using canaries (meaning the actual bird) in coal mines to detect dangerous gases. Similarly, these synthetic tests serve as early warning systems for your applications.

Setting up the infrastructure

To make this post as broadly applicable as possible and avoiding vendor lock-in, we'll use SST to deploy our application instead of opting for solutions like Vercel or AWS Amplify. With SST being built on top of Pulumi/Terraform, we don't even need to opt for Next.js, but could go with any other framework like Astro or Svelte.

Let's start with a basic sst.config.ts file. Your actual config will most likely look a bit different, but let's keep it simple for now.

// sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />

const domain = 'example.com';

export default $config({
  // ...
  async run() {
    const vpc = new sst.aws.Vpc('MyVpc');
    const cluster = new sst.aws.Cluster('MyCluster', { vpc });

    const service = new sst.aws.Service('MyService', {
      cluster,
      loadBalancer: {
        rules: [{ listen: '443/https', forward: '3000/http' }],
        domain,
      },
    });
  },
});

I've opted for a container based setup using Amazon ECS, but you can of course also go the serverless way. For more information checkout the SST docs.

Next, let's create a custom Canary class that extends the AWS Synthetics Canary to handle the build and deployment process. This is where we'll bundle our TypeScript code and create a zip file that can be deployed to AWS Lambda.

// helper/canary.ts
import * as aws from '@pulumi/aws';
import * as pulumi from '@pulumi/pulumi';
import * as esbuild from 'esbuild';
import * as fflate from 'fflate';
import * as fs from 'fs';
import * as path from 'path';
import * as rimraf from 'rimraf';

class Canary extends aws.synthetics.Canary {
  constructor(
    name: string,
    bucketId: pulumi.Output<string>,
    domainName: string,
    executionRoleArn: pulumi.Output<string>,
  ) {
    const zipFile = 'canary.zip';
    const outdir = path.resolve(process.cwd(), 'pulumi-canary/');

    if (!fs.existsSync(outdir)) {
      fs.mkdirSync(outdir);
    }

    // Clean previous builds
    rimraf.rimrafSync(outdir);
    rimraf.rimrafSync(zipFile);

    // Bundle the canary code using esbuild
    esbuild.buildSync({
      entryPoints: [path.join(__dirname, '../../synthetics/monitor.ts')],
      bundle: true,
      minify: true,
      sourcemap: false,
      platform: 'node',
      target: 'node20',
      outdir,
      external: ['Synthetics', 'SyntheticsLogger', 'puppeteer'],
    });

    // Get the bundled output file
    const [outputFile] = fs.readdirSync(outdir);

    // Create zip archive for Lambda deployment
    const zipContent = fflate.zipSync({
      'nodejs/node_modules/monitor.js': fs.readFileSync(path.resolve(outdir, outputFile)),
    });

    fs.writeFileSync(path.resolve(outdir, zipFile), zipContent);
    const code = new pulumi.asset.FileArchive(path.resolve(outdir, zipFile));

    // Create the canary
    super(name, {
      name: 'my-next-app-canary',
      runtimeVersion: 'syn-nodejs-puppeteer-10.0',
      schedule: {
        expression: 'rate(15 minutes)', // Run every 15 minutes
      },
      handler: 'monitor.handler',
      artifactS3Location: pulumi.interpolate`s3://${bucketId}`,
      runConfig: {
        environmentVariables: {
          WEBAPP_URL: `https://${domainName}/`,
        },
      },
      executionRoleArn,
      startCanary: true,
      zipFile: code.path,
    });
  }
}

To make use of it, create a setupCanary.ts helper file.

// helper/setupCanary.ts
import * as aws from '@pulumi/aws';
import * as pulumi from '@pulumi/pulumi';

import { Canary } from './canary';
import { generateCanaryPolicy } from './canary-policy';

export async function setupCanary(domainName: string) {
  // S3 bucket to store canary execution results
  const canaryResultsBucket = new aws.s3.BucketV2('canary-results', {
    bucket: `canary-results`,
    forceDestroy: true,
  });

  // IAM role for canary execution
  const canaryExecutionRole = new aws.iam.Role('canary-exec-role', {
    assumeRolePolicy: {
      Version: '2012-10-17',
      Statement: [
        {
          Action: 'sts:AssumeRole',
          Effect: 'Allow',
          Principal: {
            Service: 'lambda.amazonaws.com',
          },
        },
      ],
    },
  });

  const region = await aws.getRegion({});
  const accountId = (await aws.getCallerIdentity({})).accountId;

  // Attach policy to the canary execution role
  new aws.iam.RolePolicy('canary-exec-policy', {
    role: canaryExecutionRole.id,
    policy: canaryResultsBucket.arn.apply((arn) =>
      generateCanaryPolicy(arn, region.name, accountId),
    ),
  });

  // Create the actual canary
  new Canary('canary-webapp', canaryResultsBucket.id, domainName, canaryExecutionRole.arn);
}

This helper function creates all the essential components for our canary:

  • S3 bucket for storing canary execution artifacts
  • IAM role with appropriate permissions
  • The canary itself

You may easily extend this function to include more resources, such as a CloudWatch alarm to get notified whenever a failure occurs.

Before we make use of the setupCanary function, let's first create the missing canary-policy.ts file to ensure we have the necessary IAM permissions for the canary to function properly.

// helper/canary-policy.ts
export function generateCanaryPolicy(
  canaryResultsBucketArn: string,
  regionName: string,
  accountId: string,
) {
  return JSON.stringify({
    Version: '2012-10-17',
    Statement: [
      {
        Effect: 'Allow',
        Action: ['s3:PutObject', 's3:GetObject'],
        Resource: [`${canaryResultsBucketArn}/*`],
      },
      {
        Effect: 'Allow',
        Action: ['s3:GetBucketLocation'],
        Resource: [canaryResultsBucketArn],
      },
      {
        Effect: 'Allow',
        Action: ['logs:CreateLogStream', 'logs:PutLogEvents', 'logs:CreateLogGroup'],
        Resource: [`arn:aws:logs:${regionName}:${accountId}:log-group:/aws/lambda/cwsyn-*`],
      },
      {
        Effect: 'Allow',
        Action: ['s3:ListAllMyBuckets', 'xray:PutTraceSegments'],
        Resource: ['*'],
      },
      {
        Effect: 'Allow',
        Resource: '*',
        Action: 'cloudwatch:PutMetricData',
        Condition: {
          StringEquals: {
            'cloudwatch:namespace': 'CloudWatchSynthetics',
          },
        },
      },
    ],
  });
}

This policy grants the canary permission to:

  • Store results in S3
  • Write to CloudWatch Logs
  • Send metrics to CloudWatch
  • Use X-Ray for tracing

Now, let's finally integrate the canary into our sst.config.ts file.

// sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />

const domain = 'example.com';

export default $config({
  async run() {
    const { setupCanary } = await import('./helper/setupCanary'); 

    const vpc = new sst.aws.Vpc('MyVpc');
    const cluster = new sst.aws.Cluster('MyCluster', { vpc });

    const service = new sst.aws.Service('MyService', {
      cluster,
      loadBalancer: {
        rules: [{ listen: '443/https', forward: '3000/http' }],
        domain,
      },
    });

    await setupCanary(domain); 
  },
});

The monitoring script

Last but not least, let's create some really simple monitoring script that loads our Next.js app and checks if a specific element is present. Since this is Typescript, you'll also need to provide a proper build setup, but I'll omit this to not make this post any longer.

// synthetics/monitor.ts
// "Synthetics" is provided by the AWS runtime
import synthetics from 'Synthetics';
import { get as getEnv } from 'env-var';
import { Page } from 'puppeteer';

export const handler = async () => {
  const baseUrl = getEnv('WEBAPP_URL').required().asUrlObject();
  const url = baseUrl.href;

  // Get a Puppeteer page instance managed by AWS Synthetics
  const page: Page = await synthetics.getPage();

  await synthetics.executeStep('Load App', async () => {
    // Navigate to the application
    await page.goto(url, {
      waitUntil: ['load', 'networkidle0'],
    });

    // Check for a specific element that indicates the app loaded correctly
    await page.waitForSelector('div[data-testid=element-to-check]');
  });
};

To make the types of the AWS provided Synthetics library work, let's create a Synthetics.d.ts file.

// synthetics/types/Synthetics.d.ts
export declare module './Synthetics' {
  import type { Page } from 'puppeteer';

  export declare interface StepConfig {
    continueOnStepFailure?: boolean;
    screenshotOnStepStart?: boolean;
    screenshotOnStepSuccess?: boolean;
    screenshotOnStepFailure?: boolean;
    harFile?: boolean;
  }
  export declare function executeStep(
    stepName: string,
    functionToExecute: (timeoutInMillis: number) => Promise<unknown>,
    stepConfig?: StepConfig,
  ): Promise<void>;

  export declare function getPage(): Promise<Page>;
}

And we're done! By combining SST, Pulumi, and AWS Synthetics, we've created a robust monitoring solution that:

  • Automatically builds and deploys monitoring scripts
  • Runs comprehensive tests at a set interval
  • Tracks performance and reliability trends over time
  • Integrates seamlessly with existing SST-based infrastructure