Synthetic monitoring in Next.js with SST, Pulumi and AWS
When building web applications, monitoring uptime and functionality is crucial for maintaining a great user experience. In this post, we'll explore how to leverage Amazon CloudWatch Synthetics to monitor a Next.js application.
What are canaries?
CloudWatch Synthetics allows us to create canaries, automated tests that run continuously to monitor your endpoints and APIs. They can simulate user interactions, check for specific elements on your pages, and alert you when something goes wrong - all before your real users encounter issues.
The term "canary" originates from the historical practice of using canaries (meaning the actual bird) in coal mines to detect dangerous gases. Similarly, these synthetic tests serve as early warning systems for your applications.
Setting up the infrastructure
To make this post as broadly applicable as possible and avoiding vendor lock-in, we'll use SST to deploy our application instead of opting for solutions like Vercel or AWS Amplify. With SST being built on top of Pulumi/Terraform, we don't even need to opt for Next.js, but could go with any other framework like Astro or Svelte.
Let's start with a basic sst.config.ts
file. Your actual config will most likely look a bit
different, but let's keep it simple for now.
// sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />
const domain = 'example.com';
export default $config({
// ...
async run() {
const vpc = new sst.aws.Vpc('MyVpc');
const cluster = new sst.aws.Cluster('MyCluster', { vpc });
const service = new sst.aws.Service('MyService', {
cluster,
loadBalancer: {
rules: [{ listen: '443/https', forward: '3000/http' }],
domain,
},
});
},
});
I've opted for a container based setup using Amazon ECS, but you can of course also go the serverless way. For more information checkout the SST docs.
Next, let's create a custom Canary
class that extends the AWS Synthetics Canary to handle the
build and deployment process. This is where we'll bundle our TypeScript code and create a zip file
that can be deployed to AWS Lambda.
// helper/canary.ts
import * as aws from '@pulumi/aws';
import * as pulumi from '@pulumi/pulumi';
import * as esbuild from 'esbuild';
import * as fflate from 'fflate';
import * as fs from 'fs';
import * as path from 'path';
import * as rimraf from 'rimraf';
class Canary extends aws.synthetics.Canary {
constructor(
name: string,
bucketId: pulumi.Output<string>,
domainName: string,
executionRoleArn: pulumi.Output<string>,
) {
const zipFile = 'canary.zip';
const outdir = path.resolve(process.cwd(), 'pulumi-canary/');
if (!fs.existsSync(outdir)) {
fs.mkdirSync(outdir);
}
// Clean previous builds
rimraf.rimrafSync(outdir);
rimraf.rimrafSync(zipFile);
// Bundle the canary code using esbuild
esbuild.buildSync({
entryPoints: [path.join(__dirname, '../../synthetics/monitor.ts')],
bundle: true,
minify: true,
sourcemap: false,
platform: 'node',
target: 'node20',
outdir,
external: ['Synthetics', 'SyntheticsLogger', 'puppeteer'],
});
// Get the bundled output file
const [outputFile] = fs.readdirSync(outdir);
// Create zip archive for Lambda deployment
const zipContent = fflate.zipSync({
'nodejs/node_modules/monitor.js': fs.readFileSync(path.resolve(outdir, outputFile)),
});
fs.writeFileSync(path.resolve(outdir, zipFile), zipContent);
const code = new pulumi.asset.FileArchive(path.resolve(outdir, zipFile));
// Create the canary
super(name, {
name: 'my-next-app-canary',
runtimeVersion: 'syn-nodejs-puppeteer-10.0',
schedule: {
expression: 'rate(15 minutes)', // Run every 15 minutes
},
handler: 'monitor.handler',
artifactS3Location: pulumi.interpolate`s3://${bucketId}`,
runConfig: {
environmentVariables: {
WEBAPP_URL: `https://${domainName}/`,
},
},
executionRoleArn,
startCanary: true,
zipFile: code.path,
});
}
}
To make use of it, create a setupCanary.ts
helper file.
// helper/setupCanary.ts
import * as aws from '@pulumi/aws';
import * as pulumi from '@pulumi/pulumi';
import { Canary } from './canary';
import { generateCanaryPolicy } from './canary-policy';
export async function setupCanary(domainName: string) {
// S3 bucket to store canary execution results
const canaryResultsBucket = new aws.s3.BucketV2('canary-results', {
bucket: `canary-results`,
forceDestroy: true,
});
// IAM role for canary execution
const canaryExecutionRole = new aws.iam.Role('canary-exec-role', {
assumeRolePolicy: {
Version: '2012-10-17',
Statement: [
{
Action: 'sts:AssumeRole',
Effect: 'Allow',
Principal: {
Service: 'lambda.amazonaws.com',
},
},
],
},
});
const region = await aws.getRegion({});
const accountId = (await aws.getCallerIdentity({})).accountId;
// Attach policy to the canary execution role
new aws.iam.RolePolicy('canary-exec-policy', {
role: canaryExecutionRole.id,
policy: canaryResultsBucket.arn.apply((arn) =>
generateCanaryPolicy(arn, region.name, accountId),
),
});
// Create the actual canary
new Canary('canary-webapp', canaryResultsBucket.id, domainName, canaryExecutionRole.arn);
}
This helper function creates all the essential components for our canary:
- S3 bucket for storing canary execution artifacts
- IAM role with appropriate permissions
- The canary itself
You may easily extend this function to include more resources, such as a CloudWatch alarm to get notified whenever a failure occurs.
Before we make use of the setupCanary
function, let's first create the missing canary-policy.ts
file to ensure we have the necessary IAM permissions for the canary to function properly.
// helper/canary-policy.ts
export function generateCanaryPolicy(
canaryResultsBucketArn: string,
regionName: string,
accountId: string,
) {
return JSON.stringify({
Version: '2012-10-17',
Statement: [
{
Effect: 'Allow',
Action: ['s3:PutObject', 's3:GetObject'],
Resource: [`${canaryResultsBucketArn}/*`],
},
{
Effect: 'Allow',
Action: ['s3:GetBucketLocation'],
Resource: [canaryResultsBucketArn],
},
{
Effect: 'Allow',
Action: ['logs:CreateLogStream', 'logs:PutLogEvents', 'logs:CreateLogGroup'],
Resource: [`arn:aws:logs:${regionName}:${accountId}:log-group:/aws/lambda/cwsyn-*`],
},
{
Effect: 'Allow',
Action: ['s3:ListAllMyBuckets', 'xray:PutTraceSegments'],
Resource: ['*'],
},
{
Effect: 'Allow',
Resource: '*',
Action: 'cloudwatch:PutMetricData',
Condition: {
StringEquals: {
'cloudwatch:namespace': 'CloudWatchSynthetics',
},
},
},
],
});
}
This policy grants the canary permission to:
- Store results in S3
- Write to CloudWatch Logs
- Send metrics to CloudWatch
- Use X-Ray for tracing
Now, let's finally integrate the canary into our sst.config.ts
file.
// sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />
const domain = 'example.com';
export default $config({
async run() {
const { setupCanary } = await import('./helper/setupCanary');
const vpc = new sst.aws.Vpc('MyVpc');
const cluster = new sst.aws.Cluster('MyCluster', { vpc });
const service = new sst.aws.Service('MyService', {
cluster,
loadBalancer: {
rules: [{ listen: '443/https', forward: '3000/http' }],
domain,
},
});
await setupCanary(domain);
},
});
The monitoring script
Last but not least, let's create some really simple monitoring script that loads our Next.js app and checks if a specific element is present. Since this is Typescript, you'll also need to provide a proper build setup, but I'll omit this to not make this post any longer.
// synthetics/monitor.ts
// "Synthetics" is provided by the AWS runtime
import synthetics from 'Synthetics';
import { get as getEnv } from 'env-var';
import { Page } from 'puppeteer';
export const handler = async () => {
const baseUrl = getEnv('WEBAPP_URL').required().asUrlObject();
const url = baseUrl.href;
// Get a Puppeteer page instance managed by AWS Synthetics
const page: Page = await synthetics.getPage();
await synthetics.executeStep('Load App', async () => {
// Navigate to the application
await page.goto(url, {
waitUntil: ['load', 'networkidle0'],
});
// Check for a specific element that indicates the app loaded correctly
await page.waitForSelector('div[data-testid=element-to-check]');
});
};
To make the types of the AWS provided Synthetics library work, let's create a Synthetics.d.ts
file.
// synthetics/types/Synthetics.d.ts
export declare module './Synthetics' {
import type { Page } from 'puppeteer';
export declare interface StepConfig {
continueOnStepFailure?: boolean;
screenshotOnStepStart?: boolean;
screenshotOnStepSuccess?: boolean;
screenshotOnStepFailure?: boolean;
harFile?: boolean;
}
export declare function executeStep(
stepName: string,
functionToExecute: (timeoutInMillis: number) => Promise<unknown>,
stepConfig?: StepConfig,
): Promise<void>;
export declare function getPage(): Promise<Page>;
}
And we're done! By combining SST, Pulumi, and AWS Synthetics, we've created a robust monitoring solution that:
- Automatically builds and deploys monitoring scripts
- Runs comprehensive tests at a set interval
- Tracks performance and reliability trends over time
- Integrates seamlessly with existing SST-based infrastructure