How to Build a Serverless Screenshot API

Create a Scalable Screenshot Service Using AWS Lambda, Puppeteer, and TypeScript.

Published on December 6, 2024

In this article, we will explore how to build a serverless screenshot API that can handle millions of requests at scale. We'll dive deep into using AWS Lambda with Puppeteer to create a robust, scalable screenshot service without managing traditional servers.

Architecture Overview

The power of our screenshot API lies in its serverless architecture, which automatically scales based on demand and only charges for actual usage. Before diving into the implementation, let's understand how each component works together:

Next.js API Route: Acts as our frontend interface, handling request validation, authentication, and communication with AWS Lambda. It provides a clean, type-safe API for clients.

AWS Lambda: Powers the core screenshot functionality using Puppeteer and headless Chrome, scaling automatically with demand.

S3 Storage: Provides reliable, scalable storage for screenshots with high durability.

CloudFront CDN: Ensures fast global delivery of screenshots through edge locations.

Lambda Layers: Manages dependencies like Chromium and Sharp efficiently.

Setting Up Lambda Layers

One of the key challenges in running Puppeteer on AWS Lambda is managing the Chromium binary and its dependencies. Lambda layers solve this elegantly by allowing us to package these dependencies separately from our function code.

We use two essential layers:

Chromium Layer: Contains the Lambda-optimized Chromium binary required for Puppeteer

Sharp Layer: Provides image processing capabilities for optimizing screenshots

Here's how we create and deploy the Chromium layer:

#!/bin/bash

BUCKET_NAME="your-bucket"
LAYER_NAME="chromium-layer"
VERSION="131.0.0"
REGION="us-east-1"

# Download Chromium layer
curl -L -o "chromium-layer.zip" \
  "https://github.com/Sparticuz/chromium/releases/download/v${VERSION}/chromium-v${VERSION}-layer.zip"

# Upload to S3
aws s3 cp "chromium-layer.zip" "s3://${BUCKET_NAME}/layers/"

# Create Lambda layer
aws lambda publish-layer-version \
  --layer-name "${LAYER_NAME}" \
  --content "S3Bucket=${BUCKET_NAME},S3Key=layers/chromium-layer.zip" \
  --compatible-runtimes nodejs20.x \
  --compatible-architectures x86_64

This script handles downloading the Lambda-optimized Chromium binary, packaging it appropriately, and creating a Lambda layer. The layer is specifically configured for Node.js 20.x and x86_64 architecture to ensure optimal performance.

Lambda Function Implementation

The Lambda function is where the magic happens. It handles screenshot capture using Puppeteer, a powerful library for controlling headless Chrome. Let's break down the key components:

The function accepts parameters for customizing the screenshot:

url: The webpage to capture

width/height: Viewport dimensions

fullPage: Whether to capture the entire scrollable page

import chromium from '@sparticuz/chromium';
import puppeteer from 'puppeteer-core';
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { v4 as uuidv4 } from 'uuid';

const s3Client = new S3Client({ region: process.env.AWS_REGION });

export const handler = async (event) => {
  try {
    const { url, width = 1440, height = 900, fullPage = false } = JSON.parse(event.body);
    
    const browser = await puppeteer.launch({
      args: chromium.args,
      defaultViewport: { width, height },
      executablePath: await chromium.executablePath(),
      headless: chromium.headless,
    });

    const page = await browser.newPage();
    await page.goto(url, { waitUntil: 'networkidle0' });
    
    const screenshot = await page.screenshot({
      type: 'jpeg',
      fullPage,
      quality: 90
    });

    await browser.close();

    const filename = `screenshots/${uuidv4()}.jpeg`;
    
    await s3Client.send(new PutObjectCommand({
      Bucket: process.env.BUCKET_NAME,
      Key: filename,
      Body: screenshot,
      ContentType: 'image/jpeg'
    }));

    return {
      statusCode: 200,
      body: JSON.stringify({
        url: `${process.env.CDN_DOMAIN}/${filename}`
      })
    };
  } catch (error) {
    console.error('Error:', error);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: 'Failed to generate screenshot' })
    };
  }
}

The function launches a headless Chrome instance using Lambda-optimized settings, navigates to the specified URL, captures the screenshot, and uploads it to S3. Error handling ensures graceful failure recovery.

Next.js API Route

Our Next.js API route provides a clean interface for clients while handling important aspects like request validation and authentication. It uses Zod for robust type validation and includes built-in rate limiting and quota management.

Key features of the API route include:

Strong type validation using Zod schema

API key authentication for security

Error handling and response formatting

Proper Lambda function invocation

// app/api/screenshot/route.ts
import { validateApiKey } from '@/lib/auth';
import { InvokeCommand } from '@aws-sdk/client-lambda';
import { lambda } from '@/lib/lambda';
import { NextRequest, NextResponse } from 'next/server';
import { z } from 'zod';

const QuerySchema = z.object({
  accessKey: z.string().min(1, "Access key is required"),
  url: z.string().url(),
  width: z.coerce.number().default(1440),
  height: z.coerce.number().default(900),
  fullPage: z.coerce.boolean().default(false)
});

export async function POST(request: NextRequest) {
  try {
    const params = await request.json();
    const query = QuerySchema.parse(params);

    // Validate access key
    const isValid = await validateApiKey(query.accessKey);
    if (!isValid) {
      return NextResponse.json(
        { error: "Invalid access key" },
        { status: 401 }
      );
    }

    const command = new InvokeCommand({
      FunctionName: process.env.SCREENSHOT_LAMBDA_NAME!,
      Payload: JSON.stringify({ body: JSON.stringify(query) })
    });

    const response = await lambda.send(command);
    const result = JSON.parse(Buffer.from(response.Payload!).toString());

    return NextResponse.json(result);
  } catch (error) {
    console.error("Screenshot API error:", error);
    return NextResponse.json(
      { error: "Failed to generate screenshot" },
      { status: 500 }
    );
  }
}

The route validates incoming requests, checks authentication, and forwards valid requests to our Lambda function. It also handles error cases and formats responses consistently.

Serverless Configuration

The serverless.yml file defines our infrastructure as code, specifying how our Lambda function should be deployed and configured. This approach ensures consistent deployments and makes it easy to manage environment-specific settings.

Key configuration aspects include:

Memory Allocation: 8GB RAM to handle Chrome's requirements

Timeout: 120 seconds for processing complex pages

Environment Variables: Configuration for S3 and CDN

Lambda Layers: Integration of Chromium and Sharp

service: screenshot-lambda

provider:
  name: aws
  runtime: nodejs20.x
  memorySize: 8192
  timeout: 120
  environment:
    NODE_ENV: production
    BUCKET_NAME: ${env:BUCKET_NAME}
    CDN_DOMAIN: ${env:CDN_DOMAIN}

functions:
  screenshot:
    handler: handler.handler
    layers:
      - arn:aws:lambda:us-east-1:YOUR_ACCOUNT:layer:chromium-layer:1
      - arn:aws:lambda:us-east-1:YOUR_ACCOUNT:layer:sharp-layer:1
    events:
      - httpApi:
          path: /screenshot
          method: post

Production Considerations

When deploying this system to production, several key aspects need attention:

Memory Management

Chromium in Lambda requires careful memory management. Our implementation uses several strategies:

Configuring 8GB RAM for reliable operation

Proper cleanup after each screenshot

Monitoring memory usage patterns

Error Handling

Robust error handling is crucial for a production service:

Graceful handling of network timeouts

Recovery from browser crashes

Proper error reporting and logging

Automatic retries for transient failures

Performance Optimization

Several techniques improve performance:

CDN caching for frequent screenshots

Browser warm-up and reuse

Optimized Chromium flags

Efficient image compression

Conclusion

Building a serverless screenshot API with AWS Lambda and Puppeteer provides a scalable, cost-effective solution for capturing web pages. While the implementation requires careful consideration of various technical aspects, the resulting system can handle millions of requests efficiently.

For those looking to avoid the complexity of building and maintaining such a system, our Screenshot API service provides all these capabilities out of the box, with additional features like advanced caching, global CDN distribution, and enterprise-grade reliability.