All case studies

AWS Lambda Ticket Scraper: 5 Minutes → 15 Seconds

Removing unnecessary browser overhead and sequential execution from a scheduled ticket scraper.

Independent client work · Ticket scraper · 2023

Node.jsAWS LambdaAxiosCheerio

I replaced the sequential Puppeteer implementation with Axios/Cheerio and concurrent scraping, delivering the revised Node.js process on AWS Lambda.

THE OUTCOME5 minutes → 15 seconds

The problem

A ticket scraper needed to collect information from three additional websites. The existing Puppeteer implementation ran sequentially, took about five minutes and often failed before finishing. It ran every two hours.

The scraper ran on AWS Lambda. The target pages could be fetched and parsed without a browser.

What I did

I assessed the existing Node.js implementation and the information needed from the target websites. The key decisions were whether a browser was actually required and which scraping operations could run concurrently.

I replaced Puppeteer with Axios for HTTP requests and Cheerio for HTML parsing, and ran scraping work concurrently.

Technical details

The new implementation fetched pages directly with Axios and extracted information using Cheerio.

Independent scraping operations ran concurrently, reducing overall execution time and avoiding unnecessary browser startup overhead.

Workflow overview

Before

  1. Scheduled Lambda
  2. Sequential browser tasks
  3. Extract ticket information
  4. ~5 minutes

After

  1. Scheduled Lambda
  2. Concurrent Axios requests
  3. Cheerio extraction
  4. ~15 seconds

Direct HTTP fetching avoids browser overhead when target websites do not require JavaScript execution.

Browser automation remains appropriate for dynamically rendered websites, while concurrent requests must respect upstream service limits.

The result

Execution time improved from approximately five minutes to 15 seconds for this scraper. The client described the revised Node.js code as nearly 20× faster.

More case studies.

All case studies

Facing a similar engineering challenge?

I help companies solve difficult backend, data and AI integration problems.

Book a technical call
Hiring a senior backend, applied AI or forward-deployed engineer?Explore my experienceDownload résumé