Arcjet example: Next.js bot categories

Arcjet helps developers protect their apps in just a few lines of code. Bot detection. Rate limiting. Email validation. Attack protection. Data redaction. A developer-first approach to security.

This is an example Next.js application demonstrating advanced Arcjet bot detection. It shows how to build a bot allow list by category, by individual bot, and by filtering an individual bot out of a category so it is still denied.

Features

• Bot protection allowing bots by category, by individual bot, and by filtering an individual bot out of a category.
• Bot verification via @arcjet/inspect to detect spoofed bots.

Run locally

1. Register for a free Arcjet account.

2. Install dependencies:

npm ci

3. Rename .env.local.example to .env.local and add your Arcjet key. The example file sets ARCJET_DEMO_BOT_HEADERS=1 so the curl steps below show the diagnostic headers. Leave it unset (or 0) to keep them off.

4. Start the dev server

npm run dev

5. Open http://localhost:3000 in your browser.

Try it

The /api/arcjet route is protected by the bot detection rule configured in lib/arcjet.ts.

Warning

Diagnostic bot headers are off unless ARCJET_DEMO_BOT_HEADERS=1. When that flag is set, the route echoes X-Arcjet-Bot-Allowed and X-Arcjet-Bot-Denied so you can see which identifiers matched. That disclosure can help someone tune evasion. Do not enable those headers on a production API.

1. Request the API as curl, which belongs to CATEGORY:TOOL and is allowed:

   curl -v localhost:3000/api/arcjet

   The response headers show curl was detected and allowed because of CATEGORY:TOOL:

   x-arcjet-bot-allowed: CATEGORY:TOOL, CURL
   x-arcjet-bot-denied:

2. Change the User-Agent to Vercel's screenshot bot, which is allowed as an individual bot even though its category is not:

   curl -v -A "vercel-screenshot" localhost:3000/api/arcjet

   x-arcjet-bot-allowed: VERCEL_MONITOR_PREVIEW
   x-arcjet-bot-denied:

3. Finally, pretend to be Google's AdsBot. It is denied because we programmatically filtered it out of CATEGORY:GOOGLE, which expands the category into all its individual bots:

   curl -v -A "AdsBot-Google" localhost:3000/api/arcjet

   x-arcjet-bot-allowed:
   x-arcjet-bot-denied: GOOGLE_ADSBOT

Need help?

Check out the docs, contact support, or join our Discord server.

Contributing

All development for Arcjet examples is done in the arcjet/examples repository.

You are welcome to open an issue here or in arcjet/examples directly. However, please direct all pull requests to arcjet/examples. Take a look at our contributing guide for more information.