Scale Overview

Plan around your limits, keep browsers fast, and decide whether a browser pool fits

Scaling an agent on KERNEL comes down to three questions, in this order: how many browsers you can run and create, how fast each one starts, and whether your workload needs a browser pool. Most workloads scale on on-demand browsers alone. This guide assumes you’re comfortable creating and controlling browsers.

1. Know your limits

Three limits shape a workload at scale, and each is covered in concurrency and limits:

• Concurrency: how many browsers can exist at once across your organization. Browsers in standby still count, so delete browsers when a task finishes to free the slot.
• Create rate: how fast you can create new browsers. Exceeding it returns 429 Too Many Requests, which the SDKs retry automatically.
• Per-browser resources: how much memory each browser has, which caps how many tabs and how heavy a page one browser can handle.

Each plan has set limits, and they go up when you upgrade your plan. Enterprise limits are custom. Use project concurrency limits to split one organization’s limit across teams or environments.

2. Keep browsers fast

A browser is created in about 30ms at P50 and 105ms at P99 (performance). Most slow starts come from configuration rather than load: custom viewports, extensions, and kiosk mode restart Chromium on creation and add seconds. Check troubleshooting latency before reaching for anything else, and run your code next to the browser with Playwright execution or the code execution platform to cut the time each action takes.

3. Decide between on-demand browsers and a browser pool

We recommend defaulting to on-demand browsers, both when you’re getting started and as you scale. Browser pools fit a specific type of workload, described below.

Stick with on-demand browsers.create() when:

• you’re still building
• your configuration changes per user (a pool is one fixed config)
• you need a GPU browser (not available in pools)

Reach for a browser pool when:

• you’ve built and scaled your workload, and every run uses the same workload attributes
• you’re hitting the browsers.create() rate limit at volume
• you need the lowest possible acquisition latency (for example, a heavily customized browser config)
• traffic is steady or high-frequency enough to keep the browser pool utilized

If you’re on an Enterprise plan, speak with your account manager about applicable rate limits for browsers.create() and what’s best for your workloads.

What a browser pool gives you

A browser pool keeps a set of identically-configured browsers ready for immediate use. Compared to creating browsers on demand, it gives you:

• Lowest-latency acquisition — the browser is already booted with your configuration applied (including settings like custom viewports, extensions, and kiosk-mode live view that otherwise restart Chromium on a fresh browser), so acquire hands you one that’s ready to drive.
• Reserved, pre-configured capacity — a fixed set of browsers on your exact configuration, ready before traffic arrives.
• Higher creation throughput — acquiring from a pool isn’t subject to the rate limit on browsers.create() that high-volume workloads hit.

The tradeoff: a browser pool counts against your concurrency limit whether or not its browsers are currently acquired — a pool sized to 40 holds 40 of your limit. Idle pooled browsers aren’t billed, but they hold the slot.

Sizing a pool

Watch available_count and target 10–20% available under normal load, resizing before traffic peaks rather than during them. See Sizing a browser pool for the full guidance.

Architecture patterns

On-demand creation

Creating a browser per task is the simplest approach. It’s the right fit while you’re building and when each task needs its own configuration.

When to use:

• Early development and testing
• Configuration that changes per user or per task
• GPU browsers

Example

import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';

const kernel = new Kernel();

async function processTask(taskData: any) {
  // Create browser with extended timeout for long tasks
  const session = await kernel.browsers.create({
    stealth: true,
    timeout_seconds: 3600, // Destroy browser after 1 hour of inactivity
  });

  try {
    const browser = await chromium.connectOverCDP(session.cdp_ws_url);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Your automation logic here
    await page.goto(taskData.url);
    // ... perform work ...

    return { success: true, data: /* results */ };
  } catch (error) {
    console.error('Task failed:', error);
    return { success: false, error: error.message };
  } finally {
    // Always clean up
    await kernel.browsers.deleteByID(session.session_id);
  }
}

Single browser pool

For production workloads that run on the same configuration every time, a browser pool hands you ready-to-drive browsers, and acquiring from it isn’t subject to the browsers.create() rate limit.

When to use:

• Consistent, high-frequency workloads on a fixed configuration
• Steady request patterns, or latency-sensitive acquisition

Example

import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';

const kernel = new Kernel();
const POOL_NAME = 'production-pool';

// Initialize pool once (typically in deployment/startup)
async function initializePool() {
  await kernel.browserPools.create({
    name: POOL_NAME,
    size: 25, // Balance cost and availability
    timeout_seconds: 300, // Destroy browsers after 5 minutes of inactivity
    stealth: true,
    headless: false, // headless: true for cost savings if no live view needed
  });
}

async function processTask(taskData: any) {
  let session;
  
  try {
    // Acquire browser (returns immediately if available)
    session = await kernel.browserPools.acquire(POOL_NAME, {
      acquire_timeout_seconds: 30, // Wait up to 30s for availability
    });

    const browser = await chromium.connectOverCDP(session.cdp_ws_url);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Perform work
    await page.goto(taskData.url);
    // ... automation logic ...

    return { success: true, data: /* results */ };
  } catch (error) {
    console.error('Task failed:', error);
    return { success: false, error: error.message };
  } finally {
    // Critical: Always release back to pool
    if (session) {
      await kernel.browserPools.release(POOL_NAME, {
        session_id: session.session_id,
        reuse: true, // Reuse for efficiency
      });
    }
  }
}

Key considerations:

• Pool size should match your typical concurrency
• Always release browsers in a finally block to prevent browser pool exhaustion
• Set acquire_timeout_seconds based on your SLA requirements

Queue-based processing

When request volume exceeds your concurrency or traffic arrives in unpredictable bursts, put a task queue in front of your browsers. The example below acquires from a browser pool; the same pattern works on demand, with browsers.create() in place of acquire and deleteByID in place of release.

When to use:

• Request volume exceeds your available concurrency
• Highly variable traffic patterns
• Need to prioritize certain tasks
• Want to decouple request ingestion from processing

Example

import { Queue, Worker } from 'bullmq'; // or any queue system
import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';

const kernel = new Kernel();
const POOL_NAME = 'production-pool';
const POOL_SIZE = 50;

// Task queue configuration
const taskQueue = new Queue('browser-tasks', {
  connection: { /* Redis config */ }
});

// Worker that processes tasks
async function startWorker() {
  const worker = new Worker('browser-tasks', async (job) => {
    let session;
    
    try {
      // Acquire browser with reasonable timeout
      session = await kernel.browserPools.acquire(POOL_NAME, {
        acquire_timeout_seconds: 120,
      });

      const browser = await chromium.connectOverCDP(session.cdp_ws_url);
      const context = browser.contexts()[0];
      const page = context.pages()[0];

      // Process job data
      await page.goto(job.data.url);
      const result = await page.evaluate(() => /* extract data */);

      return { success: true, data: result };
    } catch (error) {
      // Handle errors with retry logic
      if (error.message.includes('timeout')) {
        throw new Error('RETRY'); // BullMQ will retry
      }
      throw error;
    } finally {
      if (session) {
        await kernel.browserPools.release(POOL_NAME, {
          session_id: session.session_id,
          reuse: true,
        });
      }
    }
  }, {
    connection: { /* Redis config */ },
    concurrency: POOL_SIZE, // Match pool size
  });

  return worker;
}

// Add tasks to queue
async function submitTask(taskData: any, priority?: number) {
  await taskQueue.add('process', taskData, {
    priority: priority || 5,
    attempts: 3,
    backoff: {
      type: 'exponential',
      delay: 2000,
    },
  });
}

Queue-specific considerations:

• Set worker concurrency to match or slightly exceed browser pool size
• Implement proper retry logic for transient failures
• Monitor queue depth to scale browser pools dynamically
• Use priority queues for different SLAs