Skip to main content

API Throughput

ServiceNow imposes two restrictions that limit API throughput. Understanding these restrictions helps you configure both your API client and your ServiceNow instance for optimal performance.

The Two Restrictions​

RestrictionWhat It LimitsImpact on SquidSolution
Session synchronizationOne request at a time per sessionParallel requests queue up and may be silently droppedDisable cookies in your API client
Semaphore limitsConcurrent API requests per nodeRequests rejected with HTTP 429 when limit exceededEnsure proper server configuration

Session Synchronization​

ServiceNow processes only one transaction per session at a time, preventing any single user from monopolizing server resources.

How It Affects Squid​

When your HTTP client persists cookies, all your requests share the same session. ServiceNow queues these requests and processes them one by one—even if you send them in parallel.

ServiceNow Session SynchronizationServiceNow session synchronization (session synch) can severely impact Squid API throughput. This diagram compares request handling with and without session cookies. WITHOUT COOKIES (RECOMMENDED): When the API client does not persist session cookies (JSESSIONID and BIGipServerpool), each request is treated as independent. The load balancer distributes requests across all available application nodes (Node 1, Node 2, Node 3). Each node processes requests in parallel using separate sessions. Result: All requests execute concurrently, maximizing throughput. Response: HTTP 200 for all requests processed in parallel. WITH COOKIES (PROBLEMATIC): When the API client persists session cookies, session affinity locks all requests to a single node. The BIGipServerpool cookie routes all requests to the same node (sticky session). ServiceNow's session synchronization serializes all requests for the same JSESSIONID. Requests queue up: only one processes at a time while others wait. Other nodes sit idle despite being available. If more than 10 requests queue for the same session, ServiceNow returns HTTP 202 (request accepted but not yet processed) or rejects requests entirely. CRITICAL NUMBERS: Session cookies involved are JSESSIONID (session identifier) and BIGipServerpool_* (load balancer affinity). Integration session timeout is 5 minutes (vs 30 minutes for UI sessions). Maximum waiters per session is 10 (Tokyo release and later). Node queue capacity is 150 requests total. RECOMMENDATION: Configure your HTTP client to NOT persist session cookies when calling Squid. This allows requests to be distributed across the ServiceNow cluster for parallel processing instead of being serialized through a single session.API Client(Squid / Integration)WITHOUT COOKIESWITH COOKIESLoad BalancerDistributes to any nodeLoad BalancerRoutes via BIGipServerpool cookieServiceNow ClusterNode 1Session AprocessingNode 2Session BprocessingNode 3Session CprocessingParallel ProcessingRequests execute concurrently across nodesServiceNow ClusterNode 1 (Sticky)JSESSIONID: abc123SESSION SYNCHProcessing: Req #1Waiting: #2, #3, #4...Node 2idleNode 3idleSerialized QueueOne request at a timeHTTP 200 (All Requests)All requests processed in parallelHTTP 202 Risk (Queue Full)Requests rejected if >10 waiters queuedSession Cookies:JSESSIONID + BIGipServerpool_*Integration Timeout:5 min (vs. UI: 30 min)Max Waiters:10 per session (Tokyo+)Node Queue:150 requests

Without cookies (left side): Each request creates a new session. The load balancer distributes requests across all cluster nodes, and they process in parallel.

With cookies (right side): All requests stick to one node and one session. They queue behind each other, and if more than 10 requests queue up, ServiceNow returns HTTP 202—meaning your request was silently dropped.

Why This Matters for Squid​

Squid queries routinely take several minutes for large datasets. A single long-running query would block all other requests from your integration if session cookies are enabled. Worse, additional requests may receive HTTP 202 and be silently discarded.

ScenarioWith CookiesWithout Cookies
Three parallel queriesExecute one at a time (serial)Execute simultaneously (parallel)
One 5-minute query + quick queryQuick query waits 5 minutesQuick query completes immediately
15 parallel requests5+ silently dropped (HTTP 202)All processed

The Solution​

Disable cookie persistence in your HTTP client. This ensures each request gets its own session and can process independently. See API Client Configuration for code examples in Python, JavaScript, .NET, and other languages.

Why this is safe for the server

Creating a new session per request does consume server memory (~20KB per session). However, ServiceNow specifically addressed this for integrations: since Fuji Patch 7, integration sessions have a 5-minute timeout (compared to 30 minutes for UI sessions). This short timeout means sessions from API traffic expire quickly and don't accumulate. With proper timeout configuration, the memory overhead is negligible compared to the performance benefits of parallel processing.

Semaphore Limits​

ServiceNow limits how many API requests can run concurrently on each application node. Requests beyond this limit are queued briefly, then rejected with HTTP 429 if the queue fills up.

How It Affects Squid​

The REST API semaphore pool (called "API_INT") allows only 4 concurrent requests per node with a queue depth of 50.

ServiceNow API Semaphore LimitsServiceNow uses semaphore pools to limit concurrent API request processing. Understanding these limits is essential for Squid integrations that make many concurrent requests. API_INT SEMAPHORE POOL: Each ServiceNow application node has an API_INT semaphore pool that controls inbound REST API request processing. The pool has two components: Active Threads (4 maximum per node by default) - these are the worker threads actively processing API requests. Request Queue (50 maximum depth by default) - requests wait here when all active threads are busy. NORMAL OPERATION: When a request arrives and an active thread is available, it processes immediately and returns HTTP 200. When all 4 active threads are busy, new requests queue up (positions 5 through 54). Queued requests are processed in order as threads become available. CAPACITY EXCEEDED: When the queue is full (54 total requests: 4 active + 50 queued) and request #55 arrives, it cannot be queued. ServiceNow returns HTTP 429 Too Many Requests. The client should implement retry logic with exponential backoff. IMPLICATIONS FOR SQUID: If your integration sends more than 54 concurrent requests to a single node, you will receive HTTP 429 errors. Solutions include: (1) Implement retry logic with backoff, (2) Limit concurrent requests from your client, (3) Distribute requests across the ServiceNow cluster by not using session cookies (see session synchronization diagram), (4) Request increased semaphore limits from ServiceNow (requires justification). CONFIGURATION: The default values (4 active threads, 50 queue depth) can be adjusted by ServiceNow administrators through system properties, but increases require careful capacity planning.API Requests(from Squid / integrations)ServiceNow Application NodeAPI_INT SEMAPHORE POOLActive Threads(4 max per node)Thread 1Thread 2Thread 3Thread 4Request Queue(50 max depth)Request #5Request #6... up to #54WHEN CAPACITY EXCEEDEDQueue Full (54 requests)Request #55HTTP 429 Too Many RequestsHTTP 200 SuccessRequest processed normallyCapacity ErrorReview server configuration

When all threads are busy and the queue is full, additional requests are rejected with HTTP 429. A typical ServiceNow cluster with 4 nodes can therefore handle:

  • 16 concurrent API requests (4 nodes × 4 threads)
  • 200 queued requests (4 nodes × 50 queue depth)

Why This Matters for Squid​

Unlike session synchronization (which you solve by disabling cookies), semaphore limits are platform-wide and affect all API consumers. You cannot avoid them—but you can handle them gracefully.

Squid's long-running queries occupy semaphores for extended periods, reducing capacity for other requests. During busy periods, you may encounter HTTP 429 responses.

The Solution​

HTTP 429 indicates your ServiceNow instance lacks sufficient capacity for the current load. The solution is proper server configuration—not client-side retry logic, which adds unnecessary complexity to your integration.

Work with your ServiceNow administrator to:

  • Increase transaction quota timeouts for Squid API requests
  • Configure rate limit rules that accommodate your Squid service accounts
  • Monitor semaphore utilization to identify capacity issues before they cause failures

See ServiceNow Configuration for details on these settings.

Quick Reference: HTTP Response Codes​

CodeCauseMeaningYour Action
200—SuccessRequest processed normally
202Session synchRequest NOT processed (session queue full)Check cookie handling—should not occur without cookies
429SemaphoresRequest rejected (API capacity exceeded)Review ServiceNow capacity configuration
503System loadService unavailableReview ServiceNow platform health

Next Steps​

Last updated on
We track. Ok?