Query Security

Squid validates all incoming queries to prevent security circumvention and protect against performance attacks. This document details the query validation mechanisms.

Query Validation Overview

Every API request passes through query validation:

┌─────────────────────────────────────────────────────────────────┐
│                    Incoming Query                               │
│  ?encodedQuery=base_name=server1^NQbase_name=anything           │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  Check 1: Forbidden Operators                                   │
│  ─────────────────────────────                                  │
│  Found: ^NQ (top-level OR)                                      │
│  Result: ❌ REQUEST REJECTED                                    │
└─────────────────────────────────────────────────────────────────┘

                    Query Blocked
                    HTTP 400 Bad Request

Forbidden Operators

Some query operators are always blocked because they could circumvent security controls.

The ^NQ Operator

^NQ creates a top-level OR clause in ServiceNow queries. This is dangerous because:

Security Circumvention Example:

Configuration: cmdb_ci_server
View Filter: base_company=authorized_company_sys_id

Malicious Query: ?encodedQuery=base_name=x^NQbase_company!=authorized_company_sys_id

Result WITHOUT ^NQ blocked:
  (base_company=authorized) AND (base_name=x)
  OR
  (base_company!=authorized)  ← Circumvents view filter!

The ^NQ operator effectively allows callers to bypass view filter restrictions by adding an alternative condition that matches unauthorized data.

Implementation

const ENCODED_QUERY_FORBIDDEN_OPS = /(\^NQ)/g;

// Always blocked - no configuration can enable these
if (encodedQuery.match(ENCODED_QUERY_FORBIDDEN_OPS)) {
  throw new BadRequestException(
    "Forbidden operator ^NQ detected. This operator is not allowed."
  );
}

Error Response

{
  "error": {
    "message": "Forbidden operator detected in encodedQuery",
    "detail": "The operator ^NQ is not permitted as it could circumvent security restrictions"
  }
}

Restricted Operators

Some operators are restricted by default but can be enabled per-configuration. These are primarily performance concerns rather than security exploits.

Why Restrict These Operators?

Performance comparison on cmdb_ci with 1.1 million entries:

Field	STARTSWITH	LIKE	ENDSWITH	ORDERBY
name (indexed)	2ms	1,900ms	1,900ms	48ms
environment (not indexed)	3,500ms	3,500ms	3,500ms	4,700ms

A caller could unknowingly cause significant database load, affecting your ServiceNow instance's performance.

Restricted Operator List

Operator	Description	Risk
%	ENDSWITH pattern	Full table scan
ENDSWITH	Suffix matching	Full table scan
*	LIKE pattern	Full table scan with text search
LIKE	Pattern matching	Full table scan with text search
CONTAINS	Substring search	Full table scan with text search
!*	NOT LIKE	Full table scan
NOT LIKE	Negative pattern	Full table scan
javascript:	Script execution	Arbitrary code execution
ORDERBY	Result ordering	Performance impact
123TEXTQUERY321	Text search	Full text index scan
123TEXTINDEXGROUP321	Text group search	Full text index scan

Configuration Control

Each configuration has a Restrict Encoded Query setting:

Setting	Restricted Operators
true (default for predefined)	Blocked
false	Allowed

Implementation

const ENCODED_QUERY_RESTRICTED_OPS = /(%|ENDSWITH|\*|LIKE|CONTAINS|!\*|NOT LIKE|javascript:|ORDERBY|123TEXTQUERY321|123TEXTINDEXGROUP321)/g;

if (config.queryRestricted && encodedQuery.match(ENCODED_QUERY_RESTRICTED_OPS)) {
  throw new BadRequestException(
    "Restricted operator detected. This configuration does not allow these operators."
  );
}

Enabling Restricted Operators

To allow restricted operators on a configuration:

1. Create a custom configuration (predefined configs cannot be edited)
2. Set Restrict Encoded Query to false
3. Consider setting a Limit to mitigate performance risk

warning

Only enable restricted operators when necessary. Consider using view filters (which can use these operators) instead of enabling them for callers.

View Filters vs User Queries

View filters (administrator-defined) can use any operators, including restricted ones:

Aspect	View Filter	User encodedQuery
Defined by	Administrator	API caller
Forbidden operators	Allowed	Blocked
Restricted operators	Allowed	Configurable
Can be bypassed	No	N/A

This allows administrators to use powerful query features while preventing callers from misusing them.

Supported Operators

The following operators are always allowed in encodedQuery:

Comparison Operators

• = - Equals
• != - Not equals
• < / <= / > / >= - Numeric/date comparison
• BETWEEN - Range
• MORETHAN / LESSTHAN - Relative comparison

String Operators

• STARTSWITH - Prefix matching (efficient with indexes)
• ISEMPTY / ISNOTEMPTY - Null checks
• EMPTYSTRING - Empty string check

Set Operators

• IN / NOT IN - Value lists
• SAMEAS / NSAMEAS - Field comparison

Logical Operators

• ^ - AND
• ^OR - OR (within same query level)

Field Comparison

• GT_FIELD / GT_OR_EQUALS_FIELD - Greater than field
• LT_FIELD / LT_OR_EQUALS_FIELD - Less than field

Advanced Operators

• RELATIVEGT / RELATIVELT - Relative date comparison
• VALCHANGES / CHANGESFROM / CHANGESTO - Audit trail
• DYNAMIC - Dynamic reference
• RLQUERY / ENDRLQUERY - Relationship query
• SUBQUERY / ENDSUBQUERY - Subquery
• JOIN - Join operations
• . - Dot-walking

Parameter Validation

Beyond encodedQuery operators, Squid validates all parameters:

sys_id Validation

const SYS_ID_REGEX = /^[a-z0-9]{32}$/;

// sys_id must be exactly 32 lowercase alphanumeric characters
if (!sysId.match(SYS_ID_REGEX)) {
  throw new BadRequestException("Invalid sys_id format");
}

Date/Time Validation

// ISO8601 format required
// YYYY-MM-DD'T'hh:mm(:ss)?'Z' or YYYYMMDD'T'hhmm(ss)?'Z'

if (!isValidISO8601(dateValue)) {
  throw new BadRequestException("Invalid date format. Use ISO8601.");
}

Configuration Name Validation

// Configuration names cannot contain '|' character
if (configName.includes('|')) {
  throw new BadRequestException("Invalid configuration name");
}

Multiple Value Handling

For parameters that should have a single value:

// Multiple values for encodedQuery is an error
if (encodedQueryParams.length > 1) {
  throw new BadRequestException(
    "Multiple encodedQuery parameters not allowed"
  );
}

Injection Prevention

SQL Injection

Squid uses ServiceNow's GlideRecord/GlideQuery APIs, which handle parameterization automatically. Direct SQL construction is never used.

NoSQL/Script Injection

The javascript: operator restriction prevents arbitrary script execution in queries.

Parameter Injection

All parameters are validated before use, preventing injection attacks through URL parameters.

Rate Limiting and Quotas

While not directly a query validation feature, Squid operates within ServiceNow's REST Transaction Quota:

• Default timeout: 300 seconds
• Recommended for large exports: 900 seconds
• Configurable via ServiceNow system properties

Error Messages

Query validation errors return clear, actionable messages:

{
  "error": {
    "message": "Query validation failed",
    "detail": "The operator LIKE is restricted for this configuration. Create a custom configuration with 'Restrict Encoded Query' set to false if you need this operator."
  }
}

Best Practices

For API Consumers

1. Use STARTSWITH instead of LIKE when possible
2. Avoid ORDERBY - sort results client-side
3. Use specific sys_id queries when possible
4. Use IN for multiple values instead of ^OR chains

For Administrators

1. Keep Restrict Encoded Query enabled unless necessary
2. Use view filters for complex queries (unrestricted)
3. Set appropriate Limit values
4. Monitor for performance-impacting queries

Operator Quick Reference

┌─────────────────────────────────────────────────────────────────┐
│                     FORBIDDEN (Always Blocked)                   │
├─────────────────────────────────────────────────────────────────┤
│  ^NQ                                                             │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                     RESTRICTED (Default Blocked)                 │
├─────────────────────────────────────────────────────────────────┤
│  %, ENDSWITH, *, LIKE, CONTAINS, !*, NOT LIKE                   │
│  javascript:, ORDERBY, 123TEXTQUERY321, 123TEXTINDEXGROUP321     │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                     ALLOWED (Always Permitted)                   │
├─────────────────────────────────────────────────────────────────┤
│  =, !=, <, <=, >, >=, BETWEEN, STARTSWITH                       │
│  ISEMPTY, ISNOTEMPTY, IN, NOT IN, ^, ^OR, .                     │
│  And more (see Supported Operators section)                      │
└─────────────────────────────────────────────────────────────────┘

Related Topics

• Data Access Controls - View filters
• Best Practices - Query optimization
• Architecture - Validation pipeline