Skip to main content

HTTP GET Request

URL​

The URL is https://yourinstance/api/x_a46gh_squidx/v1/data/{configName} where {configName} is the name of the configuration you are requesting. See Predefined Configurations for an overview of all predefined configurations provided by squid.

Query Parameter​

The configuration on its own is only of limited use. Query parameter allow the caller to request a specific entity, filter results, add relations to the request and further influence the behavior of squid.

relations​

Resolving references and relations is the added value of squid. References are resolved by default. Relations must be explicitly requested. See Predefined Relations for an overview of all predefined relations provided by squid.

Relations are requested as query parameters relations. squid accepts this parameter in various formats:

  • relations=rel1,rel2,rel3
  • relations=rel1&relations=rel2&relations=rel3
  • relations=rel1,rel2&relations=rel3

Use whatever fits your calling application best.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?sys_id=5f9b83bfc0a8010e005a2b3212c9dc07&relations=ci_to_user_group_dynamic_minimal

{configName}.relations​

When rendering inline content you might want to request relations for referenced entities. You can do this by adding a query parameter with the following structure:

configName.relations=rel5,rel6

This tells squid to add the given relations whenever an entity is rendered with a configuration configName.

As with relations squid accepts this parameter in various formats:

  • configName.relations=rel5,rel6,rel7
  • configName.relations=rel5&relations=rel6&relations=rel7
  • configName.relations=rel5,rel6&relations=rel7

Use whatever fits your calling application best.

Try it: TODO generate demo entity and create link.

{relationName}.filter​

{configName}.{relationName}.filter​

{relationName}.filterOn​

Work in Progress

Documentation NOT complete!

TL;DR

Filter result sets based on the values of entities referenced by a relation.

'Give me all CIs that have an Owns::Owned by relation to a CI with Lifecycle Stage Operational'.

Why​

ServiceNow does not support metadata on relations, i.e. a relation CI => Owns::Owned by => SomeOtherCI either exists or doesn't exist. There are no further qualifying properties such as lifecycle values, etc. You cannot 'deactivate' or 'retire' a relation. You can only set state on entities.

If you want to keep no longer active relations (e.g. due to regulatory requirements), you either have to create a proprietary solution (replicate cmdb_rel_ci as u_cmdb_rel_ci_history and move expired relations to that table OR extend cmdb_rel_ci with metadata - probably not a good idea) or filter relations based on some property of the returned referenced entity, shifting the effort to the consumer (UI or API).

The latter 'solution' is highly inefficient in that all relations must first be resolved and the endpoints retrieved, before disregarding any just retrieved entities that do not match the given criteria.

We can do better! As always squid attempts to make life as easy as possible for its users. We therefore offer filters on relations and while we're at it these can be applied to all relation types supported by squid (not just cmdb_rel_ci). filters are directly applied to the database query that resolves relations and are therefore the most efficient solution possible.

Performance Caveat

While this is the most efficient solution possible, that remains a relative statement. Depending on the structure of the query, the referenced configurations, the database and the possible presence of - or more importantly absence of - indexes, queries may easily take multiple seconds to execute.

This is due to the fact, that without corresponding indexes the database is forced to execute potentially multiple full table scans. As with any database optimization, the solution lies in finding and creating the correct indexes.

Don't get this caveat wrong. filters are way more efficient than any possible alternative. Your expectation of 'efficient' just might not match what you're getting.

Try it out and see what performance values you get.

How​

Relations (mostly) reference entities that are retrieved and rendered according to the configured targetConfiguration. (See Relations for details.) A targetConfiguration defines a view. filters execute an exists subquery based on this view of the targetConfiguration of the requested relation.

filters can define conditions on any properties defined in this view.

Don't worry. This only sounds complicated on first reading. Repeat two or more times and it gets easier. We might even add some graphics in the future...

Usage​

filter query parameter have the following structure:

relationName.filter=filterClause1^ORfilterClause2^ANDfilterClause3

with relationNamebeing the relation the given filter is to be applied to. relationNameis the same value as used when requesting a relation. See relations above.

Following ServiceNow GlideRecord conventions (but contrary to what you might be used to from SQL) ^OR has precedence over ^AND! The above relationName.filter query parameter would result in

(filterClause1 OR filterClause2) AND filterClause3

filterClauses have the following structure:

fieldName,op,value

with

  • fieldName - any property defined in the viewof the targetConfiguration
  • op - the GlideQuery operator. See below for list of supported operators
  • value - the value to query on. (Please URI encode any ',' possibly included in the given value.)

Multiple relationName.filter query parameter are AND combined.

Supported 'filter' GlideQuery Operators

  • =
  • !=
  • > (only for numeric values)
  • >= (only for numeric values)
  • < (only for numeric values)
  • <= (only for numeric values)
  • IN
  • STARTSWITH

TODO restricted OPS

TODO above based on SN docs. Not sure if accurate. Check if other ops (ISEMPTY, etc.) really don't work.

{configName}.{relationName}.filterOn​

encodedQuery​

ServiceNow has the concept of 'encodedQueries'.

See ServiceNow: 'Encoded query strings' for details on how to structure a query.
See ServiceNow: 'Operators available for filters and queries' for details on available operators for encodedQueries.

Usage in the context of squid​

squid uses ServiceNow database views to

a) define what data is retrieved and
b) aggregate data of multiple tables in a single JSON.

Every table that is incorporated into a ServiceNow database view must have a so-called 'Variable prefix'. By convention all squid database views use 'base' as the 'Variable prefix' for the base table. (If you want to use existing database views that do not follow this naming convention, the prefix can be changed in any specific configuration.)

The prefixes for any additional outer left joined tables are documented in the corresponding configuration documentation.

Field Names​

squid removes the database view specific 'Variable prefixes' to present the returned JSON in a more expected format. Instead of base_sys_id we return sys_id.

For ServiceNow to understand a given encodedQuery we have to use the field names defined on the database view that we are querying, i.e. we have to set conditions on base_sys_id instead of sys_id.

A future release will take care of this for you, but we're not quite there yet.

Supported encodedQuery Operators​

  • ^ / ^OR (AND/OR)
  • =, !=
  • STARTSWITH
  • ISEMPTY / ISNOTEMPTY
  • EMPTYSTRING
  • IN / NOT IN
  • < / <= / >= / > / BETWEEN / MORETHAN / LESSTHAN
  • SAMEAS / NSAMEAS
  • GT_FIELD / GT_OR_EQUALS_FIELD / LT_FIELD / LT_OR_EQAULS_FIELD
  • RELATIVEGT / RELATIVELT
  • VALCHANGES / CHANGESFROM / CHANGESTO
  • DYNAMIC
  • RLQUERY / ENDRLQUERY / SUBQUERY / ENDSUBQUERY / JOIN
  • . (dot walking)

As always with database queries handle with care. The above operators will cause full table scans if the queried fields are not indexed. squid has no plausible way of checking for the existence of an index. (This is technically possible, but the performance impact on each and every request does not justify that approach.)

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?encodedQuery=base_name=dbaix901nyc

Restricted encodedQuery Operators​

squid is built to deliver large amounts of data in the most performant way possible without impacting your ServiceNow instance. We therefore prevent encodedQueries from executing operations that have the potential of having a noticeable impact on your instance. Anything that would cause a full table scan with text search, order by operations etc. are by default rejected.

As the performance of database queries always depends on the exact query and the presence of indexes our categorizing of operators into critical (restricted) and non-critical (unrestricted) can only be a best effort.

For you to better understand why we are restricting at all we have included the following anecdotal query durations:

Querying cmdb_ci with 1.1 million entries. (These are just anecdotal values to give an impression of what we're talking about. This is NOT a full-fledged performance evaluation. Measurement after 5 warmup queries.)

FieldSTARTSWITHLIKEENDSWITHORDERBY
name - indexed2ms1900ms1900ms48ms
environment - not indexed3500ms3500ms3500ms4700ms

Restricting encoded queries in this way might sound heavy-handed, but in most use cases the caller of squid is coming from a peripheral system that has little to nothing to do with ServiceNow other than that it needs data. The caller will probably neither know nor notice any negative impact he or she is causing.

We however recognize that there may be valid use cases when these operators are necessary and useful. Configurations therefore have a property Restrict Encoded Query that is set to true on all predefined configurations. If this property is set to true the following operators are NOT allowed. If you wish to allow these operators, create a new configuration and set the Restrict Encoded Query to false.

  • % / ENDSWITH
  • * / LIKE / CONTAINS
  • !* / NOT LIKE
  • javascript (Anything that requires javascript is restricted)
  • ORDERBY
  • 123TEXTQUERY321 / 123TEXTINDEXGROUP321

Unless the requested configuration explicitly allows the use of restricted operators, including any of the restricted operators in an encodedQuery is considered an error condition and the request is rejected.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?encodedQuery=base_nameLIKEdbaix901nyc

warning

squid reserves the right to expand the above-mentioned list of restricted encoded query operators at any point in time.

ServiceNow may implement new encoded query operators or undocumented operators may come to our attention. If considered
potentially performance critical, these will be added to the above list at our discretion. Please check the release notes for any changes.

Forbidden for security reasons​

  • ^NQ - squid allows configurations to be restricted by the role of the caller and configurations to define view filter that limit the returned data. Both together are a security feature (TODO add link to corresponding documentation).
    ^NQ would add a top level OR clause to any query allowing a caller to circumvent any set restrictions.

Including a forbidden operator in an encodedQuery is considered an error condition and the request is rejected.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?encodedQuery=base_name=dbaix901nyc^NQbase_name=somethingelse

Use restricted encodedQuery operators in view_filter​

Configurations have a property called view_filter. This allows an squid administrator to add an encodedQuery to the configuration. This encodedQuery is NOT restricted in any way as it is defined by an squid administrator.

Time/Date values​

Time/Date values are a difficult topic in a ServiceNow context. ServiceNow will accept nonsense Time/Date values without throwing any kind of error and just make a best effort. This will give you the impression of your query being correct until you notice that your results do not match what you were expecting.

The following is based on trial and error:

encodedQuery=base_sys_updated_on>2024-04-22 23:00:00 will be read as '2024-04-22 23:00:00 UTC' as will

encodedQuery=base_sys_updated_on>2024-04-22T23:00:01
encodedQuery=base_sys_updated_on>2024-04-22T23:00:01Z
encodedQuery=base_sys_updated_on>2024-04-22T23:00:01nonsense(!)

warning

encodedQuery=base_sys_updated_on>2024-04-22x23:00:01 will be accepted as valid encodedQuery, but ServiceNow will only interpret the first part, i.e. '2024-04-22' resulting in a query value of '2024-04-22 00:00:00 UTC'.

Please be sure to check any encodedQueries that use Time/Date values.

encodedQuery (HTTP) query parameter format​

squid will accept encoded queries as

encodedQuery=yourEncodedQuery

Multiple values are considered an error condition.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?encodedQuery=base_name=dbaix901nyc&encodedQuery=base_asset_tag=P1000182

filterOnTags (cmdb_key_value)​

Public cloud provider have adopted tags as way of adding metadata to entities (AWS, Azure, GCP) making tags a common and established way of qualifying and accessing configuration items, especially in a dev-ops environment.

ServiceNow supports this concept by way of cmdb_key_value. This table is populated by Cloud Discovery, but may also be populated directly.

Tags are not Tags

ServiceNow uses the term Tags for a slightly different concept. See the link for details on the ServiceNow usage.

squid users are mainly peripheral systems or devOps integrations. These users generally understand 'tag' as described by the major public cloud providers. We try to make life easy for our users.

filterOnTags allows restricting the returned data to only configuration items (cmdb_key_value only allows references to configuration items) that are referenced by at least one key-value entry in cmdb_key_value that matches the given criteria.

Negative filtering, i.e. returning only configurations items that are NOT referenced by a specific key-value entry, is NOT supported.

filterOnTags does NOT change the returned data, i.e. it does not implicitly add tags to the returned configuration items. filterOnTags only filters (nomen-est-omen) data. See showTags / showTagsInline below for details on how to render tags for configuration items.

filterOnTags query parameter have the following structure:

filterOnTags=tagClause1^ORtagClause2^ANDtagClause3

Following ServiceNow GlideRecord conventions (but contrary to what you might be used to from SQL) ^OR has precedence over ^AND! The above filterOnTags query parameter would result in

(tagClause1 OR tagClause2) AND tagClause3

tagClauses have the following structure:

  • tagName / tagName=* - filters on having a tag with key=tagName ignoring value
  • tagName* / tagName*=* - filters on having a tag with keySTARTSWITHtagName ignoring value
  • tagName= - filters on having a tag with key=tagName andvalue=null
  • tagName*= - filters on having a tag with keySTARTSWITHtagName andvalue=null
  • tagName=value1(,value2,...) - filters on having a tag with key=tagName and a value of value1 OR value2
  • tagName*=value1(,value2,...) - filters on having a tag with keySTARTSWITHtagName and a value of value1 OR value2

Value wildcards (value*) are NOT supported.

Multiple filterOnTags query parameter are AND combined.

filterOnTags=tagName1=value1&filterOnTags=tagName2=value2 is equivalent to filterOnTags=tagName1=value1^ANDtagName2=value2

filterOnTeams (cmdb_rel_team)​

WORK IN PROGRESS

This documentation is work-in-progress and subject to change.

squid supports a teamsquery parameter.

Syntax: teams=group_type=group1,group2,group3

group_type is e.g. managed_by, approval
group1,group2,group3 are either sysIds of a sys_user_group or the name of a sys_user_group. If a name is given squid will attempt to resolve the name to a sysId. Multiple values are combined as OR condition.

Multiple tags are AND combined.

warning

teams=managed_by=group1,group2
is NOT the same as
teams=managed_by=group11&teams=managed_by=group2

The first expression will restrict the query to CIs that are managed_by group1 OR group2.

The second expression will restrict the query to CIs that are managed_by group1 AND group2.

showTags​

Just as relations, resolving tags requires at least one additional database query. For this reason, tags are only rendered when explicitly requested.

The query parameter showTags tells squid to retrieve and render tags belonging to a configuration item.

squid can render tags in two forms:

As Object​

As object. This is a standard JSON object with tagName (key) as property name and value as value.

{
  ...,
  "tags": {
    // sys_id of the entity
    "995390bccd34f010f8778495fd9bf7f7": {
      // key : value 
      "Owner": "SD DC Ops",
      "Location": "San Diego",
      "Environment": "Production"
    },
    "6e5314bccd34f010f8778495fd9bf72b": {
      "Owner": "SD DC Ops",
      "Environment": "Production",
      "Location": "San Diego"
    },
    "9c5314bccd34f010f8778495fd9bf710": {
      "Service": "San Diego Wiki",
      "Owner": "SD DC Ops",
      "Location": "San Diego",
      "Environment": "Production"
    },
    "821b6357c0a8018b244b9609d7010267": {
      "Owner": null
    }
  },
  ...
}

While object is easy to read intuitively it is not without problems.

ServiceNow does NOT prevent multiple tags with the same name for a single configuration item. In our example above, nothing prevents a configuration item from having two or more tags with the name Service. As object may only have one property with a given name, it is undefined which of the multiple values is returned.

The structure of the returned object is unknown beforehand, this may lead to problems when deserializing with some JSON frameworks.

For these reasons squid additionally offers

As Array​

as array

The exact same data as above (with the addition of the extra 'Service' tag for demonstration purposes) renders as shown below.

{
  ...,
  "tags": {
    "995390bccd34f010f8778495fd9bf7f7": [
      {
        "name": "Owner",
        "value": "SD DC Ops"
      },
      {
        "name": "Location",
        "value": "San Diego"
      },
      {
        "name": "Environment",
        "value": "Production"
      }
    ],
    "6e5314bccd34f010f8778495fd9bf72b": [
      {
        "name": "Owner",
        "value": "SD DC Ops"
      },
      {
        "name": "Environment",
        "value": "Production"
      },
      {
        "name": "Location",
        "value": "San Diego"
      }
    ],
    "9c5314bccd34f010f8778495fd9bf710": [
      {
        // Conflicting name 'Service'
        "name": "Service",
        "value": "Redundant Tag!"
      },
      {
        // Conflicting name 'Service'
        "name": "Service",
        "value": "San Diego Wiki"
      },
      {
        "name": "Owner",
        "value": "SD DC Ops"
      },
      {
        "name": "Location",
        "value": "San Diego"
      },
      {
        "name": "Environment",
        "value": "Production"
      }
    ],
    "821b6357c0a8018b244b9609d7010267": [
      {
        "name": "Owner",
        "value": null
      }
    ]
  },
  ...
}

This is a much harder read and more verbose, but it is able to render 'conflicting' tagNames and has a predefined structure.

Allowed showTags query parameter are:

  • showTags defaults to showTags=object
  • showTags=object
  • showTags=array

showTagsInline​

Tags may also be rendered inline.

showBlank​

By default, squid will surpress any properties with a null value.

Some client libraries might rely on a stable JSON structure and not play nice with missing JSON properties.

The query parameter showBlank will cause squid to render null JSON properties.

Try it:

sys_id​

sys_id allows retrieval of one or more entities by way of their sysId.

squid accepts this parameter in various formats:

  • sys_id=5f9b83bfc0a8010e005a2b3212c9dc07,106c5c13c61122750194a1e96cfde951,1072335fc611227500c0267a21be5dc5

sys_id=5f9b83bfc0a8010e005a2b3212c9dc07&sys_id=106c5c13c61122750194a1e96cfde951&sys_id=1072335fc611227500c0267a21be5dc5

  • sys_id=5f9b83bfc0a8010e005a2b3212c9dc07,106c5c13c61122750194a1e96cfde951&sys_id=1072335fc611227500c0267a21be5dc5

Use whatever fits your calling application best.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?sys_id=5f9b83bfc0a8010e005a2b3212c9dc07,106c5c13c61122750194a1e96cfde951,1072335fc611227500c0267a21be5dc5

updatedBefore / updatedSince / lastDiscoveredBefore / lastDiscoveredSince​

updatedBefore, updatedSince and lastDiscoveredBefore, lastDiscoveredSince are utility query parameters intended to make querying both values easier.

The expected DateTime format is ISO8601

  • "YYYY-MM-DD'T'hh:mm(:ss)?'Z'" or
  • "YYYYMMDD'T'hhmm(ss)?'Z'"

The given values are used as to build a filter of

  • base_sys_updated_on<givenDateTimeValue for updatedBefore
  • base_sys_updated_on>=givenDateTimeValue for updatedSince
  • base_sys_last_discovered<givenDateTimeValue for lastDiscoveredBefore
  • base_sys_last_discovered>=givenDateTimeValue for lastDiscoveredSince

Please notice that these values are inclusive for *since and exclusive for *before. (Feel free to give us feedback on this at support.arc46.io)

The 'Variable prefix' (base_) will automatically be adjusted for whatever value is defined in the requested configuration.

squid only accepts and returns ISO8601 Date/Time values (except when encoded in encodedQueries).

Multiple values are considered an error condition.

Try it:

preservePrefix​

squid uses ServiceNow database views to aggregate data. These views prefix all properties with so-called Variable Prefixes, e.g. base_, lin_, etc.

Most of the time you do NOT want to see these prefixes. There are however use cases where preserving the prefix is essential, e.g. when aggregating tables with name collisions. In the case of two joined tables, one with the prefix base and the other with the prefix other with both having the property name would give us a raw result set with base_name and other_name, with base_name having priority and other_name being ignored. The returned json would only contain one property name with the value of base_name

This will usually be part of the configuration ( See Configurations - Preserve Prefix), but is also available as a query parameter. preservePrefix has precedence over configuration settings.

Try it: https://demo.squid46.io/cmdb_ci_server?limit=1&preservePrefix

{config}.preservePrefix​

TODO

showConfig​

The config used to render entities is normally not part of the returned json and of no value to the recipient. During development, it may however help to know which configuration was used to render a specific entity.

Referenced entities may be retrieved based on multiple configs depending on the path by which they are referenced. The returned json will be a merge of all data returned by all relevant configs.

The query parameter showConfig will add a property squid_config to each and every returned entity as a string array of all configs used to render the entity.

{
  ...,
  "data": [
    {
      ...,
      "squid_config": [
        "cmdb_ci_server"
      ],
      ...
    }
  ],
  "relations": {},
  "referenced": {
    "067018092b2692103c92f65ac891bf66": {
      ...
      "squid_config": [
        "core_company"
      ],
      ...
    }
  }
}

Try it: Try it: https://demo.squid46.io/cmdb_ci_server?limit=1&showConfig

limit​

limit allows you to limit the amount of entities returned. This is of limited use in a production environment, but during development you might want to test your queries with just 10 returned entities instead of 10.000.

As the order of the result set is intentionally undefined, which entities are returned may vary from call to call.

Try it: https://demo.squid46.io/cmdb_ci_server_minimal?limit=5

lenient​

Do not use for production queries

We strongly suggest you use this query parameter ONLY during development.

Use in production can lead to unnoticed missing data!

squid validates configurations on execution. We might do a deep dive on why that is so in the future, but for now take our word for it, that this is the only realistically workable approach.

In consequence squid will encounter misconfigurations while executing queries.

squid now has two options:

  • crash hard and early. This is usually the safest option, but developers trying to get their queries right won't have much fun with this approach - or
  • be 'nice' and include warnings in the metadata of the returned JSON pointing developers to what went wrong.

Being nice has the risk of missing data without noticing. Who reads the metadata of a JSON in a technical API? Therefore, squid will, by default, return an error whenever it encounters a misconfiguration.

Developers tweaking their queries have the option of adding a query parameter lenient. This tells squid to play 'nice' and only add warnings to the metadata.

Regardless of this query parameter, squid will return an error in the case of an obviously false query, e.g. limit is negative, sys_id is malformed, etc.

Last updated on
We track. Ok?