Throttling Filters and Policies

To protect applications from being overused by clients, use a throttling filter to limit how many requests clients can make in a defined time.

ThrottlingFilter — limit the rate of requests

Description

Limits the rate that requests pass through a filter. The maximum number of requests that a client is allowed to make in a defined time is called the throttling rate.

The throttling filter uses a strategy based on the token bucket algorithm, which allows some bursts. Because of traffic bursts, the throttling rate can occasionally be higher than the defined limit - for example, with a throttling rate of 10 requests/10 seconds there can be more than 10 requests in the 10 second duration. However, the number of concurrent requests cannot exceed that defined for the throttling rate - for example, with a throttling rate of 10 requests/10 seconds there cannot be more than 10 concurrent requests.

When the throttling rate is reached, OpenIG issues an HTTP status code 429 Too Many Requests and a Retry-After header, whose value is rounded up to the number of seconds to wait before trying the request again.

GET http://openig.example.com:8080/throttle-scriptable HTTP/1.1

    . . .

    HTTP/1.1 429 Too Many Requests
    Retry-After: 10

Usage

{
    "name": string,
    "type": "ThrottlingFilter",
    "config": {
        "requestGroupingPolicy": expression,
        "throttlingRatePolicy": reference or inline declaration,  //Use either "policy"
        "rate": {                                                 //or "rate", but not both.
            "numberOfRequests": integer,
            "duration": duration string
        },
        "cleaningInterval": duration string,
        "executor": executor
    }
}

Properties

"requestGroupingPolicy": expression, required

An expression to identify the partition to use for the request. In many cases the partition identifies an individual client that sends requests, but it can also identify a group that sends requests. The expression can evaluate to the client IP address or user ID, or an OpenID Connect subject/issuer.

Default: Empty string. The value for this expression must not be null.

Examples

The following links provide examples of how the throttling policies are implemented:

"Example of a Mapped Throttling Policy"
"Example of a Scriptable Throttling Policy"

The following route defines a throttling rate of 6 requests/10 seconds to requests. For information about how to set up and test this example, see Configuring a Simple Throttling Filter in the Gateway Guide.

{
  "handler": {
    "type": "Chain",
    "config": {
      "filters": [
        {
          "type": "ThrottlingFilter",
          "config": {
            "requestGroupingPolicy": "${request.headers['UserId'][0]}",
            "rate": {
              "numberOfRequests": 6,
              "duration": "10 seconds"
            }
          }
        }
      ],
      "handler": "ClientHandler"
    }
  },
  "condition": "${matches(request.uri.path, '^/throttle-simple')}"
}

Javadoc

org.forgerock.openig.filter.throttling.ThrottlingFilterHeaplet

MappedThrottlingPolicy — map throttling rates to groups of requests

Description

Maps different throttling rates to different groups of requests, according to the evaluation of throttlingRateMapper.

Usage

{
    "type": "ThrottlingFilter",
    "config": {
        "requestGroupingPolicy": expression,
        "throttlingRatePolicy": {
            "type": "MappedThrottlingPolicy",
            "config": {
                "throttlingRateMapper": expression<string>,
                "throttlingRatesMapping": {
                    "mapping1": {
                        "numberOfRequests": integer,
                        "duration": duration string
                    },
                    "mapping2": {
                        "numberOfRequests": integer,
                        "duration": duration string
                    }
                },
                "defaultRate": {
                    "numberOfRequests": integer,
                    "duration": duration string
                }
            }
        }
    }
}

Properties

"throttlingRateMapper": expression, required

An expression to categorize requests for mapping to a throttling rate in the throttlingRatesMapping.

If this parameter is null or does not match any specified mappings, the default throttling rate is applied.

"throttlingRatesMapping": object, required

A map of throttling rate by request group. Requests are categorized into groups by the evaluation of the expression "throttlingRateMapper".

"mapping1" and "mapping2": string, required

The evaluation of the expression "throttlingRateMapper".

"defaultRate": object, required

The default throttling rate to apply if the evaluation of the expression "throttlingRateMapper" is null or is not mapped to a throttling rate.

The number of mappings is not limited to two.

"numberOfRequests": integer, required

The number of requests allowed through the filter in the time specified by "duration".

"duration": duration string, required

A time interval during which the number of requests passing through the filter is counted.

A duration is a lapse of time expressed in English, such as 23 hours 59 minutes and 59 seconds.

Durations are not case sensitive.

Negative durations are not supported.

The following units can be used in durations:

indefinite, infinity, undefined, unlimited: unlimited duration
zero, disabled: zero-length duration
days, day, d: days
hours, hour, h: hours
minutes, minute, min, m: minutes
seconds, second, sec, s: seconds
milliseconds, millisecond, millisec, millis, milli, ms: milliseconds
microseconds, microsecond, microsec, micros, micro, us: microseconds
nanoseconds, nanosecond, nanosec, nanos, nano, ns: nanoseconds

Example of a Mapped Throttling Policy

In the following example, requests from users in the accounts and sales departments of example.com are mapped to different throttling rates. Requests from other departments use the default throttling rate. For information about how to set up and test this example, see Configuring a Mapped Throttling Filter in the Gateway Guide.

Alice and Bob both send requests from accounts, and so they each have a throttling rate of 6 requests/10 seconds. The throttling rate is applied independently to Alice and Bob, so no matter how many requests Alice sends in 10 seconds, Bob can still send up to 6 requests in the same 10 seconds. Carol sends requests from sales, with a throttling rate of 3 requests/10 seconds. Dave sends requests from finance, with the default rate of 1 request/10 seconds.

The throttling rate is assigned according to the evaluation of throttlingRateMapper. In the example, this parameter evaluates to the value of the request header X-Forwarded-For, representing the hostname of the department.

{
  "handler": {
    "type": "Chain",
    "config": {
      "filters": [
        {
          "type": "ThrottlingFilter",
          "config": {
            "requestGroupingPolicy": "${request.headers['UserId'][0]}",
            "throttlingRatePolicy": {
              "type": "MappedThrottlingPolicy",
              "config": {
                "throttlingRateMapper": "${request.headers['X-Forwarded-For'][0]}",
                "throttlingRatesMapping": {
                  "accounts.example.com": {
                    "numberOfRequests": 6,
                    "duration": "10 seconds"
                  },
                  "sales.example.com": {
                    "numberOfRequests": 3,
                    "duration": "10 seconds"
                  }
                },
                "defaultRate": {
                  "numberOfRequests": 1,
                  "duration": "10 seconds"
                }
              }
            }
          }
        }
      ],
      "handler": "ClientHandler"
    }
  },
  "condition": "${matches(request.uri.path, '^/throttle-mapped')}"
}

Javadoc

org.forgerock.openig.filter.throttling.MappedThrottlingPolicyHeaplet

ScriptableThrottlingPolicy — script to map throttling rates

Description

Uses a script to look up throttling rates to apply to groups of requests.

Usage

{
   "type": "ThrottlingFilter",
   "config": {
       "requestGroupingPolicy": expression,
       "throttlingRatePolicy": {
           "type": "ScriptableThrottlingPolicy",
           "config": {
               "type": string,
               "file": string,     // Use either "file"
               "source": string    // or "source", but not both
           }
       }
   }
}

Properties

"type": string, required: The Internet media type (formerly MIME type) of the script. For Groovy, the value is "application/x-groovy".
"file": string, required if "source" is not used: The path to the file containing the script.

Relative paths in this field are relative to the base location for scripts, which depends on the configuration. For information, see Installing OpenIG in the Gateway Guide.

The base location for Groovy scripts is on the classpath when the scripts are executed. If a Groovy script is not in the default package, but instead has its own package name, it belongs in the directory corresponding to the package name. For example, a script in package com.example.groovy belongs under openig-base/scripts/groovy/com/example/groovy/.
"source": string, required if "file" is not used: The script as a string.

Example of a Scriptable Throttling Policy

In the following example, the DefaultRateThrottlingPolicy delegates the management of throttling to the scriptable throttling policy. For information about how to set up and test this example, see Configuring a Scriptable Throttling Filter in the Gateway Guide.

The script applies a throttling rate of 6 requests/10 seconds to requests from the accounts department of example.com. For all other requests, the script returns null. When the script returns null, the default rate of 1 request/10 seconds is applied.

The script can store the mapping for the throttling rate in memory, and can use a more complex mapping mechanism than that used in the MappedThrottlingPolicy. For example, the script can map the throttling rate for a range of IP addresses. The script can also query an LDAP directory, query an external database, or read the mapping from a file.

{
  "handler": {
    "type": "Chain",
    "config": {
      "filters": [
        {
          "type": "ThrottlingFilter",
          "config": {
            "requestGroupingPolicy": "${request.headers['UserId'][0]}",
            "throttlingRatePolicy": {
              "type": "DefaultRateThrottlingPolicy",
              "config": {
                "delegateThrottlingRatePolicy": {
                  "type": "ScriptableThrottlingPolicy",
                  "config": {
                    "type": "application/x-groovy",
                    "file": "ThrottlingScript.groovy"
                  }
                },
                "defaultRate": {
                  "numberOfRequests": 1,
                  "duration": "10 seconds"
                }
              }
            }
          }
        }
      ],
      "handler": "ClientHandler"
    }
  },
  "condition": "${matches(request.uri.path, '^/throttle-scriptable')}"
}

The groovy script maps a throttling rate for the accounts department of example.com. Other requests receive the default throttling rate.

/**
 * ThrottlingScript.groovy
 *
 * Script to throttle access for requests from the accounts department
 * of example.com. Other requests return null.
 */

if (request.headers['X-Forwarded-For'].values[0]  == 'accounts.example.com') {
    return new ThrottlingRate(6, '10 seconds')
} else {
    return null
}

Javadoc

org.forgerock.openig.filter.throttling.ScriptableThrottlingPolicy.Heaplet

DefaultRateThrottlingPolicy — default policy for throttling rate

Description

Provides a default throttling rate if the delegating throttling policy returns null.

Usage

{
   "type": "ThrottlingFilter",
   "config": {
       "requestGroupingPolicy": expression,
       "throttlingRatePolicy": {
           "type": "DefaultRateThrottlingPolicy",
           "config": {
               "delegateThrottlingRatePolicy" : reference or inline declaration,
               "defaultRate": {
                   "numberOfRequests": integer,
                   "duration": duration string
               }
           }
       }
   }
}

Properties

"delegateThrottlingRatePolicy": reference, required

The policy to which the default policy delegates the throttling rate. The DefaultRateThrottlingPolicy delegates management of throttling to the policy specified by delegateThrottlingRatePolicy.

If delegateThrottlingRatePolicy returns null, the defaultRate is used.

For information about policies to use, see MappedThrottlingPolicy(5) and ScriptableThrottlingPolicy(5).

"defaultRate": object, required

The default throttling rate to apply if the delegating policy returns null.

"numberOfRequests": integer, required

The number of requests allowed through the filter in the time specified by "duration".

"duration": duration string, required

A time interval during which the number of requests passing through the filter is counted.

A duration is a lapse of time expressed in English, such as 23 hours 59 minutes and 59 seconds.

Durations are not case sensitive.

Negative durations are not supported.

The following units can be used in durations:

indefinite, infinity, undefined, unlimited: unlimited duration
zero, disabled: zero-length duration
days, day, d: days
hours, hour, h: hours
minutes, minute, min, m: minutes
seconds, second, sec, s: seconds
milliseconds, millisecond, millisec, millis, milli, ms: milliseconds
microseconds, microsecond, microsec, micros, micro, us: microseconds
nanoseconds, nanosecond, nanosec, nanos, nano, ns: nanoseconds

Example

For an example of how this policy is used, see "Example of a Scriptable Throttling Policy" .

Javadoc

org.forgerock.openig.filter.throttling.DefaultRateThrottlingPolicyHeaplet