Package org.openidentityplatform.openig.ai.filter

Class LLMProxyFilter

java.lang.Object

org.openidentityplatform.openig.ai.filter.LLMProxyFilter

All Implemented Interfaces:

org.forgerock.http.Filter

public class LLMProxyFilter
extends Object
implements org.forgerock.http.Filter

Identity-aware LLM proxy filter.

1. Provider normalization – rewrites the upstream URI and injects the correct authentication header for the configured LLMProvider.
2. Identity extraction – reads the caller's identity from a configurable request attribute (populated upstream by, e.g., OAuth2ResourceServerFilter).
3. Token-based rate limiting – estimates the prompt-token cost of each request and enforces per-identity limits via TokenRateLimiter. Returns 429 Too Many Requests (with an exact Retry-After header) when the bucket is exhausted.
4. Response enrichment – adds X-LLM-* observability headers.

Heap configuration

 {
   "name": "OpenAIProxy",
   "type": "LLMProxyFilter",
   "config": {
     "provider"          : "OPENAI",
     "baseUrl"           : "https://api.openai.com/v1",  // optional
     "apiKey"            : "${system['llm.apiKey']}",
     "sub"               : "${attributes.sub}",          // optional, expression, default "anonymous"
     "rateLimitEnabled"  : true,                         // optional, default true
     "rate": {
       "numberOfTokens"    : 10000,    // tokens per window (burst capacity)
       "duration"          : "1 minute"
       "cleaningInterval"  : "5 minutes"  // optional, bucket eviction period
     }
   }
 }
 

Response headers

• X-LLM-Provider — provider enum name
• X-LLM-Identity — resolved identity key
• X-RateLimit-Remaining — tokens left after this request

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	LLMProxyFilter.Heaplet

Field Summary

Fields

Modifier and Type	Field	Description
static String	HEADER_LLM_IDENTITY
static String	HEADER_LLM_PROVIDER
static String	HEADER_RATE_LIMIT_REMAINING

Constructor Summary

Constructors

Constructor	Description
LLMProxyFilter(LLMProvider provider, String baseUrl, String apiKey, Expression<String> sub, boolean rateLimitEnabled, TokenRateLimiter rateLimiter)

Method Summary

Modifier and Type	Method	Description
org.forgerock.util.promise.Promise<org.forgerock.http.protocol.Response,org.forgerock.util.promise.NeverThrowsException>	filter(org.forgerock.services.context.Context context, org.forgerock.http.protocol.Request request, org.forgerock.http.Handler next)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HEADER_LLM_PROVIDER

public static final String HEADER_LLM_PROVIDER

See Also:

Constant Field Values

HEADER_LLM_IDENTITY

public static final String HEADER_LLM_IDENTITY

See Also:

Constant Field Values

HEADER_RATE_LIMIT_REMAINING

public static final String HEADER_RATE_LIMIT_REMAINING

See Also:

Constant Field Values

Constructor Detail

LLMProxyFilter

public LLMProxyFilter(LLMProvider provider,
                      String baseUrl,
                      String apiKey,
                      Expression<String> sub,
                      boolean rateLimitEnabled,
                      TokenRateLimiter rateLimiter)

Method Detail

filter

public org.forgerock.util.promise.Promise<org.forgerock.http.protocol.Response,org.forgerock.util.promise.NeverThrowsException> filter(org.forgerock.services.context.Context context,
                                                                                                                                             org.forgerock.http.protocol.Request request,
                                                                                                                                             org.forgerock.http.Handler next)

Specified by:

filter in interface org.forgerock.http.Filter