Skip to content

[4.6.0] Add custom error response formatting for AI APIs #11610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
VenukshiMendis wants to merge 1 commit into
base: 4.6.0
Choose a base branch
from
Open

[4.6.0] Add custom error response formatting for AI APIs #11610

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions en/docs/reference/config-catalog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3165,6 +3165,7 @@ enable = true
token = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
endpoint = "https://e95488c8-8511-4882-967f-ec3ae2a0f86f-prod.e1-us-east-azure.choreoapis.dev/lgpt/interceptor-service/interceptor-service-be2/v1.0"
default_request_timeout = 30
custom_error_response_sequence = "openai_error_response_formatter"

[apim.ai.failover_configurations]
failover_endpoints_limit = 10
Expand Down Expand Up @@ -3259,6 +3260,27 @@ default_request_timeout = 30</code></pre>
<p>The default timeout (in seconds) for AI API requests</p>
</div>
</div>
</div><div class="param">
<div class="param-name">
<span class="param-name-wrap"> <code>custom_error_response_sequence</code> </span>
</div>
<div class="param-info">
<div>
<p>
<span class="param-type string"> string </span>

</p>
<div class="param-default">
<span class="param-default-value">Default: <code></code></span>
</div>
<div class="param-possible">
<span class="param-possible-values">Possible Values: <code>openai_error_response_formatter</code></span>
</div>
</div>
<div class="param-description">
<p>The name of the Synapse sequence used to format gateway error responses for AI APIs. Applies only to AI APIs. If not set, AI APIs use the default error response format.</p>
</div>
</div>
</div>
</div><div class="config-wrap">
<code>[apim.ai.failover_configurations]</code>
Expand Down
46 changes: 46 additions & 0 deletions en/docs/reference/troubleshooting/error-handling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,3 +446,49 @@ message's HTTP status code as a custom error message:
{"fault":{"code":900901,"type":"Status report","message":"Runtime Error","description":"Invalid Credentials"}}
```

## Customize Error Responses for AI APIs

To enable custom error response formatting for AI APIs, add the following configuration to the `<API-M_HOME>/repository/conf/deployment.toml` file. For more information, see the [Configuration Catalog]({{base_path}}/reference/config-catalog/).

```toml
[apim.ai]
custom_error_response_sequence = "openai_error_response_formatter"
```

!!! info

In a distributed deployment, add this configuration to the `deployment.toml` file of both the **API Control Plane (ACP)** node (`<ACP_HOME>/repository/conf/deployment.toml`) and the **Universal Gateway** node (`<UNIVERSAL-GW_HOME>/repository/conf/deployment.toml`).

When this configuration is enabled, gateway error responses for AI APIs are formatted using the `<API-M_HOME>/repository/deployment/server/synapse-configs/default/sequences/openai_error_response_formatter.xml` Synapse sequence:

By default, this sequence produces the following OpenAI-compatible error response:

```json
{
"error": {
"code": "<error code>",
"type": "<error type>",
"message": "<error description>"
}
}
```

To customize the error response format, modify the `openai_error_response_formatter.xml` sequence.

!!! note

This configuration applies only to AI APIs. Non-AI APIs, and AI APIs when `custom_error_response_sequence` is not configured, continue to use the default error response format.

!!! info

This configuration is available in wso2am-4.6.0 starting from update level 39 and wso2am-universal-gw-4.6.0 starting from update level 38.

!!! info

For the customization to take effect on **tenants that existed before this update was applied**, copy the following files from the `<API-M_HOME>/repository/resources/apim-synapse-config` directory to the `<API-M_HOME>/repository/tenants/<tenant-id>/synapse-configs/default/sequences` directory:

- `_sandbox_key_error_.xml`
- `guardrail_fault.xml`

Tenants created after applying this update receive these files automatically during tenant creation. Therefore, no manual copying is required.

1 change: 1 addition & 0 deletions en/tools/config-catalog-generator/data/apim.ai.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ enable = true
token = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
endpoint = "https://e95488c8-8511-4882-967f-ec3ae2a0f86f-prod.e1-us-east-azure.choreoapis.dev/lgpt/interceptor-service/interceptor-service-be2/v1.0"
default_request_timeout = 30
custom_error_response_sequence = "openai_error_response_formatter"

[apim.ai.failover_configurations]
failover_endpoints_limit = 10
Expand Down
8 changes: 8 additions & 0 deletions en/tools/config-catalog-generator/data/configs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1212,6 +1212,14 @@
"default": "30",
"possible": "",
"description": "The default timeout (in seconds) for AI API requests"
},
{
"name": "custom_error_response_sequence",
"type": "string",
"required": false,
"default": "",
"possible": "openai_error_response_formatter",
"description": "The name of the Synapse sequence used to format gateway error responses for AI APIs. Applies only to AI APIs. If not set, AI APIs use the default error response format."
}
]
},
Expand Down