When it fires
At the end of a scheduled run of the evaluation cases (Chatbot, Scheduled runs, Run on a schedule), when the share of passed cases is below “Alert threshold (%)”. The run’s batches run in the background through Action Scheduler, so the action fires in a background job, after the run was saved and after KimaAI sent its own alert to “Alert email” and “Alert webhook (optional)”, when they are set. Runs from WP-CLI do not fire it.
Use it to send the alert where your team works, in the format your tool expects, or to open a ticket.
Parameters
| Name | Type | Description |
|---|---|---|
summary | array | The run’s result, with total, passed, percent and categories (per category its total and pass). |
failures | array | The failed cases, each with id, category, pass, failures (the reasons), answer and seconds. |
threshold | int | The alert threshold in percent, 95 by default. |
Example
add_action(
'kimaai_eval_regression',
function (array $summary, array $failures, int $threshold): void {
// The team chat expects {"text": "..."}.
$lines = [sprintf('Chatbot quality fell to %d%% (alert below %d%%).', $summary['percent'], $threshold)];
foreach ($failures as $failure) {
$lines[] = sprintf('- %s: %s', $failure['id'], implode('; ', $failure['failures']));
}
wp_remote_post('https://chat.example.com/hooks/quality', [
'headers' => ['Content-Type' => 'application/json'],
'body' => wp_json_encode(['text' => implode("\n", $lines)]),
]);
},
10,
3
);
Related
kimaai_chat_response_saved: fires for every saved answer, the evaluation’s included.