MCP tools

KimaAI Pro’s MCP server lets an AI client, such as Claude, read how the chatbot does and fix it: POST /wp-json/kimaai/v1/mcp (see the MCP REST routes). An administrator switches it on in KimaAI’s MCP settings and issues a token there or with wp kimaai mcp token create.

A tool needs a scope. A token holds one or more: read reads figures and conversations with personal data masked, read:raw reads them unmasked, write changes settings, eval cases and the index, admin holds all of them; write and read:raw include read. A client sees only the tools its token’s scopes allow. Each call is written to the MCP audit log.

usage_stats

  • Scope: read
  • Tier: premium

Token usage of the chatbot per day (input, output, total and calls) over the last days, and the current month’s total.

ArgumentTypeRequiredDescription
daysintegernoNumber of days to report (default 30).

knowledge_gaps

  • Scope: read
  • Tier: premium

The questions visitors asked recently, in clusters, ranked by negative feedback and then by volume: what visitors ask most and where the chatbot struggles. Eval sessions are left out.

ArgumentTypeRequiredDescription
daysintegernoLook back this many days (default 30).
min_clusterintegernoMinimum cluster size to report (default 2).

feedback_overview

  • Scope: read
  • Tier: premium

The feedback figures of a period: helpful and not helpful answers and their rate, the trend per day, and the answers rated down most.

ArgumentTypeRequiredDescription
rangeintegernoDays to include (default 30). Ignored if from/to are given.
fromstringnoStart date (YYYY-MM-DD), optional.
tostringnoEnd date (YYYY-MM-DD), optional.

list_conversations

  • Scope: read
  • Tier: premium

One page of rated answers with their conversation, filtered by rating, comment, provider, model or post. Personal data in the conversations is masked unless the token has read:raw.

ArgumentTypeRequiredDescription
rangeintegernoDays to include (default 30).
fromstringnoStart date (YYYY-MM-DD), optional.
tostringnoEnd date (YYYY-MM-DD), optional.
pageintegernoPage number (default 1).
per_pageintegernoRows per page (10-100, default 25).
ratingintegernoFilter by rating: 1 (up), -1 (down), 0 (all).
has_commentintegerno1 to only include rows with a comment.
providerstringnoFilter by AI provider slug.
modelstringnoFilter by AI model.
post_idintegernoFilter by originating post ID.

get_incident

  • Scope: read
  • Tier: premium

The conversation around one rated answer, three turns before and after, with how the answer was found. Masked unless the token has read:raw.

ArgumentTypeRequiredDescription
session_idstringyesThe chatbot session ID.
message_idstringyesThe rated message ID.

eval_history

  • Scope: read
  • Tier: premium

The stored eval runs with their scores, to see whether the answers get better.

ArgumentTypeRequiredDescription
limitintegernoReturn only the most recent N runs (default all, max 30).

ask_chatbot

  • Scope: read
  • Tier: premium

Asks the chatbot a question through the real pipeline and returns its answer and the tools it called, to reproduce a reported bad answer. It runs in an internal session that the analytics leave out, and it calls the AI provider.

ArgumentTypeRequiredDescription
messagestringyesThe user question to ask.
langstringnoLanguage code (e.g. “de”). Recommended: retrieval and tools are language-scoped.
post_idintegernoOriginating post ID for page-aware answers (optional).

inspect_retrieval

  • Scope: read
  • Tier: premium

Runs the chatbot’s knowledge search for a query and returns the hits with their similarity, the condensed query, the scope (language, post types) and the search’s settings.

ArgumentTypeRequiredDescription
querystringyesThe query to retrieve for.
langstringnoLanguage code (e.g. “de”) for language-scoped retrieval.

diagnose_query

  • Scope: read
  • Tier: premium

Searches the knowledge for a query and tells what most likely went wrong (content missing, weak or different wording, or content found so the routing needs a look), with a suggested fix.

ArgumentTypeRequiredDescription
querystringyesThe user query that produced a bad answer.
langstringnoLanguage code (e.g. “de”).

check_coverage

  • Scope: read
  • Tier: premium

Checks an embedding environment’s health and settings and, for a post, how many of its chunks are in the index.

ArgumentTypeRequiredDescription
envstringnoEmbedding environment id (default “default”).
post_idintegernoIf set, report how many chunks of this post are embedded.

get_settings

  • Scope: read
  • Tier: premium

Lists the settings update_settings can change, with their current values; secrets are masked.

ArgumentTypeRequiredDescription
prefixstringnoOnly paths starting with this (e.g. “chatbot.settings”). Default “chatbot”.
querystringnoOnly paths/labels containing this substring.

update_settings

  • Scope: write
  • Tier: premium

Changes settings by their path. It shows a preview unless confirm is true; an applied change returns a rollbackSessionId for rollback_change.

ArgumentTypeRequiredDescription
operationsarrayyesList of {path, value} or {path, textEdit} operations.
confirmbooleannoApply the change (default false = preview only).

rollback_change

  • Scope: write
  • Tier: premium

Undoes the settings changes made under a rollbackSessionId.

ArgumentTypeRequiredDescription
rollbackSessionIdstringyesThe session id to revert.

run_eval

  • Scope: write
  • Tier: premium

Runs the stored eval cases through the chatbot and returns how many passed; the run is stored in the eval history. Each case calls the AI provider.

ArgumentTypeRequiredDescription
onlystringnoOnly run cases in this category.
limitintegernoMaximum number of cases to run.
langstringnoLanguage code for the chatbot requests (e.g. “de”).
judgebooleannoAlso run the LLM judge (slower).

reembed

  • Scope: write
  • Tier: premium

Embeds content again to fix missing or stale knowledge: one post, a post type, or modified content. A dry run unless confirm is true; on a site whose MCP environment is production it writes only with allow_production too.

ArgumentTypeRequiredDescription
post_idintegernoEmbed just this post.
post_typestringnoEmbed this post type.
forcebooleannoRe-embed even if unchanged.
limitintegernoMaximum posts to process (default 50).
envstringnoEmbedding environment id (default “default”).
confirmbooleannoActually write vectors (default false = dry run).
allow_productionbooleannoRequired to write vectors when the environment is production; ignored on staging. Without it a confirmed run on production is forced back to a dry run.

add_eval_case

  • Scope: write
  • Tier: premium

Adds an eval case that locks in a fix: a question with what the answer must and must not contain. Returns a rollbackSessionId; run run_eval afterwards.

ArgumentTypeRequiredDescription
caseIdstringyesUnique case id.
turnsarrayyesConversation turns (usually one user question).
categorystringnoCategory (default “general”).
mustIncludearraynoAssertions the answer must contain (use “ab” for any-of).
mustNotIncludearraynoStrings the answer must not contain.
expectedFactsstringnoFree-text expected facts (for the judge).

What are your feelings