AI model execution

The check monitors that the model your service relies on is available and responds. AI providers rarely go down completely: more often a model is removed from publication, a key stops working, or a response that took two seconds starts taking forty. A regular HTTP check won't notice this — it will see the working provider website.

What exactly to check

You choose one level, not a set of checkmarks: they are nested — "model responds" without "provider responds" doesn't happen. The list of available options depends on the provider: "model is loaded into memory" only the provider's own server offers, because only it reports this sign.

Level What is checked Cost
Provider responds Provider returns a list of its models, and the response is parsed. Catches unavailable service, wrong address, and revoked key. Free
Model is with provider The specified model is in that list. Catches model removal from publication and substituted model on own server. Free
Model is loaded into memory The model is not just listed, but held by the server in memory. Catches unloaded-by-idleness model before the first user hits it: unloaded will respond, but will first spend tens of seconds loading. Only available on own server — public vendors don't report their memory. Free
Model executes request A short real request goes to the model. The only level that catches "in list but refuses requests" and model slowdown itself. Charged
Last level costs money. It executes a real request to the model on each check: providers charge for it, your server consumes resources. Check once a minute — that's over forty thousand requests a month. We don't set limits: only you know what the request costs — but interval should be reasonably from 15 minutes. By default the second level is selected, which is free.
Note: a regular API key that you already use to call models is enough. Organization admin key is not needed, and we don't ask for it.
Model ID is checked literally. Starting from the second level we search for the exact string you entered in the provider's list. Some providers list model IDs with release date, and short name works in requests, but doesn't appear in the list — and the check will say the model doesn't exist, though the model is alive. Take the name from the provider's list, not from documentation example.
Your model in internal network is inaccessible. Checks run from our servers, so a model running in local network or behind VPN we can't see — just as no one outside could see it. You can monitor what's accessible from the internet.

Form fields

Field What it means What to specify Example
Name Name of the check. Specify which model and what for is important to you. Support model
Group Allows assigning task to project or section. Select a group if you want to keep project tasks together. Main project
Provider API dialect the service uses. Choose vendor from list. Your servers on Ollama and LM Studio have separate items: only they show if model is loaded into memory. For everything else — different engine, gateway, lesser-known vendor — take "OpenAI-compatible API". OpenAI-compatible API
API address Where to request. Asked for compatible API and own server — for public vendors the address is known in advance. What exactly to enter, is hinted below the field: compatible API needs address with version, own server — address without path. https://api.openai.com/v1
API key Key for making the request. Paste key from provider's account. For GigaChat it's project authorization key — not client ID or secret separately. For Yandex AI Studio — API key of service account. Own server may have no key at all — then leave the field empty; if closed by proxy, enter its token. When updating task, empty field means "keep previous key". sk-…
Catalog Cloud catalog where models run. Asked only from Yandex AI Studio. Enter catalog ID where service account with key is set up. It's also in model ID after gpt://. b1g…
API version Key's access scope. Asked only from GigaChat. Select the one specified in your account next to authorization key: for individuals, for business with packages or pay-as-you-go. Key from one version won't be accepted in another. Individuals
Model ID of the model we're monitoring. Specify it exactly as provider returns it in models list: check is exact. Own server often has path or alias, not familiar short name. Yandex AI Studio ID includes catalog: gpt://<catalog>/yandexgpt/latest. gpt-4o-mini
What to check Check level from described above. Which are available depends on provider. Start with the second. Enable the last when it's important to know the model actually responds, not just listed. Model is with provider
Request type What model does: chat or vectors. Asked only at last level. Choose what model is designed for. Chat request to embedding model provider will rightfully reject, and check will fail on healthy model. Chat request
Timeout How much time allowed for the entire check, at any level. Also a threshold: check longer than this is failure — provider that responds but slow is also broken. Take with margin for cold load: server that loads model on first request responds noticeably slower than usual. 1 minute
Interval How often to run the check. On free levels — as convenient. On the last remember each check is charged. 5 minutes
Notification channels Where to send messages about issue and recovery. Check channels where you actually read notifications. Telegram and email
Reports Periodic summaries for the task. Enable if you want overview without manual panel access. Weekly report

When alert comes

Alert comes if selected check fails:

  • provider unavailable, responded with error, or rejected key;
  • specified model not in list — from second level on;
  • model exists but server doesn't hold it in memory — at "model is loaded into memory" level;
  • model replied with refusal — at last level;
  • entire check took longer than timeout — at any level.

When check is skipped

If provider asked to check less often, task doesn't fail or recover — check is skipped, status stays the same. This is the only such case: frequency limit means our own pushiness, not your model's problem. Everything else — silent network, provider outage, rejected key — for this task means the model doesn't work.

One task — one model

Monitoring five models — that's five tasks. And if you want to check model list often, and live request rarely, create two tasks with different levels and different intervals: task has one interval and one uptime, and mixing two questions in one, you get uptime you don't understand whose it is.

Notification settings

For each notification channel in a task additional parameters can be set. They apply only to DOWN events.

Parameter What it does
Enabled Allows or disables sending notifications via this channel for this task.
Delay for PRO Delays first DOWN notification for selected time (up to 12 hours). Convenient to avoid alerts for short-term failures.
Repeat every PRO Periodically repeats notification while task stays in DOWN status (up to 1 hour between repeats). Suitable when important not to miss prolonged failure.
"Delay" and "Repeat" parameters don't affect UP events and recovery notifications.
To the list of task types