Health
Backend health surface: latency histogram, throughput, recent 5xx, alert rules, Slack webhook.
Features
- **API usage by endpoint + key** — top callers, slow endpoints, error rates per project.
- **Delivery health per channel** — push delivered / opened / failed, email bounced / opened / clicked, SMS delivered / failed.
- **Latency + error budgets** — p50 / p95 / p99 across modules.
- **AI anomaly alerts** via email / Slack with per-org webhook routing — picks up traffic dips, send-rate spikes, error-rate excursions.
- **Audit log** with actor / resource-type / date-range filters + expandable metadata viewer.
- **Top endpoints (24h) + webhook tester + SSRF-guarded outbound probe** — debug integrations safely.
- **Promote-failed + workflow-failed alerts** surface broken Automation runs before users notice.
Common use cases
- Engineering health dashboards for the Sendora-backed parts of your stack.
- Ops alerting on delivery degradation (sudden push-failure spike from Apple, email bounce-rate breach).
- Compliance audit trail — auditor-ready filterable log of every actor action.
Key concepts
- Alert-rule fields
- name, metric, operator (gt/lt/gte/lte), threshold, windowMinutes, channelIds — plus, on the event_volume metric, an optional eventTypes array. windowMinutes doubles as the re-fire cooldown. An unknown/misspelled field is rejected (422), never silently ignored.
- eventTypes filter (per-signal dead-man's switch)
- Only on event_volume. WITHOUT it, event_volume counts every event type in the project, so `lt 1` fires only when ALL traffic stops — app analytics keep the count warm and hide a single dead job. WITH eventTypes: ["backup.healthy"], it counts only that type and fires when that heartbeat stops. Sent on any other metric it is rejected (422). API-only today; the dashboard alert builder exposes it for event_volume.
- uptime is passive, not synthetic
- uptime, error_rate, latency_p99 and throughput are derived from the API traffic Sendora RECEIVES (org-wide — API logs carry no project dimension), not from an outbound probe. If your box goes dark it stops sending and uptime can't see the silence — use an event_volume + eventTypes dead-man's switch for that. There is no synthetic/external HTTP check today.
Observability
Read health snapshot
FREEReal-time uptime + p99 latency + error rate. Polls the same source the operator dashboard renders.
curl "https://api.sendoracloud.com/api/v1/orgs/<ORG_ID>/observability/health" \
-H "x-api-key: sk_prod_…"Define an alert rule
FREEFire a Slack (or webhook) alert when a metric breaches a threshold, evaluated every minute. Create the channel under Notifications first and reference it by id. Full field set: name, metric, operator, threshold, windowMinutes, channelIds, and — on the event_volume metric — an optional eventTypes filter. ⚠ eventTypes is what makes a per-signal dead-man's switch: WITHOUT it, event_volume counts EVERY event type in the project, so `lt 1` only fires when ALL traffic stops (app analytics keep the count warm and mask a single dead job); WITH `eventTypes: ["backup.healthy"]` it counts only that type, so it fires when that one heartbeat stops. eventTypes is accepted only on event_volume — sent on any other metric it is rejected (422), and an unknown/misspelled field is rejected too (422), so a typo can't arm a rule that looks scoped but isn't. Event metrics (event_volume) can be scoped to one project; API metrics (error_rate, latency_p99, throughput, uptime) are org-wide, because API logs carry no project dimension — and uptime is derived from received traffic, not a synthetic outbound probe. The AI anomaly detector runs separately and can raise alerts on its own.
# Per-signal dead-man's switch: page when the "backup.healthy" heartbeat
# specifically stops, even while other events keep the project busy.
curl -X POST https://api.sendoracloud.com/api/v1/orgs/<ORG_ID>/observability/alert-rules \
-H "x-api-key: sk_prod_…" \
-d '{
"name": "Backups stopped",
"metric": "event_volume",
"operator": "lt",
"threshold": 1,
"windowMinutes": 360,
"eventTypes": ["backup.healthy"],
"channelIds": ["{{CHANNEL_ID}}"]
}'