Core platform
| Feature | Description |
|---|---|
| Workspaces | Multi-tenant project isolation with tiers (Free, Pro, Enterprise), quotas, and role-based access |
| Products | Three product types — Website, API, and Streaming API — each with onboarding roadmap and connection wizard |
| Members & invitations | Invite by email with roles (owner, admin, developer, viewer) and invitation lifecycle tracking |
| Usage & quotas | Monthly quota allocation with usage ledger and reset tracking |
Evaluations
| Feature | Description |
|---|---|
| Model Evaluation (API Mode) | Connect your AI model’s API endpoint and run automated red-team testing with attack datasets |
| Chatbot Evaluation (Website Mode) | Evaluate live chatbot websites using a browser control agent for end-to-end red-team testing |
| LLM-as-Judge | Configurable judgment model and prompt with vulnerability scoring |
| Security testing | Red-team your AI with attack prompts and get per-prompt pass/fail |
| Real-time console | Streaming execution logs with prompt/response/judgment bubbles |
| Scheduled evaluations | Cron-based scheduling (hourly, daily, weekly, monthly, custom) with enable/disable |
| Evaluation Market | Browse pre-configured evaluation templates (Safety, Compliance, Quality, Performance) |
| Run history | Full test run history with sortable/filterable table |
| Screenshot library | Browse captured screenshots from chatbot evaluations |
Monitoring & tracing
| Feature | Description |
|---|---|
| SDK integration | @know-your-ai SDK with DSN for automatic event capture |
| Monitoring dashboard | Requests, tokens, cost, latency, errors, provider/model distribution |
| Tracing | Span-tree visualization with d3 — supports generation, agent, tool, chain, retriever, evaluator, guardrail spans |
| Token usage charts | Time-series input/output/total token charts with configurable granularity |
Firewall
| Feature | Description |
|---|---|
| Firewall API keys | Generate and manage firewall API keys per product |
| Validation logs | History of all firewall validations with risk categorization, score, and reason |
| Token usage | Input/output token tracking with time-series analytics |
Compliance
| Feature | Description |
|---|---|
| CCPA/CPRA analysis | Automated compliance judgment with 3-tier violation scoring (Direct, Indirect, Ancillary) |
| Severity levels | None, low, medium, high, critical severity classification |
| Evidence store | Persistent evidence records with full prompt/response pairs and legal references |
| Regulatory coverage | EU AI Act, NIST AI RMF coverage with baseline vs. firewall comparison |
| Per-run reports | Detailed compliance report for each evaluation run |
AI quality metrics
Nine quality dimensions (some coming soon):- Overview, Hallucination Risk, Task Fulfillment, Reasoning Quality, Math & Calculation, Creativity, Empathy, Style & Personality, Safety & Risk
Additional features
| Feature | Description |
|---|---|
| AI chat | Built-in Gemini chat panel with session management and context-aware conversations |
| Security report | Visual before/after comparison showing baseline vs. firewall results |
| Support | In-app bug reports, attack reports, billing issues, and feature requests |
| Internationalization | Multi-language UI support |
| Theme | Light and dark mode |
Evaluation deep dive
Learn about Model Evaluation and Chatbot Evaluation.
Monitoring deep dive
Learn about SDK integration and tracing.