Skip to main content
This page outlines practical workflows for teams using Know Your AI day-to-day.

​
1) Set up your workspace

Start by creating a workspace and inviting your team.
  • Name it after your product or project
  • Assign roles: owner for the lead, admin for managers, developer for engineers, viewer for stakeholders
  • Check your quota allocation in Usage & Limit

​
2) Create and connect products

Add each AI system you want to evaluate as a product.
  • Website products: Enter the URL. Browser-Use will auto-detect input/response selectors using Gemini Vision. You can watch evaluations live via VNC.
  • API products: Enter your endpoint URL and authentication token. Supports OpenAI-compatible and custom APIs.
  • Streaming API products: Configure for streaming response endpoints.
Follow the onboarding roadmap on each product page: Select Datasets → Connect API → Start Test → Complete.

​
3) Build your dataset library

Use a mix of sources:
  • Browse the Dataset Marketplace for community and system-curated attack prompts
  • Upload custom datasets (JSON/CSV) in Workspace Datasets with input, golden answer, and knowledge fields
  • Cover the key attack categories: jailbreak, prompt injection, data extraction, harmful content, PII leakage, bias, hallucination

​
4) Run security evaluations

Compose and execute evaluation runs.
  • Go to Compose Evaluation and select your datasets
  • Configure the prompt count (or use random sampling)
  • Start the run and watch the real-time console for streaming execution logs
  • For website products, use the VNC viewer for live browser observation

​
5) Review results

Each evaluation run produces:
  • Security score with vulnerability percentage
  • Per-prompt pass/fail with judge analysis explaining each decision
  • Compliance report with CCPA/CPRA violation analysis
  • Screenshots captured during browser automation (website products)
  • Execution logs with prompt, response, and judgment details

​
6) Set up scheduled evaluations

Automate recurring security testing:
  • Go to Scheduled Tasks in your product sidebar
  • Configure schedules: hourly, daily, weekly, monthly, or custom cron
  • Enable/disable schedules as needed
  • Track scheduled run results in Test History

​
7) Integrate SDK monitoring

Deploy the @know-your-ai SDK to monitor your AI system in production:
  • Install the SDK and configure your DSN from product settings
  • Monitor in the Monitoring Dashboard: requests, tokens, cost, latency, errors
  • Debug issues in the Tracing view with span-tree visualization
  • Track token usage trends over time

​
8) Enable the firewall

Add real-time protection with the firewall:
  • Generate a Firewall API key in your product’s Firewall settings
  • Integrate the firewall SDK into your application
  • Monitor validations in Firewall Logs with risk categorization
  • Track token usage in Firewall Token Usage

​
9) Track compliance

Stay on top of regulatory requirements:
  • Review the Compliance dashboard for aggregated violation counts
  • Explore per-run compliance reports with evidence
  • Track CCPA/CPRA, EU AI Act, and NIST AI RMF coverage
  • Use the Security Report for baseline vs. firewall comparisons

​
Product sidebar reference

When inside a product, the sidebar provides:

Get started

Set up your first evaluation.

Features

Full feature reference.