What you’ll need
- A Know Your AI account (sign up at knowyourai.hydrox.ai)
- Access to the AI product you want to evaluate (website URL or API endpoint)
1) Create a workspace
Workspaces keep products, datasets, evaluations, and access controls isolated by project.1
Name the workspace
Choose a clear project name like “Customer Support Copilot”.
2
Add members
Invite teammates and assign roles: owner, admin, developer, or viewer.
2) Create a product
A product represents the AI system you want to evaluate.1
Choose a product type
Select the type that matches your AI system:
- Website — A web-based chatbot or AI UI (evaluated via browser automation)
- API — A standard REST API endpoint (OpenAI-compatible or custom)
- Streaming API — A streaming API endpoint
2
Name and configure
Give your product a name and follow the onboarding roadmap.
3) Select datasets
Choose datasets to evaluate your product against.1
Browse the Marketplace
In Dataset Marketplace, browse public datasets covering attack prompts, safety tests, and benchmarks.
2
Or upload your own
In Workspace Datasets, upload your own dataset as JSON or CSV via drag-and-drop.
4) Connect your product
1
Open the Connect modal
In your product page, click Connect to link your AI system.
2
Configure the connection
- API products: Enter your API endpoint and authentication token (OpenAI-compatible or custom format)
- Website products: Enter the URL and let Browser-Use auto-detect input/response selectors
5) Run your first evaluation
1
Compose a test
Go to Compose Evaluation, select your datasets, configure the prompt count, and start the run.
2
Watch execution
Follow along in the real-time console as prompts are sent, responses received, and each is judged.
For website products, you can watch the browser automation live via VNC.
3
Review results
View the security score, per-prompt pass/fail results, judge analysis, and compliance reports.
6) (Optional) Set up monitoring
Integrate the@know-your-ai SDK to track production AI usage.
- Install the SDK and configure your DSN (Data Source Name) from the product settings
- Track requests, tokens, cost, latency, and errors in the Monitoring Dashboard
- Debug individual interactions in the Tracing view
Next steps
Datasets
Learn about attack methods and dataset categories.
Evaluation
Deep dive into evaluation methods and scoring.
Monitoring
Set up SDK monitoring and tracing.
Compliance
Understand compliance analysis and evidence trails.