Get started

Create your first workspace, add a product, and run a security evaluation.

This guide gets you from zero to your first evaluation run with Know Your AI.

What you’ll need

• A Know Your AI account (sign up at knowyourai.hydrox.ai)
• Access to the AI product you want to evaluate (website URL or API endpoint)

1) Create a workspace

Workspaces keep products, datasets, evaluations, and access controls isolated by project.

1. Name the workspace

   Choose a clear project name like “Customer Support Copilot”.

2. Add members

   Invite teammates and assign roles: owner, admin, developer, or viewer.

2) Create a product

A product represents the AI system you want to evaluate.

1. Choose a product type

   Select the type that matches your AI system:

   • Website — A web-based chatbot or AI UI (evaluated via browser automation)
   • API — A standard REST API endpoint (OpenAI-compatible or custom)
   • Streaming API — A streaming API endpoint
2. Name and configure

   Give your product a name and follow the onboarding roadmap.

3) Select datasets

Choose datasets to evaluate your product against.

1. Browse the Marketplace

   In Dataset Marketplace, browse public datasets covering attack prompts, safety tests, and benchmarks.

2. Or upload your own

   In Workspace Datasets, upload your own dataset as JSON or CSV via drag-and-drop.

4) Connect your product

1. Open the Connect modal

   In your product page, click Connect to link your AI system.

2. Configure the connection

   • API products: Enter your API endpoint and authentication token (OpenAI-compatible or custom format)
   • Website products: Enter the URL and let Browser-Use auto-detect input/response selectors

5) Run your first evaluation

1. Compose a test

   Go to Compose Evaluation, select your datasets, configure the prompt count, and start the run.

2. Watch execution

   Follow along in the real-time console as prompts are sent, responses received, and each is judged. For website products, you can watch the browser automation live via VNC.

3. Review results

   View the security score, per-prompt pass/fail results, judge analysis, and compliance reports.

6) (Optional) Set up monitoring

Integrate the @know-your-ai SDK to track production AI usage.

• Install the SDK and configure your DSN (Data Source Name) from the product settings
• Track requests, tokens, cost, latency, and errors in the Monitoring Dashboard
• Debug individual interactions in the Tracing view

Next steps

Datasets

Learn about attack methods and dataset categories.

Evaluation

Deep dive into evaluation methods and scoring.

Monitoring

Set up SDK monitoring and tracing.

Compliance

Understand compliance analysis and evidence trails.