Skip to content

Guardrails UI: include in config editor #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nishika26 wants to merge 13 commits into from
Closed

Guardrails UI: include in config editor #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
464887c
Evals UI: fixing score and config display
nishika26 Mar 8, 2026
da559ec
score display fix
nishika26 Mar 9, 2026
067ea27
removing un-needed words from comment
nishika26 Mar 9, 2026
99a4610
coderabbit review
nishika26 Mar 9, 2026
9cc6b14
vector store id to knowledge base id
nishika26 Mar 9, 2026
2bbcfbd
ciderabbit and vector store to knowledge base
nishika26 Mar 9, 2026
855281a
Guardrails UI: add guardrails to config
nishika26 Mar 10, 2026
2455ed6
second commit: made guardrail page better
nishika26 Mar 15, 2026
9690443
third commit including ban list creation
nishika26 Mar 16, 2026
b0b3572
remove guardrail token from keystore, add to env
nishika26 Mar 17, 2026
6a8493c
remove guardrail token from keystore, add to env
nishika26 Mar 17, 2026
4871f2a
remove guardrail token from keystore
nishika26 Mar 17, 2026
9ee79ac
remove guardrail token from keystore
nishika26 Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
NEXT_PUBLIC_BACKEND_URL=

GUARDRAIL_TOKEN=
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add blank line at the end of the file.

As per dotenv-linter, environment files should end with a blank line.

📝 Proposed fix 
 GUARDRAILS_TOKEN=
+
🧰 Tools
🪛 dotenv-linter (4.0.0)

[warning] 3-3: [EndingBlankLine] No blank line at the end of the file

(EndingBlankLine)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.env.example at line 3, Add a trailing blank newline at the end of the
.env.example file so it ends with an empty line (i.e., after the existing
GUARDRAIL_TOKEN= entry); update the file to ensure the final character is a
newline to satisfy dotenv-linter.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check which form of the guardrails token variable is used in the codebase

echo "=== Searching for GUARDRAILS_TOKEN (plural) ==="
rg -n 'GUARDRAILS_TOKEN' --type=ts --type=js -C2

echo ""
echo "=== Searching for GUARDRAIL_TOKEN (singular) ==="
rg -n 'GUARDRAIL_TOKEN' --type=ts --type=js -C2

Repository: ProjectTech4DevAI/kaapi-frontend

Length of output: 2381

🏁 Script executed:

cat -n .env.example | head -10

Repository: ProjectTech4DevAI/kaapi-frontend

Length of output: 139

Environment variable name mismatch will cause runtime errors.

The .env.example file defines GUARDRAIL_TOKEN (singular), but the code reads GUARDRAILS_TOKEN (plural). This mismatch will cause guardrails validator/config routes to fail at runtime with "Missing GUARDRAILS_TOKEN environment variable" errors.

🔧 Proposed fix 
-GUARDRAIL_TOKEN=
+GUARDRAILS_TOKEN=
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
GUARDRAIL_TOKEN=
GUARDRAILS_TOKEN=
🧰 Tools
🪛 dotenv-linter (4.0.0)

[warning] 3-3: [EndingBlankLine] No blank line at the end of the file

(EndingBlankLine)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.env.example at line 3, Update the environment variable in the example to
match the code: replace GUARDRAIL_TOKEN with GUARDRAILS_TOKEN (the name the code
reads) so the runtime finds the expected variable; look for occurrences of
GUARDRAIL_TOKEN and change them to GUARDRAILS_TOKEN in the .env example (or
alternatively change the code references to GUARDRAIL_TOKEN if you prefer to
standardize on the singular name), ensuring the symbol GUARDRAILS_TOKEN used by
the guardrails validator/config routes is present.

44 changes: 44 additions & 0 deletions app/api/apikeys/verify/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import { NextRequest, NextResponse } from 'next/server';

const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8000';
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Default backend port is inconsistent with sibling guardrails routes.

Line 3 falls back to http://localhost:8000, while other proxy routes use http://localhost:8001. This can silently break local verification when NEXT_PUBLIC_BACKEND_URL is unset.

🔧 Proposed fix 
-const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8000';
+const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8001';
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8000';
const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8001';
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@app/api/apikeys/verify/route.ts` at line 3, The fallback backend URL in
route.ts uses the constant backendUrl set to 'http://localhost:8000', which is
inconsistent with sibling guardrails routes; update the backendUrl default to
'http://localhost:8001' so local verification uses the same proxy port when
NEXT_PUBLIC_BACKEND_URL is unset, and scan other proxy/guardrails routes to
ensure they all use the 8001 fallback for consistency.


export async function GET(request: NextRequest) {
try {
// Get the API key from request headers
const apiKey = request.headers.get('X-API-KEY');
if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

// Forward the request to the actual backend
const url = `${backendUrl}/api/v1/apikeys/verify`;

const response = await fetch(url, {
method: 'GET',
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : {};

Comment on lines +27 to +30
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Guard JSON parsing to avoid false 500s on non-JSON backend bodies.

At Line 29, JSON.parse(text) can throw for plain-text/error HTML responses, causing this route to return a proxy failure instead of the backend status.

🔧 Proposed fix 
-    const text = await response.text();
-    const data = text ? JSON.parse(text) : {};
+    const text = await response.text();
+    let data: unknown = {};
+    if (text) {
+      try {
+        data = JSON.parse(text);
+      } catch {
+        data = { message: text };
+      }
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : {};
// Handle empty responses (204 No Content, etc.)
const text = await response.text();
let data: unknown = {};
if (text) {
try {
data = JSON.parse(text);
} catch {
data = { message: text };
}
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@app/api/apikeys/verify/route.ts` around lines 27 - 30, The current code calls
JSON.parse(text) unguarded which will throw on non-JSON backend bodies and turn
valid backend errors into 500s; update the parsing in the route handler to only
parse when the response is JSON (check response.headers.get('content-type') for
'application/json' or 'application/hal+json') and fallback to returning the raw
text (or an empty object) when not JSON, or wrap JSON.parse in a try/catch and
use the caught error to fall back to the raw text; adjust the variables `text`
and `data` in route.ts (the response handling block) so downstream logic
consumes a safe `data` value without letting parse errors propagate.

// Return the response with the same status code
if (!response.ok) {
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
Comment on lines +39 to +41
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Do not expose raw internal error details to clients.

Line 40 returns error.message directly, which can leak internal service/network details. Keep details in server logs and return a generic client-safe error payload.

🔧 Proposed fix 
     return NextResponse.json(
-      { error: 'Failed to forward request to backend', details: error.message },
+      { error: 'Failed to forward request to backend' },
       { status: 500 }
     );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
return NextResponse.json(
{ error: 'Failed to forward request to backend' },
{ status: 500 }
);
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@app/api/apikeys/verify/route.ts` around lines 39 - 41, The response currently
returns raw error.message to clients from the NextResponse.json call in the
verify route; remove exposing internal error details by returning a generic
client-safe payload (e.g. { error: "Failed to forward request to backend" }) and
keep the full error in server logs instead (use the existing logger or
console.error to log error and any request context). Ensure the status remains
500, do not include error.message in the JSON, and optionally generate/log an
errorId if you want to correlate client errors with server logs.

);
}
}
3 changes: 2 additions & 1 deletion app/api/evaluations/[id]/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,12 @@ export async function GET(

const searchParams = request.nextUrl.searchParams;
const exportFormat = searchParams.get('export_format') || 'row';
const resyncScore = searchParams.get('resync_score') || 'false';

// Build URL with query parameters
const url = new URL(`${backendUrl}/api/v1/evaluations/${id}`);
url.searchParams.set('get_trace_info', 'true');
url.searchParams.set('resync_score', 'false');
url.searchParams.set('resync_score', resyncScore);
url.searchParams.set('export_format', exportFormat);

console.log(`[REAL BACKEND] Fetching evaluation ${id} from ${url.toString()}`);
Expand Down
157 changes: 157 additions & 0 deletions app/api/guardrails/ban_lists/[ban_list_id]/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
import { NextResponse } from 'next/server';

const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8001';

export async function GET(
request: Request,
{ params }: { params: Promise<{ ban_list_id: string }> }
) {
const { ban_list_id } = await params;
const apiKey = request.headers.get('X-API-KEY');

if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

try {
const url = `${backendUrl}/api/v1/guardrails/ban_lists/${ban_list_id}`;

console.log('[GET /api/guardrails/ban_lists/[ban_list_id]] Forwarding to:', url);

const response = await fetch(url, {
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

console.log('[GET /api/guardrails/ban_list/[ban_list_id]] Backend response status:', response.status, response.statusText);

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : {};

console.log('[GET /api/guardrails/ban_list/[ban_list_id]] Backend response data:', JSON.stringify(data, null, 2));

if (!response.ok) {
console.error('[GET /api/guardrails/ban_list/[ban_list_id]] Backend error:', response.status, data);
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
);
}
}

export async function PUT(
request: Request,
{ params }: { params: Promise<{ ban_list_id: string }> }
) {
const { ban_list_id } = await params;
const apiKey = request.headers.get('X-API-KEY');

if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

try {
// Get the JSON body from the request
const body = await request.json();

const url = `${backendUrl}/api/v1/guardrails/ban_lists/${ban_list_id}`;

console.log('[PUT /api/guardrails/ban_list/[ban_list_id]] Forwarding to:', url);
console.log('[PUT /api/guardrails/ban_list/[ban_list_id]] Body:', JSON.stringify(body, null, 2));

const response = await fetch(url, {
method: 'PUT',
body: JSON.stringify(body),
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

console.log('[PUT /api/guardrails/ban_list/[ban_list_id]] Backend response status:', response.status, response.statusText);

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : { success: true };

console.log('[PUT /api/guardrails/ban_list/[ban_list_id]] Backend response data:', JSON.stringify(data, null, 2));

if (!response.ok) {
console.error('[PUT /api/guardrails/ban_list/[ban_list_id]] Backend error:', response.status, data);
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
);
}
}

export async function DELETE(
request: Request,
{ params }: { params: Promise<{ ban_list_id: string }> }
) {
const { ban_list_id } = await params;
const apiKey = request.headers.get('X-API-KEY');

if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

try {
const url = `${backendUrl}/api/v1/guardrails/ban_lists/${ban_list_id}`;

console.log('[DELETE /api/guardrails/ban_lists/[ban_list_id]] Forwarding to:', url);

const response = await fetch(url, {
method: 'DELETE',
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

console.log('[DELETE /api/guardrails/ban_list/[ban_list_id]] Backend response status:', response.status, response.statusText);

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : { success: true };

console.log('[DELETE /api/guardrails/ban_list/[ban_list_id]] Backend response data:', JSON.stringify(data, null, 2));

if (!response.ok) {
console.error('[DELETE /api/guardrails/ban_list/[ban_list_id]] Backend error:', response.status, data);
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
);
}
}
104 changes: 104 additions & 0 deletions app/api/guardrails/ban_lists/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
import { NextRequest, NextResponse } from 'next/server';

const backendUrl = process.env.NEXT_PUBLIC_BACKEND_URL || 'http://localhost:8001';

export async function GET(request: NextRequest) {
try {
// Get the Kaapi API key from request headers
const apiKey = request.headers.get('X-API-KEY');
if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

const url = `${backendUrl}/api/v1/guardrails/ban_lists`;

console.log('[GET /api/guardrails/ban_list] Forwarding to:', url);

// Forward the request to the actual backend
const response = await fetch(url, {
method: 'GET',
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

console.log('[GET /api/guardrails/ban_list] Backend response status:', response.status, response.statusText);

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : { data: [] };

console.log('[GET /api/guardrails/ban_list] Backend response data:', JSON.stringify(data, null, 2));

// Return the response with the same status code
if (!response.ok) {
console.error('[GET /api/guardrails/ban_list] Backend error:', response.status, data);
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
);
}
}

export async function POST(request: NextRequest) {
try {
// Get the Kaapi API key from request headers
const apiKey = request.headers.get('X-API-KEY');
if (!apiKey) {
return NextResponse.json(
{ error: 'Missing X-API-KEY header' },
{ status: 401 }
);
}

// Get the JSON body from the request
const body = await request.json();

const url = `${backendUrl}/api/v1/guardrails/ban_lists`;

console.log('[POST /api/guardrails/ban_list] Forwarding to:', url);
console.log('[POST /api/guardrails/ban_list] Body:', JSON.stringify(body, null, 2));

// Forward the request to the actual backend
const response = await fetch(url, {
method: 'POST',
body: JSON.stringify(body),
headers: {
'X-API-KEY': apiKey,
'Content-Type': 'application/json',
},
});

console.log('[POST /api/guardrails/ban_list] Backend response status:', response.status, response.statusText);

// Handle empty responses (204 No Content, etc.)
const text = await response.text();
const data = text ? JSON.parse(text) : { success: true };

console.log('[POST /api/guardrails/ban_list] Backend response data:', JSON.stringify(data, null, 2));

// Return the response with the same status code
if (!response.ok) {
console.error('[POST /api/guardrails/ban_list] Backend error:', response.status, data);
return NextResponse.json(data, { status: response.status });
}

return NextResponse.json(data, { status: response.status });
} catch (error: any) {
console.error('Proxy error:', error);
return NextResponse.json(
{ error: 'Failed to forward request to backend', details: error.message },
{ status: 500 }
);
}
}
Loading