Test run feature

[ralyodio]

📋 Product Requirements Document

PRD: Test run feature

Issue: #135
Milestone: Phase 3: Frontend (SvelteKit)
Labels: run-console, hacktoberfest

---

PRD: Test Run Feature

Overview

The objective of the "Test Run Feature" is to enhance MeshHook's capabilities by allowing users to execute workflow tests directly from the Run Console. This aligns with MeshHook's goals by improving user experience, facilitating debugging, and ensuring workflow accuracy before deployment. It serves the crucial need for users to validate their workflows in a controlled environment, ensuring that each component functions as expected under various conditions.

Functional Requirements

1. Test Execution: Users must be able to initiate test runs of workflows from the Run Console.
2. Parameter Configuration: Enable users to input or modify parameters (payloads, headers) for test executions.
3. Step-by-Step Execution: Allow users to execute workflows step-by-step to facilitate debugging.
4. Test Run Logs: Display live logs and results for each step in the test run.
5. Result Visualization: Provide a visual representation of the test run's outcome, highlighting success, failure, and skipped steps.
6. Historical Test Runs: Store and allow users to view past test runs, including parameters used and the results for each run.
7. Environment Isolation: Ensure that test runs are executed in an isolated environment to prevent interference with live data.

Non-Functional Requirements

• Performance: Test runs should complete within an acceptable timeframe, not significantly exceeding the execution time of live runs.
• Reliability: Ensure high availability of the test run feature, with proper error handling and recovery mechanisms in place.
• Security: Test runs must operate under the existing security model, ensuring that sensitive data remains protected.
• Usability: The UI/UX for initiating test runs and viewing results should be intuitive and align with the existing design system.

Technical Specifications

Architecture Context

This feature integrates with the existing SvelteKit-based frontend for the Run Console and leverages Supabase Realtime for displaying live logs. It will interact with backend components to initiate and monitor test runs, requiring close integration with the orchestrator and HTTP Executor components.

Implementation Approach

1. Design UI/UX: Develop the interface for initiating test runs and viewing results within the Run Console.
2. API Extension: Extend the backend API to support initiating test runs with customizable parameters.
3. Orchestrator Modification: Enhance the orchestrator to differentiate between test and live runs, ensuring isolation.
4. Logging and Storage: Implement mechanisms to capture and display test run logs, storing historical test run data.
5. Integration Testing: Ensure the feature integrates seamlessly with existing components, particularly focusing on the live log display and parameter customization.
6. User Documentation: Update user guides and documentation to include the test run feature.

Data Model Changes

• TestRuns: A new table to store metadata, parameters, and results of each test run.
  • Columns: id, workflow_id, created_at, parameters, status, result.

API Endpoints

• POST /test-runs: Initiates a new test run with provided parameters.
• GET /test-runs/{id}: Retrieves the status and result of a specific test run.
• GET /test-runs: Lists historical test runs for a workflow.

Acceptance Criteria

• Users can initiate test runs from the Run Console.
• Test runs allow for customizable parameters.
• Live logs are displayed during the test run.
• Results of test runs are accurately recorded and can be reviewed.
• Test runs do not affect live data or operations.
• New functionality fits seamlessly into the existing UI/UX.
• Documentation is updated to reflect the new feature.
• All new code is covered by unit and integration tests.

Dependencies

• Supabase Realtime for live log updates.
• Existing backend API and orchestrator for workflow execution.
• SvelteKit/Svelte 5 for frontend development.

Implementation Notes

Development Guidelines

• Follow the established coding standards and project structure.
• Ensure code is well-commented and adheres to security best practices.
• Write unit and integration tests alongside new features.

Testing Strategy

• Unit Tests: For all new backend logic and utility functions.
• Integration Tests: Covering the interaction between the frontend, API, and orchestrator.
• E2E Tests: Simulating user actions for initiating and viewing test runs.

Security Considerations

• Validate all user inputs to prevent injection attacks.
• Ensure test runs are executed with the same security constraints as live executions.

Monitoring & Observability

• Extend existing monitoring to cover the test run feature, particularly focusing on performance and error rates.
• Ensure all actions and outcomes are logged for auditability.

Related Documentation

• Update the Run Console section in the User Guide to include test run instructions.
• Technical documentation should be updated to describe the changes in API endpoints and data models.

---

This PRD was AI-generated using gpt-4-turbo-preview from GitHub issue #135
Generated: 2025-10-10

📎 Generated Documentation

• 📄 PRD Document: 135-test-run-feature.md
• 🎨 PlantUML Diagram: 135-test-run-feature.puml
• 🖼️ Diagram Image: 135-test-run-feature.png

---

This issue body was auto-generated from the PRD. Original issue content is preserved in the PRD document.
Last updated: 2025-10-10