Skip to main content

Testworthy Documentation

Welcome to Testworthy - a modern, AI-powered platform that streamlines software testing workflows for quality assurance teams. This documentation will help you understand, deploy, and use the platform effectively.

Quick Navigation​

  • New to Testworthy? Start with the Introduction to understand core concepts
  • Ready to get started? Jump to the Quick Start Guide
  • End user? Browse the User Guide for detailed feature documentation
  • Using MCP? See the MCP User Guide for IDE integration with AI assistants

What is Testworthy?​

Testworthy is a comprehensive AI-powered test management platform that transforms how quality assurance teams approach software testing. Built as an enterprise SaaS solution, it combines intelligent automation with advanced test management capabilities to help teams deliver high-quality software faster and more efficiently.

Enterprise Platform Overview​

Testworthy is designed as a multi-tenant SaaS platform with enterprise-grade features including subscription management, organization workspaces, advanced security, and seamless integrations. Whether you're a small QA team or managing enterprise-scale testing, Testworthy scales with your needs.

Key Features​

1. Advanced Test Management​

  • Hierarchical Organization: Organize tests with projects, test suites, sections, and test cases with nested hierarchies
  • Comprehensive Test Lifecycle: Create, execute, track, and analyze tests with complete version control
  • Test Plans and Runs: Build structured test plans, execute coordinated test runs, and track detailed results
  • Milestone Management: Organize testing around project milestones with progress tracking
  • Custom Fields: Extend test cases with custom metadata fields to match your workflow
  • Version Control: Complete history tracking with the ability to restore previous versions

2. AI-Powered Automation​

  • AI Integration: Leverage powerful AI for intelligent test generation and analysis
  • Automated Test Generation: Generate comprehensive test cases from requirements or uploaded code
  • Test Case Healing: AI-powered improvement and maintenance of existing test cases
  • Automation Script Export: Generate executable automation scripts for Playwright, Selenium, and Cypress
  • Code Analysis: Upload codebases or PRD documents for automated project structure generation
  • Credit Management: Transparent AI usage tracking with flexible credit quota system

3. Repository Integration & PR Analysis​

  • GitHub, GitLab & Bitbucket Integration: Connect repositories with webhook support for automated analysis
  • Pull Request Analysis: Automatic analysis of code changes with AI-powered test recommendations
  • Test Impact Detection: Intelligent suggestions for which tests need updates based on code changes
  • Automated Test Creation: Generate new test cases automatically from repository changes
  • Confidence Scoring: AI-ranked suggestions with detailed reasoning

4. Enterprise Multi-Tenancy​

  • Organization Workspaces: Custom subdomain-based workspaces with complete data isolation
  • Advanced User Management: Role-based access control with 6 roles (Platform Admin, Tenant Owner, Project Manager, QA, Developer, Guest) and project-level roles
  • User Creation: Direct user creation by administrators with role assignment, credit quota allocation, and password policy enforcement
  • Subscription Management: Per-organization billing and quota management
  • Security & Compliance: Enterprise-grade security with encrypted storage and audit trails

5. Billing & Subscription Management​

  • Flexible Subscription Tiers: Trial, Starter, Standard, and Professional plans with transparent pricing
  • Stripe Integration: Secure payment processing with self-service billing portal
  • Credit-Based Usage: Pay-as-you-use AI features with monthly credit quotas and credit top-ups
  • Grace Period Management: Industry-leading subscription controls with cancellation undo
  • Payment & Invoice Management: Complete payment history and automated PDF invoicing

6. Defect Management & External Integration​

  • External Issue Tracking: Direct integration with Jira, GitHub Issues, and Bugzilla
  • Defect Lifecycle: Complete status tracking from creation to resolution
  • Attachment Support: Link files and evidence directly to defects
  • Activity Audit Trail: Complete history of all defect-related activities

7. Advanced Reporting & Analytics​

  • Multiple Report Types: Generate milestone, test plan, and test run reports
  • Scheduled Reports: Automated daily, weekly, or monthly PDF report delivery
  • Real-time Dashboards: Live project health and execution metrics
  • Usage Analytics: Detailed AI credit usage tracking and cost management
  • Access Control: Fine-grained report sharing with selected users or teams

8. Collaboration & Communication​

  • Real-time Comments: Discussion threads on test cases with email notifications
  • Activity Feeds: Comprehensive audit logs for all project activities
  • Email Notifications: Automated alerts for quota limits, project updates, and more
  • Team Collaboration: Multi-user editing with conflict resolution

9. Security & Authentication​

  • JWT Authentication: Secure token-based authentication with refresh token support
  • OTP Verification: Email-based verification for account security and password resets
  • Rate Limiting: Built-in protection against abuse and DoS attacks
  • Encrypted Storage: Sensitive data encrypted at rest with industry-standard encryption
  • Password Policies: Configurable password strength requirements and security policies

10. Modern User Experience​

  • Responsive Design: Modern, intuitive interface that works on all devices
  • Theme Support: Light and dark mode support with user preferences
  • Performance Optimized: Fast loading with lazy loading, caching, and smart pagination
  • Real-time Updates: Live notifications and status updates across the platform

How the Platform Works​

Testworthy is built as a modern, scalable SaaS platform with enterprise-grade architecture:

  • Backend: FastAPI-powered REST API with 150+ endpoints handling all business logic, multi-tenant data isolation, AI processing, and external integrations. Features comprehensive security, rate limiting, and audit logging.
  • Frontend: React-based single-page application with TypeScript, providing a responsive, modern interface optimized for performance with lazy loading, virtual scrolling, and intelligent caching.
  • AI Integration: Deep integration with advanced AI for automated test generation, code analysis, test healing, and automation script creation, with transparent credit usage tracking.
  • Multi-Tenancy: Subdomain-based organization isolation ensuring complete data separation between customers with independent billing and user management.
  • Integrations: Native webhook support for GitHub, GitLab, and Bitbucket, Stripe billing integration, email notification system, and cloud storage for file management.
  • Security: JWT-based authentication, OTP verification, role-based access control, encrypted sensitive data storage, and comprehensive audit trails.

Getting Started​

For New Organizations​

  1. Organization Signup: Create your organization with a custom subdomain (yourcompany.testworthy.us)
  2. Choose Your Plan: Select from Trial, Starter, Standard, or Professional subscription tiers
  3. Setup Billing: Configure payment methods and manage your AI credit quotas
  4. Invite Your Team: Add team members with appropriate roles and permissions

Project Setup & Test Management​

  1. Create Projects: Start projects manually or use AI-powered import from your codebase
  2. Organize Tests: Structure your testing with hierarchical suites, sections, and custom fields
  3. Setup Integrations: Connect GitHub/GitLab/Bitbucket repositories for automated PR analysis
  4. Configure AI Settings: Setup AI integration and manage credit usage

Daily Workflow​

  1. Execute Test Runs: Create and execute test plans with real-time result tracking
  2. Leverage AI Automation: Generate tests, heal existing cases, and export automation scripts
  3. Track Defects: Link and manage defects with external issue trackers like Jira
  4. Monitor & Report: Use dashboards, scheduled reports, and analytics to track progress

Advanced Enterprise Capabilities​

AI & Automation​

  • Advanced AI: Advanced natural language processing for intelligent test generation and analysis
  • Code Analysis: Upload entire codebases for automated project structure generation
  • Test Case Healing: AI-powered improvement of existing test cases with interactive editing
  • Automation Export: Generate ready-to-run Playwright, Selenium, and Cypress scripts
  • Credit Management: Transparent usage tracking with flexible credit quota and billing options

Repository Integration​

  • Webhook Support: Real-time integration with GitHub, GitLab, and Bitbucket repositories
  • PR Analysis: Automatic analysis of pull requests with AI-powered test recommendations
  • Impact Detection: Intelligent suggestions for test updates based on code changes
  • Automated Test Creation: Generate new test cases automatically from repository changes

Enterprise Features​

  • Multi-Tenant Architecture: Complete data isolation with custom subdomains
  • Advanced Security: JWT authentication, OTP verification, encrypted storage, audit trails
  • Subscription Management: Flexible billing with Stripe integration and self-service portal
  • Role-Based Access: Granular permissions with 6 organization roles and project-level access control
  • Compliance Ready: Complete audit trails and activity logging for enterprise compliance

Reporting & Analytics​

  • Scheduled Reports: Automated PDF report generation and delivery
  • Real-time Dashboards: Live project health and performance metrics
  • Usage Analytics: Detailed AI credit usage and cost tracking
  • Custom Access Control: Fine-grained report sharing and access management

Typical User Tasks​

  • Creating and Editing Test Cases: Use the intuitive forms and modals to add or update test cases, assign them to suites and milestones, and track their history.
  • Running and Managing Test Runs: Launch test runs, mark results, and analyze execution statistics.
  • Using AI to Accelerate Testing: Import a codebase and let the AI generate a full suite of tests, or use the healing feature to keep tests up to date with code changes.
  • Managing Users and Permissions: Admins can add, edit, or remove users, assign roles, and control access to projects and features.
  • Connecting Integrations: Set up connections to GitHub, GitLab, Bitbucket, Jira, and other tools to automate workflows and improve traceability.
  • Viewing Reports and Dashboards: Access high-level and detailed analytics to monitor project health and testing effectiveness.

Platform Structure​

  • Modular Design: Each major feature (projects, test cases, runs, AI, integrations, users) is organized as a separate module, making navigation and management straightforward.
  • Extensible UI: The interface is built with reusable components, supporting advanced features like drag-and-drop, inline editing, and real-time updates.
  • Customizable Workflows: Adapt the platform to your team’s needs with custom fields, prompt templates, and flexible integrations.

Security and Reliability​

  • Authentication: Secure login with JWT-based tokens and refresh logic to keep sessions safe.
  • Role-Based Access: Only authorized users can access sensitive data or perform administrative actions.
  • Backups and Recovery: Automated backup scripts and systemd services ensure your data is safe and recoverable.
  • Performance Optimizations: The platform is designed for speed, with efficient data loading, caching, and background processing.

Support and Documentation​

  • Guides and Tutorials: Step-by-step guides are available for setup, deployment, and advanced features.
  • In-App Help: Tooltips, contextual help, and documentation links are integrated throughout the application.
  • Community and Support: Reach out to the support team or community forums for help with advanced use cases or troubleshooting.

Testworthy-Pro brings together the best of modern test management, AI automation, and seamless integrations to help your team deliver high-quality software, faster and with greater confidence. Whether you’re managing a small QA team or orchestrating large-scale enterprise testing, Testworthy-Pro adapts to your workflow and grows with your needs.