Skip to main content

Core Concepts

Understanding Testworthy's enterprise platform architecture and core components is essential for effective test management at scale. These concepts form the foundation of how tests are organized, executed, and tracked within our multi-tenant SaaS platform.

Enterprise Architecture​

Testworthy is built as a multi-tenant SaaS platform designed for enterprise organizations. The hierarchical structure provides complete isolation between organizations while enabling sophisticated testing workflows, team collaboration, and advanced automation capabilities.

Organizations​

Organizations are the top-level entity in the platform, representing your company or enterprise workspace. Each organization provides:

Multi-Tenant Isolation:

  • Custom Subdomain: Dedicated workspace (yourcompany.testworthy.us)
  • Independent Billing: Stripe-powered subscription management with self-service portal
  • Data Separation: Complete security isolation between organizations

User Management:

  • Role-Based Access Control: 6 roles (Platform Admin, Tenant Owner, Project Manager, QA, Developer, Guest)
  • User Creation: Direct user creation by administrators with role assignment and credit quota allocation
  • Permission Matrix: Role-based control over organization and project-level access
  • Activity Auditing: Comprehensive activity logging for compliance

Enterprise Features:

  • Subscription Tiers: Trial, Starter, Standard, Professional plans — all features included, tiers differ by team size and AI credits
  • Usage Analytics: AI credit consumption tracking and cost management
  • Grace Period Management: Subscription controls with cancellation undo
  • Activity Logging: Activity logs and audit trails for compliance

Projects​

Projects represent individual software products, applications, or components under test. Within an organization, projects provide the primary organizational structure for test management.

AI-Powered Creation:

  • Codebase Import: Upload ZIP files and generate comprehensive test structures using advanced AI
  • PRD Import: Upload Product Requirements Documents (PDF, Word, TXT, Markdown) for AI-powered test generation
  • Cost Estimation: Preview AI credit usage before generation
  • Intelligent Analysis: Code structure analysis and automatic test case generation
  • Framework Detection: Support for multiple programming languages and testing frameworks

Repository Integration:

  • GitHub/GitLab/Bitbucket Webhooks: Automated PR analysis and test recommendations
  • Code Change Monitoring: Real-time notifications of repository updates
  • Security: Secure webhook configuration with validation
  • Automated Testing: AI-powered test case suggestions based on code changes

Advanced Configuration:

  • Custom Fields: 6 field types (text, number, date, select, multi-select, boolean)
  • Workflow Integration: Connect with existing development and QA workflows
  • Team Management: Project-specific member roles and permissions
  • Analytics Dashboard: Health metrics, execution trends, and productivity insights

Hierarchical Test Organization​

Test Suites​

Collections of related test cases providing the primary organizational structure for large-scale test management.

Features:

  • Nested Organization: Multi-level hierarchy for complex product structures
  • Bulk Operations: Bulk add test cases to plans and bulk create test runs

Sections​

Sophisticated hierarchical organization within suites, supporting parent-child relationships for complex test structures.

Organization:

  • Multi-Level Nesting: Depth tracking for detailed categorization
  • Display Ordering: Manually control the sort order of sections within their parent container

Custom Fields (Data Model)​

Extensible metadata system designed for enterprise workflow integration. The data model supports the following field types:

Field Types:

  • Text: Free-form text input for descriptions and notes
  • Number: Numerical values for priorities, estimates, or metrics
  • Date: Timeline tracking, milestones, and deadlines
  • Select: Dropdown options for standardized categorization
  • Multi-Select: Multiple option selection for complex tagging
  • Boolean: Yes/no flags for binary properties

Configuration:

  • Per-Project Configuration: Custom fields are defined at the project level
  • Required Fields: Mark fields as required to ensure metadata is always captured

Test Cases​

Individual testing scenarios enhanced with enterprise-grade capabilities and AI integration.

AI Enhancement Features:

  • AI Integration: Advanced test case improvement and healing
  • Code Analysis: Intelligent suggestions based on application code

Version Control:

  • Complete History: Track all changes with detailed audit trails
  • Rollback Capability: Restore previous versions with one click

Automation Integration:

  • Script Generation: AI-powered generation for Playwright, Selenium, and Cypress frameworks with configurable AI provider.
  • Multi-Language Support: Playwright (TypeScript, JavaScript, Python, Java, C#), Selenium (Python, JavaScript, Java, C#, Ruby, Kotlin), Cypress (JavaScript, TypeScript)
  • Smart Selectors: Prioritized selector strategy — role-based → test-id → label → CSS as last resort
  • Script Management: One script per framework per test case, with version history, edit mode, and copy-to-clipboard
  • Runner Execution: Execute generated scripts directly via the Testworthy Runner with automatic framework detection and result capture

Defect Management:

  • Jira Integration: Create Jira issues directly from failed tests, fetch live issue status, and link test cases to Jira keys (e.g., "PROJ-123"). Supports project-level default configuration with optional per-user credential overrides
  • GitHub Issues & Bugzilla: Link defects by external issue ID with supported defect types
  • Automatic Creation: Generate defects directly from failed test executions
  • Live Status Fetch: Retrieve current issue status from Jira in real-time when viewing linked defects
  • Traceability Matrix: Complete mapping between tests and defects via the TestCaseDefect records

Test Plans & Execution​

Test Plans​

Organized collections of test cases designed for specific testing objectives with advanced planning capabilities.

Planning Features:

  • Milestone Association: Link plans to release schedules and deadlines
  • Test Case Selection: Add individual test cases or entire suites to plans
  • Reordering: Arrange test cases in the desired execution order

Test Runs​

Execution instances with real-time collaboration and comprehensive tracking capabilities.

Execution:

  • Progress Tracking: Real-time dashboard showing test run status and pass/fail rates
  • Concurrent Execution: Multiple testers can work on the same run

Result Management:

  • Comprehensive Status: Passed, Failed, Blocked, Skipped, Partial
  • Evidence Collection: Comments and attachments for each result
  • Defect Logging: Immediately log defects from failed test executions

Enterprise Billing & Subscription Management​

Subscription Tiers​

Flexible subscription plans designed for organizations of all sizes:

Trial (Free):

  • Up to 3 team members
  • 350 AI credits/month
  • All features included

Starter ($49/month):

  • Up to 10 team members
  • 3,500 AI credits/month
  • All features included

Standard ($99/month):

  • Up to 20 team members
  • 9,000 AI credits/month
  • All features included

Professional ($199/month):

  • Unlimited team members
  • 12,000 AI credits/month
  • All features included

All plans include the full feature set: AI test generation, GitHub/GitLab/Bitbucket integration, Jira integration, automation script export, and email support.

All paid plans support monthly or yearly billing cycles with automatic proration on plan changes. Yearly billing is available at a 20% discount.

Credit Economy​

Transparent AI usage management with a dual-level quota architecture:

Dual-Level Quotas:

  • Organization Level: Main credit pool based on subscription tier (350–12,000 credits/month)
  • User Level: Individual allocations distributed from the organization pool by tenant owners
  • Purchased Credits: One-time top-up credits that expire after 365 days (FIFO deduction, oldest first)

Usage Tracking:

  • Per-Operation Breakdown: Detailed tracking by AI operation type (project import, test case healing, chunk processing, automation script generation, GitHub integration analysis)
  • AI Provider & Model Tracking: Records which AI provider and model were used per operation
  • Cost Estimation: Each operation records estimated USD cost based on input/output token counts
  • User Attribution: Track credit usage by team member with individual quota enforcement
  • Monthly Reset: Monthly credit quotas reset on the 1st of each month; purchased credits persist across resets but expire after 365 days

Cost Management:

  • Quota Distribution: Tenant owners allocate credits from the organization pool to individual users via the Manage Allocations modal
  • Purchased Credits: Buy additional credit packages via Stripe (expire after 365 days)
  • Deduction Priority: Monthly quota is consumed first; purchased credits are used only after the monthly allocation is exhausted

Advanced Integration Capabilities​

Repository Integration​

Sophisticated code repository monitoring and automation:

Supported Platforms:

  • GitHub: Full webhook support and PR analysis
  • GitLab: Webhook support for merge request events
  • Bitbucket: Full webhook support and PR analysis
  • Security: Encrypted webhook validation and secure token management

Automated Workflows:

  • PR Analysis: AI-powered test recommendations for code changes
  • Test Recommendations: Intelligent suggestions with confidence scoring
  • Auto-Creation: Optional automatic creation of recommended test cases

External System Integration​

Defect Management:

  • Jira Integration: Create issues from failed tests, fetch live issue status, and link test cases to Jira keys
  • GitHub Issues: Link defects by external issue ID
  • Bugzilla: Link defects by external issue ID

Reporting & Analytics​

Scheduled Reports​

  • Report Types: Milestone, Test Plan, and Test Run reports
  • Schedule Frequency: Daily, Weekly, or Monthly
  • PDF Generation: Reports are generated in PDF format

Analytics​

  • Dashboard: Overview of test execution metrics, pass/fail rates, and recent activity
  • Project Analytics: Per-project charts and health metrics
  • AI Usage Analytics: Credit consumption tracking by operation type, provider, model, and estimated cost

Security​

Data Protection:

  • Encryption: Sensitive data (API tokens, credentials) encrypted at rest
  • Multi-Tenant Isolation: Complete data separation between organizations
  • Access Controls: Role-based permissions across organizations and projects
  • Audit Trails: Activity logging for user actions

Authentication:

  • JWT Authentication: Secure session management with refresh tokens
  • OTP Verification: Email-based two-factor authentication
  • Password Policy: Strong password requirements enforced

Getting Started​

  1. Organization Setup: Create your organization and configure billing subscription
  2. Team Onboarding: Create team members with appropriate role assignments
  3. AI Configuration: Set up AI provider and model preferences
  4. Repository Integration: Connect GitHub/GitLab for automated PR analysis
  5. Project Creation: Use AI-powered codebase import or manual project setup
  6. Test Management: Create test plans, execute test runs, and manage defects
  7. Analytics: Monitor dashboard metrics and AI usage

Workflow Best Practices​

Organization Level:

  • Configure role-based access control for security
  • Set up billing management with appropriate subscription tier
  • Establish AI usage policies and quota distribution

Project Level:

  • Use AI-powered codebase analysis for comprehensive test generation
  • Set up repository webhooks for automated test recommendations

Execution Level:

  • Create test plans to organize execution efforts
  • Use automation script generation for regression testing
  • Integrate defect management with Jira or other tracking systems
  • Monitor dashboard analytics for testing health

Tip: Leverage Testworthy's AI capabilities throughout your testing lifecycle — from initial test generation through ongoing maintenance and enhancement.

By understanding these core concepts, your organization can fully leverage Testworthy's capabilities for scalable, collaborative, and intelligent test management.