VideoAnnotator Testing Overview¶

Current Test Structure 20 AUG 2025¶

VideoAnnotator uses a 3-tier organized test system designed for efficient development and comprehensive validation. All tests are located in the tests/ directory with the following structure:

tests/
├── unit/                     # Fast, isolated tests (<30 seconds)
│   ├── batch/               # Batch processing components (5 files)
│   ├── storage/             # File backends and validation (1 file)
│   └── utils/               # Utility functions (2 files)
├── integration/             # Cross-component tests (8 files)
│   ├── test_batch_orchestration.py
│   ├── test_simple_workflows.py
│   └── ...
├── pipelines/               # Full pipeline tests (12 files)
│   ├── test_person_tracking.py
│   ├── test_face_analysis.py
│   ├── test_audio_pipeline.py
│   └── ...
└── scripts/                 # Test execution scripts
    ├── test_fast.py         # Unit tests only
    ├── test_integration.py  # Unit + Integration
    ├── test_pipelines.py    # Pipeline tests
    └── test_all.py          # Complete suite

Test Execution Workflows¶

Development Workflow (Recommended)¶

# Fast feedback during development
python scripts/test_fast.py                # ~30 seconds, 125+ tests

# Pre-commit validation
python scripts/test_integration.py         # ~5 minutes

# Full validation
python scripts/test_all.py                 # Complete suite with reporting

Pipeline-Specific Testing¶

# Test specific pipelines
pytest tests/pipelines/test_person_tracking.py -v
pytest tests/pipelines/test_face_analysis.py -v

# Enable integration tests (requires models)
TEST_INTEGRATION=1 pytest tests/pipelines/ -v

Advanced Usage¶

# Test by category
pytest -m unit                             # Unit tests only
pytest -m integration                      # Integration tests only
pytest -m pipeline                         # Pipeline tests only

# Performance testing
pytest -m performance --benchmark-only     # Benchmarking tests

Current Test Coverage¶

Pipeline Coverage Status¶

Pipeline	Tests	Status	Coverage
Person Tracking	9/9	✅ 100%	YOLO11, ByteTrack, pose estimation
Face Analysis	14/15	✅ 93.3%	DeepFace, emotions, age, gender
Audio Processing	5 files	✅ Available	Whisper, LAION, speech recognition
Scene Detection	Available	✅ Ready	PySceneDetect + CLIP
OpenFace3	Comprehensive	✅ Complete	Full facial behavior analysis
LAION Face/Voice	Available	✅ Ready	Advanced AI model testing

Component Coverage¶

Component	Tests	Status	Notes
Batch Processing	5 files	✅ 100%	Orchestrator, recovery, progress tracking
Storage Backend	1 file	✅ Complete	File storage validation
Utils/Analysis	2 files	⚠️ 90%	Size analysis needs attention

Test Execution Statistics¶

Total Tests: ~125 unit tests + integration + pipeline tests
Fast Execution: 104/125 unit tests pass in <30 seconds
Success Rate: 83.2% (stable and consistent)
Integration Tests: Real model execution (YOLO11, DeepFace, Whisper)

Configuration & Environment¶

Environment Variables¶

# Enable integration tests with real models
export TEST_INTEGRATION=1

# Enable DeepFace-specific tests
export TEST_DEEPFACE=1

Pytest Configuration¶

The test suite uses pytest with custom markers defined in pyproject.toml:

unit: Fast, isolated tests
integration: Cross-component tests
pipeline: Full pipeline tests
performance: Benchmarking tests
gpu: GPU-accelerated tests
real_models: Tests using real ML models

Current Issues & Limitations¶

Known Issues¶

Size Analysis Integration Test: One failing test (functionality may be incomplete)
Windows File Cleanup: Occasional permission errors in test teardown (cosmetic)

Pending Improvements¶

Audio pipeline test isolation refinement
Performance benchmarking test category
CI/CD integration workflows

Development Guidelines¶

Adding New Tests¶

Location: Place tests in appropriate tier directory
Unit tests → tests/unit/[component]/
Integration tests → tests/integration/
Pipeline tests → tests/pipelines/
Naming: Follow pattern test_[component]_[functionality].py

Markers: Use appropriate pytest markers

@pytest.mark.unit
@pytest.mark.integration
@pytest.mark.pipeline

Test Quality Standards¶

Unit Tests: Fast (<1s per test), isolated, mocked dependencies
Integration Tests: Real components, moderate speed (<5min total)
Pipeline Tests: Full workflows, real models, slower execution acceptable
All Tests: Clear docstrings, meaningful assertions, proper cleanup

Running Specific Test Categories¶

# By directory
pytest tests/unit/                          # All unit tests
pytest tests/integration/                   # All integration tests
pytest tests/pipelines/                     # All pipeline tests

# By marker
pytest -m unit                              # Unit tests only
pytest -m "integration and not slow"       # Fast integration tests

# By pattern
pytest -k "person_tracking"                # Person tracking tests
pytest -k "face and not slow"              # Face tests excluding slow ones

Integration with Development Workflow¶

Recommended Development Process¶

During Development: python scripts/test_fast.py for immediate feedback
Before Commit: python scripts/test_integration.py for validation
Before Release: python scripts/test_all.py for comprehensive testing
CI/CD Pipeline: Tiered execution based on change scope

Model Requirements¶

Person Tracking: YOLO11 models (auto-download)
Face Analysis: DeepFace models (auto-download)
Audio Processing: Whisper models (auto-download)
OpenFace3: Separate installation required (see requirements_openface.txt)

Maintenance & Updates¶

Regular Maintenance¶

Monitor test success rates and execution times
Update integration tests when adding new models
Refresh test data fixtures periodically
Review and update performance benchmarks

When Adding New Features¶

Add unit tests for new functionality
Add integration tests for cross-component features
Add pipeline tests for complete workflows
Update this documentation as needed

This testing infrastructure provides a solid foundation for reliable VideoAnnotator development with fast feedback loops, comprehensive coverage, and professional workflows.