refactor: remove VMware DRS references and enhance clarity in WorkerConnection documentation
All checks were successful
Build Worker Base and Application Images / check-base-changes (push) Successful in 7s
Build Worker Base and Application Images / build-base (push) Has been skipped
Build Worker Base and Application Images / build-docker (push) Successful in 2m44s
Build Worker Base and Application Images / deploy-stack (push) Successful in 18s
All checks were successful
Build Worker Base and Application Images / check-base-changes (push) Successful in 7s
Build Worker Base and Application Images / build-base (push) Has been skipped
Build Worker Base and Application Images / build-docker (push) Successful in 2m44s
Build Worker Base and Application Images / deploy-stack (push) Successful in 18s
This commit is contained in:
parent
ac85caca39
commit
4251a9b06d
1 changed files with 9 additions and 9 deletions
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Overview
|
||||
|
||||
The Camera Module implements a pure VMware DRS-like declarative architecture for managing connections to Python ML workers. This system uses the database as the single source of truth for desired subscription state, with automatic regeneration and reconciliation providing intelligent camera management, real-time object detection, and AI-powered content selection with automatic load balancing capabilities.
|
||||
The Camera Module implements a pure declarative architecture for managing connections to Python ML workers. This system uses the database as the single source of truth for desired subscription state, with automatic regeneration and reconciliation providing intelligent camera management, real-time object detection, and AI-powered content selection with automatic load balancing capabilities.
|
||||
|
||||
**Key Architectural Principle**: Database mutations trigger complete state regeneration rather than incremental updates, ensuring consistency and eliminating complex state synchronization issues.
|
||||
|
||||
|
@ -44,7 +44,7 @@ Core distributed cluster implementation that handles declarative state managemen
|
|||
**Master Mode Responsibilities:**
|
||||
- Maintains WebSocket connections to all Python workers
|
||||
- Manages desired vs actual subscription state separation
|
||||
- Implements VMware DRS-like global rebalancing algorithm
|
||||
- Implements intelligent global rebalancing algorithm
|
||||
- Processes automatic reconciliation every 30 seconds
|
||||
- Responds to slave join/leave events from MasterElection
|
||||
- Generates fresh pre-signed model URLs for worker assignments
|
||||
|
@ -201,10 +201,10 @@ All Redis data uses **manual cleanup only** (no TTL) to ensure:
|
|||
- Predictable cleanup during planned maintenance
|
||||
- Debug visibility into system state history
|
||||
|
||||
## Pure Declarative Architecture (VMware DRS-like)
|
||||
## Pure Declarative Architecture
|
||||
|
||||
### Concept Overview
|
||||
The system implements a pure declarative approach similar to VMware Distributed Resource Scheduler (DRS), where:
|
||||
The system implements a pure declarative approach where:
|
||||
- **Database**: Single source of truth for desired state (Display+Camera+Playlist combinations)
|
||||
- **Actual State**: What subscriptions are currently running on workers (stored in `worker:actual_subscriptions`)
|
||||
- **Regeneration**: Master regenerates complete desired state from database on every change notification
|
||||
|
@ -261,8 +261,8 @@ async handleDatabaseChange(changeType: string, entityId: string) {
|
|||
}
|
||||
}
|
||||
|
||||
// VMware DRS-like worker selection (unchanged)
|
||||
function findBestWorkerVMwareDRS(workers, currentLoads) {
|
||||
// Intelligent worker selection (unchanged)
|
||||
function findBestWorkerForLoad(workers, currentLoads) {
|
||||
return workers
|
||||
.map(worker => ({
|
||||
worker,
|
||||
|
@ -280,7 +280,7 @@ function findBestWorkerVMwareDRS(workers, currentLoads) {
|
|||
3. **Complete Regeneration**: Master queries database for all active Display+Camera+Playlist combinations
|
||||
4. **Desired State Creation**: Master generates fresh desired subscriptions from database query results
|
||||
5. **Diff Analysis**: Master compares fresh desired state vs current actual state on workers
|
||||
6. **Global Reconciliation**: Master applies VMware DRS algorithm to reconcile differences
|
||||
6. **Global Reconciliation**: Master applies intelligent load balancing algorithm to reconcile differences
|
||||
7. **Worker Commands**: Master sends subscription/unsubscription commands to workers
|
||||
8. **State Update**: Master updates actual subscription state in Redis
|
||||
|
||||
|
@ -1483,7 +1483,7 @@ This interface specification provides external services with a clear understandi
|
|||
- **Service Layer Simplicity**: Services just update database + trigger regeneration - no subscription logic
|
||||
- **Operational Resilience**: System is self-healing and predictable - any database change triggers complete reconciliation
|
||||
|
||||
### VMware DRS-like Benefits
|
||||
### Declarative Architecture Benefits
|
||||
- **Global Optimization**: Every regeneration considers all subscriptions globally for optimal load balancing
|
||||
- **Automatic Recovery**: System automatically heals from any inconsistent state by regenerating from database
|
||||
- **Resource Efficiency**: Workers assigned based on real-time CPU/memory metrics with load balancing
|
||||
|
@ -1495,4 +1495,4 @@ This interface specification provides external services with a clear understandi
|
|||
- **Memory Efficiency**: No persistent state storage outside of database and current worker assignments
|
||||
- **Network Efficiency**: Minimal command protocol reduces Redis pub/sub overhead
|
||||
|
||||
This pure declarative architecture provides the reliability and simplicity of Kubernetes-style declarative resource management while maintaining the performance and scalability needed for real-time camera processing systems.
|
||||
This pure declarative architecture provides the reliability and simplicity of container orchestration-style declarative resource management while maintaining the performance and scalability needed for real-time camera processing systems.
|
Loading…
Add table
Add a link
Reference in a new issue