Update markdown
All checks were successful
Build Worker Base and Application Images / check-base-changes (push) Successful in 8s
Build Worker Base and Application Images / build-base (push) Has been skipped
Build Worker Base and Application Images / build-docker (push) Successful in 2m15s
Build Worker Base and Application Images / deploy-stack (push) Successful in 8s
All checks were successful
Build Worker Base and Application Images / check-base-changes (push) Successful in 8s
Build Worker Base and Application Images / build-base (push) Has been skipped
Build Worker Base and Application Images / build-docker (push) Successful in 2m15s
Build Worker Base and Application Images / deploy-stack (push) Successful in 8s
This commit is contained in:
parent
1c21f417ce
commit
cfc7503a14
3 changed files with 327 additions and 61 deletions
139
CLAUDE.md
139
CLAUDE.md
|
@ -1,13 +1,23 @@
|
|||
# Python Detector Worker - CLAUDE.md
|
||||
|
||||
## Project Overview
|
||||
This is a FastAPI-based computer vision detection worker that processes video streams from RTSP/HTTP sources and runs YOLO-based machine learning pipelines for object detection and classification. The system is designed to work within a larger CMS (Content Management System) architecture.
|
||||
This is a FastAPI-based computer vision detection worker that processes video streams from RTSP/HTTP sources and runs advanced YOLO-based machine learning pipelines for multi-class object detection and parallel classification. The system features comprehensive database integration, Redis support, and hierarchical pipeline execution designed to work within a larger CMS (Content Management System) architecture.
|
||||
|
||||
### Key Features
|
||||
- **Multi-Class Detection**: Simultaneous detection of multiple object classes (e.g., Car + Frontal)
|
||||
- **Parallel Processing**: Concurrent execution of classification branches using ThreadPoolExecutor
|
||||
- **Database Integration**: Automatic PostgreSQL schema management and record updates
|
||||
- **Redis Actions**: Image storage with region cropping and pub/sub messaging
|
||||
- **Pipeline Synchronization**: Branch coordination with `waitForBranches` functionality
|
||||
- **Dynamic Field Mapping**: Template-based field resolution for database operations
|
||||
|
||||
## Architecture & Technology Stack
|
||||
- **Framework**: FastAPI with WebSocket support
|
||||
- **ML/CV**: PyTorch, Ultralytics YOLO, OpenCV
|
||||
- **Containerization**: Docker (Python 3.13-bookworm base)
|
||||
- **Data Storage**: Redis integration for action handling
|
||||
- **Data Storage**: Redis integration for action handling + PostgreSQL for persistent storage
|
||||
- **Database**: Automatic schema management with gas_station_1 database
|
||||
- **Parallel Processing**: ThreadPoolExecutor for concurrent classification
|
||||
- **Communication**: WebSocket-based real-time protocol
|
||||
|
||||
## Core Components
|
||||
|
@ -24,9 +34,20 @@ This is a FastAPI-based computer vision detection worker that processes video st
|
|||
### Pipeline System (`siwatsystem/pympta.py`)
|
||||
- **MPTA file handling** - ZIP archives containing model configurations
|
||||
- **Hierarchical pipeline execution** with detection → classification branching
|
||||
- **Redis action system** for image saving and message publishing
|
||||
- **Multi-class detection** - Simultaneous detection of multiple classes (Car + Frontal)
|
||||
- **Parallel processing** - Concurrent classification branches with ThreadPoolExecutor
|
||||
- **Redis action system** - Image saving with region cropping and message publishing
|
||||
- **PostgreSQL integration** - Automatic table creation and combined updates
|
||||
- **Dynamic model loading** with GPU optimization
|
||||
- **Configurable trigger classes and confidence thresholds**
|
||||
- **Branch synchronization** - waitForBranches coordination for database updates
|
||||
|
||||
### Database System (`siwatsystem/database.py`)
|
||||
- **DatabaseManager class** for PostgreSQL operations
|
||||
- **Automatic table creation** with gas_station_1.car_frontal_info schema
|
||||
- **Combined update operations** with field mapping from branch results
|
||||
- **Session management** with UUID generation
|
||||
- **Error handling** and connection management
|
||||
|
||||
### Testing & Debugging
|
||||
- **Protocol test script** (`test_protocol.py`) for WebSocket communication validation
|
||||
|
@ -92,33 +113,61 @@ This is a FastAPI-based computer vision detection worker that processes video st
|
|||
|
||||
## Model Pipeline (MPTA) Format
|
||||
|
||||
### Structure
|
||||
### Enhanced Structure
|
||||
- **ZIP archive** containing models and configuration
|
||||
- **pipeline.json** - Main configuration file
|
||||
- **pipeline.json** - Main configuration file with Redis + PostgreSQL settings
|
||||
- **Model files** - YOLO .pt files for detection/classification
|
||||
- **Redis configuration** - Optional for action execution
|
||||
- **Multi-model support** - Detection + multiple classification models
|
||||
|
||||
### Pipeline Flow
|
||||
1. **Detection stage** - YOLO object detection with bounding boxes
|
||||
2. **Trigger evaluation** - Check if detected class matches trigger conditions
|
||||
3. **Classification stage** - Crop detected region and run classification model
|
||||
4. **Action execution** - Redis operations (image saving, message publishing)
|
||||
### Advanced Pipeline Flow
|
||||
1. **Multi-class detection stage** - YOLO detection of Car + Frontal simultaneously
|
||||
2. **Validation stage** - Check for expected classes (flexible matching)
|
||||
3. **Database initialization** - Create initial record with session_id
|
||||
4. **Redis actions** - Save cropped frontal images with expiration
|
||||
5. **Parallel classification** - Concurrent brand and body type classification
|
||||
6. **Branch synchronization** - Wait for all classification branches to complete
|
||||
7. **Database update** - Combined update with all classification results
|
||||
|
||||
### Branch Configuration
|
||||
### Enhanced Branch Configuration
|
||||
```json
|
||||
{
|
||||
"modelId": "detector-v1",
|
||||
"modelFile": "detector.pt",
|
||||
"triggerClasses": ["car", "truck"],
|
||||
"minConfidence": 0.5,
|
||||
"branches": [{
|
||||
"modelId": "classifier-v1",
|
||||
"modelFile": "classifier.pt",
|
||||
"crop": true,
|
||||
"triggerClasses": ["car"],
|
||||
"minConfidence": 0.3,
|
||||
"actions": [...]
|
||||
}]
|
||||
"modelId": "car_frontal_detection_v1",
|
||||
"modelFile": "car_frontal_detection_v1.pt",
|
||||
"multiClass": true,
|
||||
"expectedClasses": ["Car", "Frontal"],
|
||||
"triggerClasses": ["Car", "Frontal"],
|
||||
"minConfidence": 0.8,
|
||||
"actions": [
|
||||
{
|
||||
"type": "redis_save_image",
|
||||
"region": "Frontal",
|
||||
"key": "inference:{display_id}:{timestamp}:{session_id}:{filename}",
|
||||
"expire_seconds": 600
|
||||
}
|
||||
],
|
||||
"branches": [
|
||||
{
|
||||
"modelId": "car_brand_cls_v1",
|
||||
"modelFile": "car_brand_cls_v1.pt",
|
||||
"parallel": true,
|
||||
"crop": true,
|
||||
"cropClass": "Frontal",
|
||||
"triggerClasses": ["Frontal"],
|
||||
"minConfidence": 0.85
|
||||
}
|
||||
],
|
||||
"parallelActions": [
|
||||
{
|
||||
"type": "postgresql_update_combined",
|
||||
"table": "car_frontal_info",
|
||||
"key_field": "session_id",
|
||||
"waitForBranches": ["car_brand_cls_v1", "car_bodytype_cls_v1"],
|
||||
"fields": {
|
||||
"car_brand": "{car_brand_cls_v1.brand}",
|
||||
"car_body_type": "{car_bodytype_cls_v1.body_type}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
|
@ -173,6 +222,9 @@ docker run -p 8000:8000 -v ./models:/app/models detector-worker
|
|||
- **opencv-python**: Computer vision operations
|
||||
- **websockets**: WebSocket client/server
|
||||
- **redis**: Redis client for action execution
|
||||
- **psycopg2-binary**: PostgreSQL database adapter
|
||||
- **scipy**: Scientific computing for advanced algorithms
|
||||
- **filterpy**: Kalman filtering and state estimation
|
||||
|
||||
## Security Considerations
|
||||
- Model files are loaded from trusted sources only
|
||||
|
@ -180,9 +232,46 @@ docker run -p 8000:8000 -v ./models:/app/models detector-worker
|
|||
- WebSocket connections handle disconnects gracefully
|
||||
- Resource usage is monitored to prevent DoS
|
||||
|
||||
## Database Integration
|
||||
|
||||
### Schema Management
|
||||
The system automatically creates and manages PostgreSQL tables:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS gas_station_1.car_frontal_info (
|
||||
display_id VARCHAR(255),
|
||||
captured_timestamp VARCHAR(255),
|
||||
session_id VARCHAR(255) PRIMARY KEY,
|
||||
license_character VARCHAR(255) DEFAULT NULL,
|
||||
license_type VARCHAR(255) DEFAULT 'No model available',
|
||||
car_brand VARCHAR(255) DEFAULT NULL,
|
||||
car_model VARCHAR(255) DEFAULT NULL,
|
||||
car_body_type VARCHAR(255) DEFAULT NULL,
|
||||
created_at TIMESTAMP DEFAULT NOW(),
|
||||
updated_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
### Workflow
|
||||
1. **Detection**: When both "Car" and "Frontal" are detected, create initial database record with UUID session_id
|
||||
2. **Redis Storage**: Save cropped frontal image to Redis with session_id in key
|
||||
3. **Parallel Processing**: Run brand and body type classification concurrently
|
||||
4. **Synchronization**: Wait for all branches to complete using `waitForBranches`
|
||||
5. **Database Update**: Update record with combined classification results using field mapping
|
||||
|
||||
### Field Mapping
|
||||
Templates like `{car_brand_cls_v1.brand}` are resolved to actual classification results:
|
||||
- `car_brand_cls_v1.brand` → "Honda"
|
||||
- `car_bodytype_cls_v1.body_type` → "Sedan"
|
||||
|
||||
## Performance Optimizations
|
||||
- GPU acceleration when CUDA is available
|
||||
- Shared camera streams reduce resource usage
|
||||
- Frame queue optimization (single latest frame)
|
||||
- Model caching across subscriptions
|
||||
- Trigger class filtering for faster inference
|
||||
- Trigger class filtering for faster inference
|
||||
- Parallel processing with ThreadPoolExecutor for classification branches
|
||||
- Multi-class detection reduces inference passes
|
||||
- Region-based cropping minimizes processing overhead
|
||||
- Database connection pooling and prepared statements
|
||||
- Redis image storage with automatic expiration
|
Loading…
Add table
Add a link
Reference in a new issue