Update markdown
	
		
			
	
		
	
	
		
	
		
			All checks were successful
		
		
	
	
		
			
				
	
				Build Worker Base and Application Images / check-base-changes (push) Successful in 8s
				
			
		
			
				
	
				Build Worker Base and Application Images / build-base (push) Has been skipped
				
			
		
			
				
	
				Build Worker Base and Application Images / build-docker (push) Successful in 2m15s
				
			
		
			
				
	
				Build Worker Base and Application Images / deploy-stack (push) Successful in 8s
				
			
		
		
	
	
				
					
				
			
		
			All checks were successful
		
		
	
	Build Worker Base and Application Images / check-base-changes (push) Successful in 8s
				
			Build Worker Base and Application Images / build-base (push) Has been skipped
				
			Build Worker Base and Application Images / build-docker (push) Successful in 2m15s
				
			Build Worker Base and Application Images / deploy-stack (push) Successful in 8s
				
			This commit is contained in:
		
							parent
							
								
									1c21f417ce
								
							
						
					
					
						commit
						cfc7503a14
					
				
					 3 changed files with 327 additions and 61 deletions
				
			
		
							
								
								
									
										139
									
								
								CLAUDE.md
									
										
									
									
									
								
							
							
						
						
									
										139
									
								
								CLAUDE.md
									
										
									
									
									
								
							| 
						 | 
				
			
			@ -1,13 +1,23 @@
 | 
			
		|||
# Python Detector Worker - CLAUDE.md
 | 
			
		||||
 | 
			
		||||
## Project Overview
 | 
			
		||||
This is a FastAPI-based computer vision detection worker that processes video streams from RTSP/HTTP sources and runs YOLO-based machine learning pipelines for object detection and classification. The system is designed to work within a larger CMS (Content Management System) architecture.
 | 
			
		||||
This is a FastAPI-based computer vision detection worker that processes video streams from RTSP/HTTP sources and runs advanced YOLO-based machine learning pipelines for multi-class object detection and parallel classification. The system features comprehensive database integration, Redis support, and hierarchical pipeline execution designed to work within a larger CMS (Content Management System) architecture.
 | 
			
		||||
 | 
			
		||||
### Key Features
 | 
			
		||||
- **Multi-Class Detection**: Simultaneous detection of multiple object classes (e.g., Car + Frontal)
 | 
			
		||||
- **Parallel Processing**: Concurrent execution of classification branches using ThreadPoolExecutor
 | 
			
		||||
- **Database Integration**: Automatic PostgreSQL schema management and record updates
 | 
			
		||||
- **Redis Actions**: Image storage with region cropping and pub/sub messaging
 | 
			
		||||
- **Pipeline Synchronization**: Branch coordination with `waitForBranches` functionality
 | 
			
		||||
- **Dynamic Field Mapping**: Template-based field resolution for database operations
 | 
			
		||||
 | 
			
		||||
## Architecture & Technology Stack
 | 
			
		||||
- **Framework**: FastAPI with WebSocket support
 | 
			
		||||
- **ML/CV**: PyTorch, Ultralytics YOLO, OpenCV
 | 
			
		||||
- **Containerization**: Docker (Python 3.13-bookworm base)
 | 
			
		||||
- **Data Storage**: Redis integration for action handling
 | 
			
		||||
- **Data Storage**: Redis integration for action handling + PostgreSQL for persistent storage
 | 
			
		||||
- **Database**: Automatic schema management with gas_station_1 database
 | 
			
		||||
- **Parallel Processing**: ThreadPoolExecutor for concurrent classification
 | 
			
		||||
- **Communication**: WebSocket-based real-time protocol
 | 
			
		||||
 | 
			
		||||
## Core Components
 | 
			
		||||
| 
						 | 
				
			
			@ -24,9 +34,20 @@ This is a FastAPI-based computer vision detection worker that processes video st
 | 
			
		|||
### Pipeline System (`siwatsystem/pympta.py`)
 | 
			
		||||
- **MPTA file handling** - ZIP archives containing model configurations
 | 
			
		||||
- **Hierarchical pipeline execution** with detection → classification branching
 | 
			
		||||
- **Redis action system** for image saving and message publishing
 | 
			
		||||
- **Multi-class detection** - Simultaneous detection of multiple classes (Car + Frontal)
 | 
			
		||||
- **Parallel processing** - Concurrent classification branches with ThreadPoolExecutor
 | 
			
		||||
- **Redis action system** - Image saving with region cropping and message publishing
 | 
			
		||||
- **PostgreSQL integration** - Automatic table creation and combined updates
 | 
			
		||||
- **Dynamic model loading** with GPU optimization
 | 
			
		||||
- **Configurable trigger classes and confidence thresholds**
 | 
			
		||||
- **Branch synchronization** - waitForBranches coordination for database updates
 | 
			
		||||
 | 
			
		||||
### Database System (`siwatsystem/database.py`)
 | 
			
		||||
- **DatabaseManager class** for PostgreSQL operations
 | 
			
		||||
- **Automatic table creation** with gas_station_1.car_frontal_info schema
 | 
			
		||||
- **Combined update operations** with field mapping from branch results
 | 
			
		||||
- **Session management** with UUID generation
 | 
			
		||||
- **Error handling** and connection management
 | 
			
		||||
 | 
			
		||||
### Testing & Debugging
 | 
			
		||||
- **Protocol test script** (`test_protocol.py`) for WebSocket communication validation
 | 
			
		||||
| 
						 | 
				
			
			@ -92,33 +113,61 @@ This is a FastAPI-based computer vision detection worker that processes video st
 | 
			
		|||
 | 
			
		||||
## Model Pipeline (MPTA) Format
 | 
			
		||||
 | 
			
		||||
### Structure
 | 
			
		||||
### Enhanced Structure
 | 
			
		||||
- **ZIP archive** containing models and configuration
 | 
			
		||||
- **pipeline.json** - Main configuration file
 | 
			
		||||
- **pipeline.json** - Main configuration file with Redis + PostgreSQL settings
 | 
			
		||||
- **Model files** - YOLO .pt files for detection/classification
 | 
			
		||||
- **Redis configuration** - Optional for action execution
 | 
			
		||||
- **Multi-model support** - Detection + multiple classification models
 | 
			
		||||
 | 
			
		||||
### Pipeline Flow
 | 
			
		||||
1. **Detection stage** - YOLO object detection with bounding boxes
 | 
			
		||||
2. **Trigger evaluation** - Check if detected class matches trigger conditions
 | 
			
		||||
3. **Classification stage** - Crop detected region and run classification model
 | 
			
		||||
4. **Action execution** - Redis operations (image saving, message publishing)
 | 
			
		||||
### Advanced Pipeline Flow
 | 
			
		||||
1. **Multi-class detection stage** - YOLO detection of Car + Frontal simultaneously
 | 
			
		||||
2. **Validation stage** - Check for expected classes (flexible matching)
 | 
			
		||||
3. **Database initialization** - Create initial record with session_id
 | 
			
		||||
4. **Redis actions** - Save cropped frontal images with expiration
 | 
			
		||||
5. **Parallel classification** - Concurrent brand and body type classification
 | 
			
		||||
6. **Branch synchronization** - Wait for all classification branches to complete
 | 
			
		||||
7. **Database update** - Combined update with all classification results
 | 
			
		||||
 | 
			
		||||
### Branch Configuration
 | 
			
		||||
### Enhanced Branch Configuration
 | 
			
		||||
```json
 | 
			
		||||
{
 | 
			
		||||
  "modelId": "detector-v1",
 | 
			
		||||
  "modelFile": "detector.pt",
 | 
			
		||||
  "triggerClasses": ["car", "truck"],
 | 
			
		||||
  "minConfidence": 0.5,
 | 
			
		||||
  "branches": [{
 | 
			
		||||
    "modelId": "classifier-v1", 
 | 
			
		||||
    "modelFile": "classifier.pt",
 | 
			
		||||
    "crop": true,
 | 
			
		||||
    "triggerClasses": ["car"],
 | 
			
		||||
    "minConfidence": 0.3,
 | 
			
		||||
    "actions": [...]
 | 
			
		||||
  }]
 | 
			
		||||
  "modelId": "car_frontal_detection_v1",
 | 
			
		||||
  "modelFile": "car_frontal_detection_v1.pt",
 | 
			
		||||
  "multiClass": true,
 | 
			
		||||
  "expectedClasses": ["Car", "Frontal"],
 | 
			
		||||
  "triggerClasses": ["Car", "Frontal"],
 | 
			
		||||
  "minConfidence": 0.8,
 | 
			
		||||
  "actions": [
 | 
			
		||||
    {
 | 
			
		||||
      "type": "redis_save_image",
 | 
			
		||||
      "region": "Frontal",
 | 
			
		||||
      "key": "inference:{display_id}:{timestamp}:{session_id}:{filename}",
 | 
			
		||||
      "expire_seconds": 600
 | 
			
		||||
    }
 | 
			
		||||
  ],
 | 
			
		||||
  "branches": [
 | 
			
		||||
    {
 | 
			
		||||
      "modelId": "car_brand_cls_v1",
 | 
			
		||||
      "modelFile": "car_brand_cls_v1.pt",
 | 
			
		||||
      "parallel": true,
 | 
			
		||||
      "crop": true,
 | 
			
		||||
      "cropClass": "Frontal",
 | 
			
		||||
      "triggerClasses": ["Frontal"],
 | 
			
		||||
      "minConfidence": 0.85
 | 
			
		||||
    }
 | 
			
		||||
  ],
 | 
			
		||||
  "parallelActions": [
 | 
			
		||||
    {
 | 
			
		||||
      "type": "postgresql_update_combined",
 | 
			
		||||
      "table": "car_frontal_info",
 | 
			
		||||
      "key_field": "session_id",
 | 
			
		||||
      "waitForBranches": ["car_brand_cls_v1", "car_bodytype_cls_v1"],
 | 
			
		||||
      "fields": {
 | 
			
		||||
        "car_brand": "{car_brand_cls_v1.brand}",
 | 
			
		||||
        "car_body_type": "{car_bodytype_cls_v1.body_type}"
 | 
			
		||||
      }
 | 
			
		||||
    }
 | 
			
		||||
  ]
 | 
			
		||||
}
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
| 
						 | 
				
			
			@ -173,6 +222,9 @@ docker run -p 8000:8000 -v ./models:/app/models detector-worker
 | 
			
		|||
- **opencv-python**: Computer vision operations
 | 
			
		||||
- **websockets**: WebSocket client/server
 | 
			
		||||
- **redis**: Redis client for action execution
 | 
			
		||||
- **psycopg2-binary**: PostgreSQL database adapter
 | 
			
		||||
- **scipy**: Scientific computing for advanced algorithms
 | 
			
		||||
- **filterpy**: Kalman filtering and state estimation
 | 
			
		||||
 | 
			
		||||
## Security Considerations
 | 
			
		||||
- Model files are loaded from trusted sources only
 | 
			
		||||
| 
						 | 
				
			
			@ -180,9 +232,46 @@ docker run -p 8000:8000 -v ./models:/app/models detector-worker
 | 
			
		|||
- WebSocket connections handle disconnects gracefully
 | 
			
		||||
- Resource usage is monitored to prevent DoS
 | 
			
		||||
 | 
			
		||||
## Database Integration
 | 
			
		||||
 | 
			
		||||
### Schema Management
 | 
			
		||||
The system automatically creates and manages PostgreSQL tables:
 | 
			
		||||
 | 
			
		||||
```sql
 | 
			
		||||
CREATE TABLE IF NOT EXISTS gas_station_1.car_frontal_info (
 | 
			
		||||
    display_id VARCHAR(255),
 | 
			
		||||
    captured_timestamp VARCHAR(255),
 | 
			
		||||
    session_id VARCHAR(255) PRIMARY KEY,
 | 
			
		||||
    license_character VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    license_type VARCHAR(255) DEFAULT 'No model available',
 | 
			
		||||
    car_brand VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    car_model VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    car_body_type VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    created_at TIMESTAMP DEFAULT NOW(),
 | 
			
		||||
    updated_at TIMESTAMP DEFAULT NOW()
 | 
			
		||||
);
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
### Workflow
 | 
			
		||||
1. **Detection**: When both "Car" and "Frontal" are detected, create initial database record with UUID session_id
 | 
			
		||||
2. **Redis Storage**: Save cropped frontal image to Redis with session_id in key
 | 
			
		||||
3. **Parallel Processing**: Run brand and body type classification concurrently
 | 
			
		||||
4. **Synchronization**: Wait for all branches to complete using `waitForBranches`
 | 
			
		||||
5. **Database Update**: Update record with combined classification results using field mapping
 | 
			
		||||
 | 
			
		||||
### Field Mapping
 | 
			
		||||
Templates like `{car_brand_cls_v1.brand}` are resolved to actual classification results:
 | 
			
		||||
- `car_brand_cls_v1.brand` → "Honda"
 | 
			
		||||
- `car_bodytype_cls_v1.body_type` → "Sedan"
 | 
			
		||||
 | 
			
		||||
## Performance Optimizations
 | 
			
		||||
- GPU acceleration when CUDA is available
 | 
			
		||||
- Shared camera streams reduce resource usage
 | 
			
		||||
- Frame queue optimization (single latest frame)
 | 
			
		||||
- Model caching across subscriptions
 | 
			
		||||
- Trigger class filtering for faster inference
 | 
			
		||||
- Trigger class filtering for faster inference
 | 
			
		||||
- Parallel processing with ThreadPoolExecutor for classification branches
 | 
			
		||||
- Multi-class detection reduces inference passes
 | 
			
		||||
- Region-based cropping minimizes processing overhead
 | 
			
		||||
- Database connection pooling and prepared statements
 | 
			
		||||
- Redis image storage with automatic expiration
 | 
			
		||||
							
								
								
									
										173
									
								
								pympta.md
									
										
									
									
									
								
							
							
						
						
									
										173
									
								
								pympta.md
									
										
									
									
									
								
							| 
						 | 
				
			
			@ -32,14 +32,15 @@ This modular structure allows for creating complex and efficient inference logic
 | 
			
		|||
 | 
			
		||||
## `pipeline.json` Specification
 | 
			
		||||
 | 
			
		||||
This file defines the entire pipeline logic. The root object contains a `pipeline` key for the pipeline definition and an optional `redis` key for Redis configuration.
 | 
			
		||||
This file defines the entire pipeline logic. The root object contains a `pipeline` key for the pipeline definition, optional `redis` key for Redis configuration, and optional `postgresql` key for database integration.
 | 
			
		||||
 | 
			
		||||
### Top-Level Object Structure
 | 
			
		||||
 | 
			
		||||
| Key        | Type   | Required | Description                                             |
 | 
			
		||||
| ---------- | ------ | -------- | ------------------------------------------------------- |
 | 
			
		||||
| `pipeline` | Object | Yes      | The root node object of the pipeline.                   |
 | 
			
		||||
| `redis`    | Object | No       | Configuration for connecting to a Redis server.         |
 | 
			
		||||
| Key          | Type   | Required | Description                                             |
 | 
			
		||||
| ------------ | ------ | -------- | ------------------------------------------------------- |
 | 
			
		||||
| `pipeline`   | Object | Yes      | The root node object of the pipeline.                   |
 | 
			
		||||
| `redis`      | Object | No       | Configuration for connecting to a Redis server.         |
 | 
			
		||||
| `postgresql` | Object | No       | Configuration for connecting to a PostgreSQL database.  |
 | 
			
		||||
 | 
			
		||||
### Redis Configuration (`redis`)
 | 
			
		||||
 | 
			
		||||
| 
						 | 
				
			
			@ -50,6 +51,16 @@ This file defines the entire pipeline logic. The root object contains a `pipelin
 | 
			
		|||
| `password` | String | No       | The password for Redis authentication.                  |
 | 
			
		||||
| `db`       | Number | No       | The Redis database number to use. Defaults to `0`.      |
 | 
			
		||||
 | 
			
		||||
### PostgreSQL Configuration (`postgresql`)
 | 
			
		||||
 | 
			
		||||
| Key        | Type   | Required | Description                                             |
 | 
			
		||||
| ---------- | ------ | -------- | ------------------------------------------------------- |
 | 
			
		||||
| `host`     | String | Yes      | The hostname or IP address of the PostgreSQL server.    |
 | 
			
		||||
| `port`     | Number | Yes      | The port number of the PostgreSQL server.              |
 | 
			
		||||
| `database` | String | Yes      | The database name to connect to.                       |
 | 
			
		||||
| `username` | String | Yes      | The username for database authentication.              |
 | 
			
		||||
| `password` | String | Yes      | The password for database authentication.              |
 | 
			
		||||
 | 
			
		||||
### Node Object Structure
 | 
			
		||||
 | 
			
		||||
| Key                 | Type          | Required | Description                                                                                                                            |
 | 
			
		||||
| 
						 | 
				
			
			@ -59,12 +70,17 @@ This file defines the entire pipeline logic. The root object contains a `pipelin
 | 
			
		|||
| `minConfidence`     | Float         | Yes      | The minimum confidence score (0.0 to 1.0) required for a detection to be considered valid and potentially trigger a branch.              |
 | 
			
		||||
| `triggerClasses`    | Array<String> | Yes      | A list of class names that, when detected by the parent, can trigger this node. For the root node, this lists all classes of interest. |
 | 
			
		||||
| `crop`              | Boolean       | No       | If `true`, the image is cropped to the parent's detection bounding box before being passed to this node's model. Defaults to `false`.    |
 | 
			
		||||
| `cropClass`         | String        | No       | The specific class to use for cropping (e.g., "Frontal" for frontal view cropping).                                                   |
 | 
			
		||||
| `multiClass`        | Boolean       | No       | If `true`, enables multi-class detection mode where multiple classes can be detected simultaneously.                                   |
 | 
			
		||||
| `expectedClasses`   | Array<String> | No       | When `multiClass` is true, defines which classes are expected. At least one must be detected for processing to continue.              |
 | 
			
		||||
| `parallel`          | Boolean       | No       | If `true`, this branch will be processed in parallel with other parallel branches.                                                    |
 | 
			
		||||
| `branches`          | Array<Node>   | No       | A list of child node objects that can be triggered by this node's detections.                                                          |
 | 
			
		||||
| `actions`           | Array<Action> | No       | A list of actions to execute upon a successful detection in this node.                                                                 |
 | 
			
		||||
| `parallelActions`   | Array<Action> | No       | A list of actions to execute after all specified branches have completed.                                                             |
 | 
			
		||||
 | 
			
		||||
### Action Object Structure
 | 
			
		||||
 | 
			
		||||
Actions allow the pipeline to interact with Redis. They are executed sequentially for a given detection.
 | 
			
		||||
Actions allow the pipeline to interact with Redis and PostgreSQL databases. They are executed sequentially for a given detection.
 | 
			
		||||
 | 
			
		||||
#### Action Context & Dynamic Keys
 | 
			
		||||
 | 
			
		||||
| 
						 | 
				
			
			@ -72,7 +88,12 @@ All actions have access to a dynamic context for formatting keys and messages. T
 | 
			
		|||
 | 
			
		||||
- All key-value pairs from the detection result (e.g., `class`, `confidence`, `id`).
 | 
			
		||||
- `{timestamp_ms}`: The current Unix timestamp in milliseconds.
 | 
			
		||||
- `{timestamp}`: Formatted timestamp string (YYYY-MM-DDTHH-MM-SS).
 | 
			
		||||
- `{uuid}`: A unique identifier (UUID4) for the detection event.
 | 
			
		||||
- `{filename}`: Generated filename with UUID.
 | 
			
		||||
- `{camera_id}`: Full camera subscription identifier.
 | 
			
		||||
- `{display_id}`: Display identifier extracted from subscription.
 | 
			
		||||
- `{session_id}`: Session ID for database operations.
 | 
			
		||||
- `{image_key}`: If a `redis_save_image` action has already been executed for this event, this placeholder will be replaced with the key where the image was stored.
 | 
			
		||||
 | 
			
		||||
#### `redis_save_image`
 | 
			
		||||
| 
						 | 
				
			
			@ -83,6 +104,9 @@ Saves the current image frame (or cropped sub-image) to a Redis key.
 | 
			
		|||
| ---------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------- |
 | 
			
		||||
| `type`           | String | Yes      | Must be `"redis_save_image"`.                                                                           |
 | 
			
		||||
| `key`            | String | Yes      | The Redis key to save the image to. Can contain any of the dynamic placeholders.                        |
 | 
			
		||||
| `region`         | String | No       | Specific detected region to crop and save (e.g., "Frontal").                                           |
 | 
			
		||||
| `format`         | String | No       | Image format: "jpeg" or "png". Defaults to "jpeg".                                                     |
 | 
			
		||||
| `quality`        | Number | No       | JPEG quality (1-100). Defaults to 90.                                                                  |
 | 
			
		||||
| `expire_seconds` | Number | No       | If provided, sets an expiration time (in seconds) for the Redis key.                                    |
 | 
			
		||||
 | 
			
		||||
#### `redis_publish`
 | 
			
		||||
| 
						 | 
				
			
			@ -95,35 +119,98 @@ Publishes a message to a Redis channel.
 | 
			
		|||
| `channel` | String | Yes      | The Redis channel to publish the message to.                                                            |
 | 
			
		||||
| `message` | String | Yes      | The message to publish. Can contain any of the dynamic placeholders, including `{image_key}`.           |
 | 
			
		||||
 | 
			
		||||
### Example `pipeline.json` with Redis
 | 
			
		||||
#### `postgresql_update_combined`
 | 
			
		||||
 | 
			
		||||
This example demonstrates a pipeline that detects vehicles, saves a uniquely named image of each detection that expires in one hour, and then publishes a notification with the image key.
 | 
			
		||||
Updates PostgreSQL database with results from multiple branches after they complete.
 | 
			
		||||
 | 
			
		||||
| Key                | Type          | Required | Description                                                                                             |
 | 
			
		||||
| ------------------ | ------------- | -------- | ------------------------------------------------------------------------------------------------------- |
 | 
			
		||||
| `type`             | String        | Yes      | Must be `"postgresql_update_combined"`.                                                                 |
 | 
			
		||||
| `table`            | String        | Yes      | The database table name (will be prefixed with `gas_station_1.` schema).                              |
 | 
			
		||||
| `key_field`        | String        | Yes      | The field to use as the update key (typically "session_id").                                           |
 | 
			
		||||
| `key_value`        | String        | Yes      | Template for the key value (e.g., "{session_id}").                                                     |
 | 
			
		||||
| `waitForBranches`  | Array<String> | Yes      | List of branch model IDs to wait for completion before executing update.                               |
 | 
			
		||||
| `fields`           | Object        | Yes      | Field mapping object where keys are database columns and values are templates (e.g., "{branch.field}").|
 | 
			
		||||
 | 
			
		||||
### Complete Example `pipeline.json`
 | 
			
		||||
 | 
			
		||||
This example demonstrates a comprehensive pipeline for vehicle detection with parallel classification and database integration:
 | 
			
		||||
 | 
			
		||||
```json
 | 
			
		||||
{
 | 
			
		||||
  "redis": {
 | 
			
		||||
    "host": "redis.local",
 | 
			
		||||
    "host": "10.100.1.3",
 | 
			
		||||
    "port": 6379,
 | 
			
		||||
    "password": "your-super-secret-password"
 | 
			
		||||
    "password": "your-redis-password",
 | 
			
		||||
    "db": 0
 | 
			
		||||
  },
 | 
			
		||||
  "postgresql": {
 | 
			
		||||
    "host": "10.100.1.3",
 | 
			
		||||
    "port": 5432,
 | 
			
		||||
    "database": "inference",
 | 
			
		||||
    "username": "root",
 | 
			
		||||
    "password": "your-db-password"
 | 
			
		||||
  },
 | 
			
		||||
  "pipeline": {
 | 
			
		||||
    "modelId": "vehicle-detector",
 | 
			
		||||
    "modelFile": "vehicle_model.pt",
 | 
			
		||||
    "minConfidence": 0.6,
 | 
			
		||||
    "triggerClasses": ["car", "truck"],
 | 
			
		||||
    "modelId": "car_frontal_detection_v1",
 | 
			
		||||
    "modelFile": "car_frontal_detection_v1.pt",
 | 
			
		||||
    "crop": false,
 | 
			
		||||
    "triggerClasses": ["Car", "Frontal"],
 | 
			
		||||
    "minConfidence": 0.8,
 | 
			
		||||
    "multiClass": true,
 | 
			
		||||
    "expectedClasses": ["Car", "Frontal"],
 | 
			
		||||
    "actions": [
 | 
			
		||||
      {
 | 
			
		||||
        "type": "redis_save_image",
 | 
			
		||||
        "key": "detections:{class}:{timestamp_ms}:{uuid}",
 | 
			
		||||
        "expire_seconds": 3600
 | 
			
		||||
        "region": "Frontal",
 | 
			
		||||
        "key": "inference:{display_id}:{timestamp}:{session_id}:{filename}",
 | 
			
		||||
        "expire_seconds": 600,
 | 
			
		||||
        "format": "jpeg",
 | 
			
		||||
        "quality": 90
 | 
			
		||||
      },
 | 
			
		||||
      {
 | 
			
		||||
        "type": "redis_publish",
 | 
			
		||||
        "channel": "vehicle_events",
 | 
			
		||||
        "message": "{\"event\":\"new_detection\",\"class\":\"{class}\",\"confidence\":{confidence},\"image_key\":\"{image_key}\"}"
 | 
			
		||||
        "channel": "car_detections",
 | 
			
		||||
        "message": "{\"event\":\"frontal_detected\"}"
 | 
			
		||||
      }
 | 
			
		||||
    ],
 | 
			
		||||
    "branches": []
 | 
			
		||||
    "branches": [
 | 
			
		||||
      {
 | 
			
		||||
        "modelId": "car_brand_cls_v1",
 | 
			
		||||
        "modelFile": "car_brand_cls_v1.pt",
 | 
			
		||||
        "crop": true,
 | 
			
		||||
        "cropClass": "Frontal",
 | 
			
		||||
        "resizeTarget": [224, 224],
 | 
			
		||||
        "triggerClasses": ["Frontal"],
 | 
			
		||||
        "minConfidence": 0.85,
 | 
			
		||||
        "parallel": true,
 | 
			
		||||
        "branches": []
 | 
			
		||||
      },
 | 
			
		||||
      {
 | 
			
		||||
        "modelId": "car_bodytype_cls_v1",
 | 
			
		||||
        "modelFile": "car_bodytype_cls_v1.pt",
 | 
			
		||||
        "crop": true,
 | 
			
		||||
        "cropClass": "Car",
 | 
			
		||||
        "resizeTarget": [224, 224],
 | 
			
		||||
        "triggerClasses": ["Car"],
 | 
			
		||||
        "minConfidence": 0.85,
 | 
			
		||||
        "parallel": true,
 | 
			
		||||
        "branches": []
 | 
			
		||||
      }
 | 
			
		||||
    ],
 | 
			
		||||
    "parallelActions": [
 | 
			
		||||
      {
 | 
			
		||||
        "type": "postgresql_update_combined",
 | 
			
		||||
        "table": "car_frontal_info",
 | 
			
		||||
        "key_field": "session_id",
 | 
			
		||||
        "key_value": "{session_id}",
 | 
			
		||||
        "waitForBranches": ["car_brand_cls_v1", "car_bodytype_cls_v1"],
 | 
			
		||||
        "fields": {
 | 
			
		||||
          "car_brand": "{car_brand_cls_v1.brand}",
 | 
			
		||||
          "car_body_type": "{car_bodytype_cls_v1.body_type}"
 | 
			
		||||
        }
 | 
			
		||||
      }
 | 
			
		||||
    ]
 | 
			
		||||
  }
 | 
			
		||||
}
 | 
			
		||||
```
 | 
			
		||||
| 
						 | 
				
			
			@ -134,7 +221,7 @@ The `pympta` module exposes two main functions.
 | 
			
		|||
 | 
			
		||||
### `load_pipeline_from_zip(zip_source: str, target_dir: str) -> dict`
 | 
			
		||||
 | 
			
		||||
Loads, extracts, and parses an `.mpta` file to build a pipeline tree in memory. It also establishes a Redis connection if configured in `pipeline.json`.
 | 
			
		||||
Loads, extracts, and parses an `.mpta` file to build a pipeline tree in memory. It also establishes Redis and PostgreSQL connections if configured in `pipeline.json`.
 | 
			
		||||
 | 
			
		||||
- **Parameters:**
 | 
			
		||||
  - `zip_source` (str): The file path to the local `.mpta` zip archive.
 | 
			
		||||
| 
						 | 
				
			
			@ -142,7 +229,7 @@ Loads, extracts, and parses an `.mpta` file to build a pipeline tree in memory.
 | 
			
		|||
- **Returns:**
 | 
			
		||||
  - A dictionary representing the root node of the pipeline, ready to be used with `run_pipeline`. Returns `None` if loading fails.
 | 
			
		||||
 | 
			
		||||
### `run_pipeline(frame, node: dict, return_bbox: bool = False)`
 | 
			
		||||
### `run_pipeline(frame, node: dict, return_bbox: bool = False, context: dict = None)`
 | 
			
		||||
 | 
			
		||||
Executes the inference pipeline on a single image frame.
 | 
			
		||||
 | 
			
		||||
| 
						 | 
				
			
			@ -150,12 +237,43 @@ Executes the inference pipeline on a single image frame.
 | 
			
		|||
  - `frame`: The input image frame (e.g., a NumPy array from OpenCV).
 | 
			
		||||
  - `node` (dict): The pipeline node to execute (typically the root node returned by `load_pipeline_from_zip`).
 | 
			
		||||
  - `return_bbox` (bool): If `True`, the function returns a tuple `(detection, bounding_box)`. Otherwise, it returns only the `detection`.
 | 
			
		||||
  - `context` (dict): Optional context dictionary containing camera_id, display_id, session_id for action formatting.
 | 
			
		||||
- **Returns:**
 | 
			
		||||
  - The final detection result from the last executed node in the chain. A detection is a dictionary like `{'class': 'car', 'confidence': 0.95, 'id': 1}`. If no detection meets the criteria, it returns `None` (or `(None, None)` if `return_bbox` is `True`).
 | 
			
		||||
 | 
			
		||||
## Database Integration
 | 
			
		||||
 | 
			
		||||
The pipeline system includes automatic PostgreSQL database management:
 | 
			
		||||
 | 
			
		||||
### Table Schema (`gas_station_1.car_frontal_info`)
 | 
			
		||||
 | 
			
		||||
The system automatically creates and manages the following table structure:
 | 
			
		||||
 | 
			
		||||
```sql
 | 
			
		||||
CREATE TABLE IF NOT EXISTS gas_station_1.car_frontal_info (
 | 
			
		||||
    display_id VARCHAR(255),
 | 
			
		||||
    captured_timestamp VARCHAR(255),
 | 
			
		||||
    session_id VARCHAR(255) PRIMARY KEY,
 | 
			
		||||
    license_character VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    license_type VARCHAR(255) DEFAULT 'No model available',
 | 
			
		||||
    car_brand VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    car_model VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    car_body_type VARCHAR(255) DEFAULT NULL,
 | 
			
		||||
    created_at TIMESTAMP DEFAULT NOW(),
 | 
			
		||||
    updated_at TIMESTAMP DEFAULT NOW()
 | 
			
		||||
);
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
### Workflow
 | 
			
		||||
 | 
			
		||||
1. **Initial Record Creation**: When both "Car" and "Frontal" are detected, an initial database record is created with a UUID session_id.
 | 
			
		||||
2. **Redis Storage**: Vehicle images are stored in Redis with keys containing the session_id.
 | 
			
		||||
3. **Parallel Classification**: Brand and body type classification run concurrently.
 | 
			
		||||
4. **Database Update**: After all branches complete, the database record is updated with classification results.
 | 
			
		||||
 | 
			
		||||
## Usage Example
 | 
			
		||||
 | 
			
		||||
This snippet, inspired by `pipeline_webcam.py`, shows how to use `pympta` to load a pipeline and process an image from a webcam.
 | 
			
		||||
This snippet shows how to use `pympta` with the enhanced features:
 | 
			
		||||
 | 
			
		||||
```python
 | 
			
		||||
import cv2
 | 
			
		||||
| 
						 | 
				
			
			@ -181,9 +299,14 @@ while True:
 | 
			
		|||
    if not ret:
 | 
			
		||||
        break
 | 
			
		||||
 | 
			
		||||
    # 4. Run the pipeline on the current frame
 | 
			
		||||
    # The function will handle the entire logic tree (e.g., find a car, then find its license plate).
 | 
			
		||||
    detection_result, bounding_box = run_pipeline(frame, model_tree, return_bbox=True)
 | 
			
		||||
    # 4. Run the pipeline on the current frame with context
 | 
			
		||||
    context = {
 | 
			
		||||
        "camera_id": "display-001;cam-001",
 | 
			
		||||
        "display_id": "display-001", 
 | 
			
		||||
        "session_id": None  # Will be generated automatically
 | 
			
		||||
    }
 | 
			
		||||
    
 | 
			
		||||
    detection_result, bounding_box = run_pipeline(frame, model_tree, return_bbox=True, context=context)
 | 
			
		||||
 | 
			
		||||
    # 5. Display the results
 | 
			
		||||
    if detection_result:
 | 
			
		||||
| 
						 | 
				
			
			
 | 
			
		|||
							
								
								
									
										76
									
								
								worker.md
									
										
									
									
									
								
							
							
						
						
									
										76
									
								
								worker.md
									
										
									
									
									
								
							| 
						 | 
				
			
			@ -2,6 +2,12 @@
 | 
			
		|||
 | 
			
		||||
This document outlines the WebSocket-based communication protocol between the CMS backend and a detector worker. As a worker developer, your primary responsibility is to implement a WebSocket server that adheres to this protocol.
 | 
			
		||||
 | 
			
		||||
The current Python Detector Worker implementation supports advanced computer vision pipelines with:
 | 
			
		||||
- Multi-class YOLO detection with parallel processing
 | 
			
		||||
- PostgreSQL database integration with automatic schema management
 | 
			
		||||
- Redis integration for image storage and pub/sub messaging
 | 
			
		||||
- Hierarchical pipeline execution with detection → classification branching
 | 
			
		||||
 | 
			
		||||
## 1. Connection
 | 
			
		||||
 | 
			
		||||
The worker must run a WebSocket server, preferably on port `8000`. The backend system, which is managed by a container orchestration service, will automatically discover and establish a WebSocket connection to your worker.
 | 
			
		||||
| 
						 | 
				
			
			@ -25,14 +31,34 @@ To enable modularity and dynamic configuration, the backend will send you a URL
 | 
			
		|||
2. Extracting its contents.
 | 
			
		||||
3. Interpreting the contents to configure its internal pipeline.
 | 
			
		||||
 | 
			
		||||
**The contents of the `.mpta` file are entirely up to the user who configures the model in the CMS.** This allows for maximum flexibility. For example, the archive could contain:
 | 
			
		||||
**The current implementation supports comprehensive pipeline configurations including:**
 | 
			
		||||
 | 
			
		||||
- AI/ML Models: Pre-trained models for libraries like TensorFlow, PyTorch, or ONNX.
 | 
			
		||||
- Configuration Files: A `config.json` or `pipeline.yaml` that defines a sequence of operations, specifies model paths, or sets detection thresholds.
 | 
			
		||||
- Scripts: Custom Python scripts for pre-processing or post-processing.
 | 
			
		||||
- API Integration Details: A JSON file with endpoint information and credentials for interacting with third-party detection services.
 | 
			
		||||
- **AI/ML Models**: YOLO models (.pt files) for detection and classification
 | 
			
		||||
- **Pipeline Configuration**: `pipeline.json` defining hierarchical detection→classification workflows
 | 
			
		||||
- **Multi-class Detection**: Simultaneous detection of multiple object classes (e.g., Car + Frontal)
 | 
			
		||||
- **Parallel Processing**: Concurrent execution of classification branches with ThreadPoolExecutor
 | 
			
		||||
- **Database Integration**: PostgreSQL configuration for automatic table creation and updates
 | 
			
		||||
- **Redis Actions**: Image storage with region cropping and pub/sub messaging
 | 
			
		||||
- **Dynamic Field Mapping**: Template-based field resolution for database operations
 | 
			
		||||
 | 
			
		||||
Essentially, the `.mpta` file is a self-contained package that tells your worker *how* to process the video stream for a given subscription.
 | 
			
		||||
**Enhanced MPTA Structure:**
 | 
			
		||||
```
 | 
			
		||||
pipeline.mpta/
 | 
			
		||||
├── pipeline.json          # Main configuration with redis/postgresql settings
 | 
			
		||||
├── car_detection.pt       # Primary YOLO detection model
 | 
			
		||||
├── brand_classifier.pt    # Classification model for car brands
 | 
			
		||||
├── bodytype_classifier.pt # Classification model for body types
 | 
			
		||||
└── ...
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
The `pipeline.json` now supports advanced features like:
 | 
			
		||||
- Multi-class detection with `expectedClasses` validation
 | 
			
		||||
- Parallel branch processing with `parallel: true`
 | 
			
		||||
- Database actions with `postgresql_update_combined`
 | 
			
		||||
- Redis actions with region-specific image cropping
 | 
			
		||||
- Branch synchronization with `waitForBranches`
 | 
			
		||||
 | 
			
		||||
Essentially, the `.mpta` file is a self-contained package that tells your worker *how* to process the video stream for a given subscription, including complex multi-stage AI pipelines with database persistence.
 | 
			
		||||
 | 
			
		||||
## 4. Messages from Worker to Backend
 | 
			
		||||
 | 
			
		||||
| 
						 | 
				
			
			@ -79,6 +105,15 @@ Sent when the worker detects a relevant object. The `detection` object should be
 | 
			
		|||
 | 
			
		||||
- **Type:** `imageDetection`
 | 
			
		||||
 | 
			
		||||
**Enhanced Detection Capabilities:**
 | 
			
		||||
 | 
			
		||||
The current implementation supports multi-class detection with parallel classification processing. When a vehicle is detected, the system:
 | 
			
		||||
 | 
			
		||||
1. **Multi-Class Detection**: Simultaneously detects "Car" and "Frontal" classes
 | 
			
		||||
2. **Parallel Processing**: Runs brand and body type classification concurrently
 | 
			
		||||
3. **Database Integration**: Automatically creates and updates PostgreSQL records
 | 
			
		||||
4. **Redis Storage**: Saves cropped frontal images with expiration
 | 
			
		||||
 | 
			
		||||
**Payload Example:**
 | 
			
		||||
 | 
			
		||||
```json
 | 
			
		||||
| 
						 | 
				
			
			@ -88,19 +123,38 @@ Sent when the worker detects a relevant object. The `detection` object should be
 | 
			
		|||
  "timestamp": "2025-07-14T12:34:56.789Z",
 | 
			
		||||
  "data": {
 | 
			
		||||
    "detection": {
 | 
			
		||||
      "carModel": "Civic",
 | 
			
		||||
      "class": "Car",
 | 
			
		||||
      "confidence": 0.92,
 | 
			
		||||
      "carBrand": "Honda",
 | 
			
		||||
      "carYear": 2023,
 | 
			
		||||
      "carModel": "Civic",
 | 
			
		||||
      "bodyType": "Sedan",
 | 
			
		||||
      "licensePlateText": "ABCD1234",
 | 
			
		||||
      "licensePlateConfidence": 0.95
 | 
			
		||||
      "branch_results": {
 | 
			
		||||
        "car_brand_cls_v1": {
 | 
			
		||||
          "class": "Honda",
 | 
			
		||||
          "confidence": 0.89,
 | 
			
		||||
          "brand": "Honda"
 | 
			
		||||
        },
 | 
			
		||||
        "car_bodytype_cls_v1": {
 | 
			
		||||
          "class": "Sedan", 
 | 
			
		||||
          "confidence": 0.85,
 | 
			
		||||
          "body_type": "Sedan"
 | 
			
		||||
        }
 | 
			
		||||
      }
 | 
			
		||||
    },
 | 
			
		||||
    "modelId": 101,
 | 
			
		||||
    "modelName": "US-LPR-and-Vehicle-ID"
 | 
			
		||||
    "modelName": "Car Frontal Detection V1"
 | 
			
		||||
  }
 | 
			
		||||
}
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
**Database Integration:**
 | 
			
		||||
 | 
			
		||||
Each detection automatically:
 | 
			
		||||
- Creates a record in `gas_station_1.car_frontal_info` table
 | 
			
		||||
- Generates a unique `session_id` for tracking
 | 
			
		||||
- Updates the record with classification results after parallel processing completes
 | 
			
		||||
- Stores cropped frontal images in Redis with the session_id as key
 | 
			
		||||
 | 
			
		||||
### 4.3. Patch Session
 | 
			
		||||
 | 
			
		||||
> **Note:** Patch messages are only used when the worker can't keep up and needs to retroactively send detections. Normally, detections should be sent in real-time using `imageDetection` messages. Use `patchSession` only to update session data after the fact.
 | 
			
		||||
| 
						 | 
				
			
			
 | 
			
		|||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue