diff --git a/.gitea/workflows/build.yml b/.gitea/workflows/build.yml deleted file mode 100644 index dad25b3..0000000 --- a/.gitea/workflows/build.yml +++ /dev/null @@ -1,34 +0,0 @@ -name: Build Backend Application and Docker Image - -on: - push: - branches: - - main - workflow_dispatch: - -jobs: - build-docker: - runs-on: ubuntu-latest - permissions: - packages: write - steps: - - name: Checkout code - uses: actions/checkout@v3 - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v2 - - - name: Login to GitHub Container Registry - uses: docker/login-action@v3 - with: - registry: git.siwatsystem.com - username: ${{ github.actor }} - password: ${{ secrets.RUNNER_TOKEN }} - - - name: Build and push Docker image - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile - push: true - tags: git.siwatsystem.com/adsist-cms/worker:latest \ No newline at end of file diff --git a/.gitignore b/.gitignore index ff8c99d..fab3efb 100644 --- a/.gitignore +++ b/.gitignore @@ -6,7 +6,4 @@ app.log __pycache__/ .mptacache -mptas -detector_worker.log -.gitignore -no_frame_debug.log +mptas \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 3177259..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,188 +0,0 @@ -# Python Detector Worker - CLAUDE.md - -## Project Overview -This is a FastAPI-based computer vision detection worker that processes video streams from RTSP/HTTP sources and runs YOLO-based machine learning pipelines for object detection and classification. The system is designed to work within a larger CMS (Content Management System) architecture. - -## Architecture & Technology Stack -- **Framework**: FastAPI with WebSocket support -- **ML/CV**: PyTorch, Ultralytics YOLO, OpenCV -- **Containerization**: Docker (Python 3.13-bookworm base) -- **Data Storage**: Redis integration for action handling -- **Communication**: WebSocket-based real-time protocol - -## Core Components - -### Main Application (`app.py`) -- **FastAPI WebSocket server** for real-time communication -- **Multi-camera stream management** with shared stream optimization -- **HTTP REST endpoint** for image retrieval (`/camera/{camera_id}/image`) -- **Threading-based frame readers** for RTSP streams and HTTP snapshots -- **Model loading and inference** using MPTA (Machine Learning Pipeline Archive) format -- **Session management** with display identifier mapping -- **Resource monitoring** (CPU, memory, GPU usage via psutil) - -### Pipeline System (`siwatsystem/pympta.py`) -- **MPTA file handling** - ZIP archives containing model configurations -- **Hierarchical pipeline execution** with detection → classification branching -- **Redis action system** for image saving and message publishing -- **Dynamic model loading** with GPU optimization -- **Configurable trigger classes and confidence thresholds** - -### Testing & Debugging -- **Protocol test script** (`test_protocol.py`) for WebSocket communication validation -- **Pipeline webcam utility** (`pipeline_webcam.py`) for local testing with visual output -- **RTSP streaming debug tool** (`debug/rtsp_webcam.py`) using GStreamer - -## Code Conventions & Patterns - -### Logging -- **Structured logging** using Python's logging module -- **File + console output** to `detector_worker.log` -- **Debug level separation** for detailed troubleshooting -- **Context-aware messages** with camera IDs and model information - -### Error Handling -- **Graceful failure handling** with retry mechanisms (configurable max_retries) -- **Thread-safe operations** using locks for streams and models -- **WebSocket disconnect handling** with proper cleanup -- **Model loading validation** with detailed error reporting - -### Configuration -- **JSON configuration** (`config.json`) for runtime parameters: - - `poll_interval_ms`: Frame processing interval - - `max_streams`: Concurrent stream limit - - `target_fps`: Target frame rate - - `reconnect_interval_sec`: Stream reconnection delay - - `max_retries`: Maximum retry attempts (-1 for unlimited) - -### Threading Model -- **Frame reader threads** for each camera stream (RTSP/HTTP) -- **Shared stream optimization** - multiple subscriptions can reuse the same camera stream -- **Async WebSocket handling** with concurrent task management -- **Thread-safe data structures** with proper locking mechanisms - -## WebSocket Protocol - -### Message Types -- **subscribe**: Start camera stream with model pipeline -- **unsubscribe**: Stop camera stream processing -- **requestState**: Request current worker status -- **setSessionId**: Associate display with session identifier -- **patchSession**: Update session data -- **stateReport**: Periodic heartbeat with system metrics -- **imageDetection**: Detection results with timestamp and model info - -### Subscription Format -```json -{ - "type": "subscribe", - "payload": { - "subscriptionIdentifier": "display-001;cam-001", - "rtspUrl": "rtsp://...", // OR snapshotUrl - "snapshotUrl": "http://...", - "snapshotInterval": 5000, - "modelUrl": "http://...model.mpta", - "modelId": 101, - "modelName": "Vehicle Detection", - "cropX1": 100, "cropY1": 200, - "cropX2": 300, "cropY2": 400 - } -} -``` - -## Model Pipeline (MPTA) Format - -### Structure -- **ZIP archive** containing models and configuration -- **pipeline.json** - Main configuration file -- **Model files** - YOLO .pt files for detection/classification -- **Redis configuration** - Optional for action execution - -### Pipeline Flow -1. **Detection stage** - YOLO object detection with bounding boxes -2. **Trigger evaluation** - Check if detected class matches trigger conditions -3. **Classification stage** - Crop detected region and run classification model -4. **Action execution** - Redis operations (image saving, message publishing) - -### Branch Configuration -```json -{ - "modelId": "detector-v1", - "modelFile": "detector.pt", - "triggerClasses": ["car", "truck"], - "minConfidence": 0.5, - "branches": [{ - "modelId": "classifier-v1", - "modelFile": "classifier.pt", - "crop": true, - "triggerClasses": ["car"], - "minConfidence": 0.3, - "actions": [...] - }] -} -``` - -## Stream Management - -### Shared Streams -- Multiple subscriptions can share the same camera URL -- Reference counting prevents premature stream termination -- Automatic cleanup when last subscription ends - -### Frame Processing -- **Queue-based buffering** with single frame capacity (latest frame only) -- **Configurable polling interval** based on target FPS -- **Automatic reconnection** with exponential backoff - -## Development & Testing - -### Local Development -```bash -# Install dependencies -pip install -r requirements.txt - -# Run the worker -python app.py - -# Test protocol compliance -python test_protocol.py - -# Test pipeline with webcam -python pipeline_webcam.py --mpta-file path/to/model.mpta --video 0 -``` - -### Docker Deployment -```bash -# Build container -docker build -t detector-worker . - -# Run with volume mounts for models -docker run -p 8000:8000 -v ./models:/app/models detector-worker -``` - -### Testing Commands -- **Protocol testing**: `python test_protocol.py` -- **Pipeline validation**: `python pipeline_webcam.py --mpta-file --video 0` -- **RTSP debugging**: `python debug/rtsp_webcam.py` - -## Dependencies -- **fastapi[standard]**: Web framework with WebSocket support -- **uvicorn**: ASGI server -- **torch, torchvision**: PyTorch for ML inference -- **ultralytics**: YOLO implementation -- **opencv-python**: Computer vision operations -- **websockets**: WebSocket client/server -- **redis**: Redis client for action execution - -## Security Considerations -- Model files are loaded from trusted sources only -- Redis connections use authentication when configured -- WebSocket connections handle disconnects gracefully -- Resource usage is monitored to prevent DoS - -## Performance Optimizations -- GPU acceleration when CUDA is available -- Shared camera streams reduce resource usage -- Frame queue optimization (single latest frame) -- Model caching across subscriptions -- Trigger class filtering for faster inference \ No newline at end of file diff --git a/app.py b/app.py index 60beb27..cbf6186 100644 --- a/app.py +++ b/app.py @@ -5,7 +5,6 @@ import time import queue import torch import cv2 -import numpy as np import base64 import logging import threading @@ -14,9 +13,8 @@ import asyncio import psutil import zipfile from urllib.parse import urlparse -from fastapi import FastAPI, WebSocket, HTTPException +from fastapi import FastAPI, WebSocket from fastapi.websockets import WebSocketDisconnect -from fastapi.responses import Response from websockets.exceptions import ConnectionClosedError from ultralytics import YOLO @@ -29,12 +27,6 @@ app = FastAPI() # "models" now holds a nested dict: { camera_id: { modelId: model_tree } } models: Dict[str, Dict[str, Any]] = {} streams: Dict[str, Dict[str, Any]] = {} -# Store session IDs per display -session_ids: Dict[str, int] = {} -# Track shared camera streams by camera URL -camera_streams: Dict[str, Dict[str, Any]] = {} -# Map subscriptions to their camera URL -subscription_to_camera: Dict[str, str] = {} with open("config.json", "r") as f: config = json.load(f) @@ -49,456 +41,145 @@ max_retries = config.get("max_retries", 3) # Configure logging logging.basicConfig( - level=logging.INFO, # Set to INFO level for less verbose output - format="%(asctime)s [%(levelname)s] %(name)s: %(message)s", + level=logging.DEBUG, + format="%(asctime)s [%(levelname)s] %(message)s", handlers=[ - logging.FileHandler("detector_worker.log"), # Write logs to a file - logging.StreamHandler() # Also output to console + logging.FileHandler("app.log"), + logging.StreamHandler() ] ) -# Create a logger specifically for this application -logger = logging.getLogger("detector_worker") -logger.setLevel(logging.DEBUG) # Set app-specific logger to DEBUG level - -# Ensure all other libraries (including root) use at least INFO level -logging.getLogger().setLevel(logging.INFO) - -logger.info("Starting detector worker application") -logger.info(f"Configuration: Target FPS: {TARGET_FPS}, Max streams: {max_streams}, Max retries: {max_retries}") - # Ensure the models directory exists os.makedirs("models", exist_ok=True) -logger.info("Ensured models directory exists") # Constants for heartbeat and timeouts HEARTBEAT_INTERVAL = 2 # seconds WORKER_TIMEOUT_MS = 10000 -logger.debug(f"Heartbeat interval set to {HEARTBEAT_INTERVAL} seconds") # Locks for thread-safe operations streams_lock = threading.Lock() models_lock = threading.Lock() -logger.debug("Initialized thread locks") # Add helper to download mpta ZIP file from a remote URL def download_mpta(url: str, dest_path: str) -> str: try: - logger.info(f"Starting download of model from {url} to {dest_path}") os.makedirs(os.path.dirname(dest_path), exist_ok=True) response = requests.get(url, stream=True) if response.status_code == 200: - file_size = int(response.headers.get('content-length', 0)) - logger.info(f"Model file size: {file_size/1024/1024:.2f} MB") - downloaded = 0 with open(dest_path, "wb") as f: for chunk in response.iter_content(chunk_size=8192): f.write(chunk) - downloaded += len(chunk) - if file_size > 0 and downloaded % (file_size // 10) < 8192: # Log approximately every 10% - logger.debug(f"Download progress: {downloaded/file_size*100:.1f}%") - logger.info(f"Successfully downloaded mpta file from {url} to {dest_path}") + logging.info(f"Downloaded mpta file from {url} to {dest_path}") return dest_path else: - logger.error(f"Failed to download mpta file (status code {response.status_code}): {response.text}") + logging.error(f"Failed to download mpta file (status code {response.status_code})") return None except Exception as e: - logger.error(f"Exception downloading mpta file from {url}: {str(e)}", exc_info=True) + logging.error(f"Exception downloading mpta file from {url}: {e}") return None -# Add helper to fetch snapshot image from HTTP/HTTPS URL -def fetch_snapshot(url: str): - try: - response = requests.get(url, timeout=10) - if response.status_code == 200: - # Convert response content to numpy array - nparr = np.frombuffer(response.content, np.uint8) - # Decode image - frame = cv2.imdecode(nparr, cv2.IMREAD_COLOR) - if frame is not None: - logger.debug(f"Successfully fetched snapshot from {url}, shape: {frame.shape}") - return frame - else: - logger.error(f"Failed to decode image from snapshot URL: {url}") - return None - else: - logger.error(f"Failed to fetch snapshot (status code {response.status_code}): {url}") - return None - except Exception as e: - logger.error(f"Exception fetching snapshot from {url}: {str(e)}") - return None - -# Helper to get crop coordinates from stream -def get_crop_coords(stream): - return { - "cropX1": stream.get("cropX1"), - "cropY1": stream.get("cropY1"), - "cropX2": stream.get("cropX2"), - "cropY2": stream.get("cropY2") - } - -#################################################### -# REST API endpoint for image retrieval -#################################################### -@app.get("/camera/{camera_id}/image") -async def get_camera_image(camera_id: str): - """ - Get the current frame from a camera as JPEG image - """ - try: - with streams_lock: - if camera_id not in streams: - logger.warning(f"Camera ID '{camera_id}' not found in streams. Current streams: {list(streams.keys())}") - raise HTTPException(status_code=404, detail=f"Camera {camera_id} not found or not active") - - stream = streams[camera_id] - buffer = stream["buffer"] - logger.debug(f"Camera '{camera_id}' buffer size: {buffer.qsize()}, buffer empty: {buffer.empty()}") - logger.debug(f"Buffer queue contents: {getattr(buffer, 'queue', None)}") - - if buffer.empty(): - logger.warning(f"No frame available for camera '{camera_id}'. Buffer is empty.") - raise HTTPException(status_code=404, detail=f"No frame available for camera {camera_id}") - - # Get the latest frame (non-blocking) - try: - frame = buffer.queue[-1] # Get the most recent frame without removing it - except IndexError: - logger.warning(f"Buffer queue is empty for camera '{camera_id}' when trying to access last frame.") - raise HTTPException(status_code=404, detail=f"No frame available for camera {camera_id}") - # Encode frame as JPEG - success, buffer_img = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 85]) - if not success: - raise HTTPException(status_code=500, detail="Failed to encode image as JPEG") - - # Return image as binary response - return Response(content=buffer_img.tobytes(), media_type="image/jpeg") - - except HTTPException: - raise - except Exception as e: - logger.error(f"Error retrieving image for camera {camera_id}: {str(e)}", exc_info=True) - raise HTTPException(status_code=500, detail=f"Internal server error: {str(e)}") - #################################################### # Detection and frame processing functions #################################################### @app.websocket("/") async def detect(websocket: WebSocket): - logger.info("WebSocket connection accepted") + logging.info("WebSocket connection accepted") persistent_data_dict = {} async def handle_detection(camera_id, stream, frame, websocket, model_tree, persistent_data): try: - # Apply crop if specified - cropped_frame = frame - if all(coord is not None for coord in [stream.get("cropX1"), stream.get("cropY1"), stream.get("cropX2"), stream.get("cropY2")]): - cropX1, cropY1, cropX2, cropY2 = stream["cropX1"], stream["cropY1"], stream["cropX2"], stream["cropY2"] - cropped_frame = frame[cropY1:cropY2, cropX1:cropX2] - logger.debug(f"Applied crop coordinates ({cropX1}, {cropY1}, {cropX2}, {cropY2}) to frame for camera {camera_id}") - - logger.debug(f"Processing frame for camera {camera_id} with model {stream['modelId']}") - start_time = time.time() - detection_result = run_pipeline(cropped_frame, model_tree) - process_time = (time.time() - start_time) * 1000 - logger.debug(f"Detection for camera {camera_id} completed in {process_time:.2f}ms") - - # Log the raw detection result for debugging - logger.debug(f"Raw detection result for camera {camera_id}:\n{json.dumps(detection_result, indent=2, default=str)}") - - # Direct class result (no detections/classifications structure) - if detection_result and isinstance(detection_result, dict) and "class" in detection_result and "confidence" in detection_result: - highest_confidence_detection = { - "class": detection_result.get("class", "none"), - "confidence": detection_result.get("confidence", 1.0), - "box": [0, 0, 0, 0] # Empty bounding box for classifications - } - # Handle case when no detections found or result is empty - elif not detection_result or not detection_result.get("detections"): - # Check if we have classification results - if detection_result and detection_result.get("classifications"): - # Get the highest confidence classification - classifications = detection_result.get("classifications", []) - highest_confidence_class = max(classifications, key=lambda x: x.get("confidence", 0)) if classifications else None - - if highest_confidence_class: - highest_confidence_detection = { - "class": highest_confidence_class.get("class", "none"), - "confidence": highest_confidence_class.get("confidence", 1.0), - "box": [0, 0, 0, 0] # Empty bounding box for classifications - } - else: - highest_confidence_detection = { - "class": "none", - "confidence": 1.0, - "box": [0, 0, 0, 0] - } - else: - highest_confidence_detection = { - "class": "none", - "confidence": 1.0, - "box": [0, 0, 0, 0] - } - else: - # Find detection with highest confidence - detections = detection_result.get("detections", []) - highest_confidence_detection = max(detections, key=lambda x: x.get("confidence", 0)) if detections else { - "class": "none", - "confidence": 1.0, - "box": [0, 0, 0, 0] - } - - # Convert detection format to match protocol - flatten detection attributes - detection_dict = {} - - # Handle different detection result formats - if isinstance(highest_confidence_detection, dict): - # Copy all fields from the detection result - for key, value in highest_confidence_detection.items(): - if key not in ["box", "id"]: # Skip internal fields - detection_dict[key] = value - - # Extract display identifier for session ID lookup - subscription_parts = stream["subscriptionIdentifier"].split(';') - display_identifier = subscription_parts[0] if subscription_parts else None - session_id = session_ids.get(display_identifier) if display_identifier else None - + detection_result = run_pipeline(frame, model_tree) detection_data = { "type": "imageDetection", - "subscriptionIdentifier": stream["subscriptionIdentifier"], - "timestamp": time.strftime("%Y-%m-%dT%H:%M:%S.%fZ", time.gmtime()), + "cameraIdentifier": camera_id, + "timestamp": time.time(), "data": { - "detection": detection_dict, + "detection": detection_result if detection_result else None, "modelId": stream["modelId"], "modelName": stream["modelName"] } } - - # Add session ID if available - if session_id is not None: - detection_data["sessionId"] = session_id - - if highest_confidence_detection["class"] != "none": - logger.info(f"Camera {camera_id}: Detected {highest_confidence_detection['class']} with confidence {highest_confidence_detection['confidence']:.2f} using model {stream['modelName']}") - - # Log session ID if available - subscription_parts = stream["subscriptionIdentifier"].split(';') - display_identifier = subscription_parts[0] if subscription_parts else None - session_id = session_ids.get(display_identifier) if display_identifier else None - if session_id: - logger.debug(f"Detection associated with session ID: {session_id}") - + logging.debug(f"Sending detection data for camera {camera_id}: {detection_data}") await websocket.send_json(detection_data) - logger.debug(f"Sent detection data to client for camera {camera_id}") return persistent_data except Exception as e: - logger.error(f"Error in handle_detection for camera {camera_id}: {str(e)}", exc_info=True) + logging.error(f"Error in handle_detection for camera {camera_id}: {e}") return persistent_data def frame_reader(camera_id, cap, buffer, stop_event): retries = 0 - logger.info(f"Starting frame reader thread for camera {camera_id}") - frame_count = 0 - last_log_time = time.time() - try: - # Log initial camera status and properties - if cap.isOpened(): - width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) - height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) - fps = cap.get(cv2.CAP_PROP_FPS) - logger.info(f"Camera {camera_id} opened successfully with resolution {width}x{height}, FPS: {fps}") - else: - logger.error(f"Camera {camera_id} failed to open initially") - while not stop_event.is_set(): try: - if not cap.isOpened(): - logger.error(f"Camera {camera_id} is not open before trying to read") - # Attempt to reopen - cap = cv2.VideoCapture(streams[camera_id]["rtsp_url"]) - time.sleep(reconnect_interval) - continue - - logger.debug(f"Attempting to read frame from camera {camera_id}") ret, frame = cap.read() - if not ret: - logger.warning(f"Connection lost for camera: {camera_id}, retry {retries+1}/{max_retries}") + logging.warning(f"Connection lost for camera: {camera_id}, retry {retries+1}/{max_retries}") cap.release() time.sleep(reconnect_interval) retries += 1 if retries > max_retries and max_retries != -1: - logger.error(f"Max retries reached for camera: {camera_id}, stopping frame reader") + logging.error(f"Max retries reached for camera: {camera_id}") break # Re-open - logger.info(f"Attempting to reopen RTSP stream for camera: {camera_id}") cap = cv2.VideoCapture(streams[camera_id]["rtsp_url"]) if not cap.isOpened(): - logger.error(f"Failed to reopen RTSP stream for camera: {camera_id}") + logging.error(f"Failed to reopen RTSP stream for camera: {camera_id}") continue - logger.info(f"Successfully reopened RTSP stream for camera: {camera_id}") continue - - # Successfully read a frame - frame_count += 1 - current_time = time.time() - # Log frame stats every 5 seconds - if current_time - last_log_time > 5: - logger.info(f"Camera {camera_id}: Read {frame_count} frames in the last {current_time - last_log_time:.1f} seconds") - frame_count = 0 - last_log_time = current_time - - logger.debug(f"Successfully read frame from camera {camera_id}, shape: {frame.shape}") retries = 0 - # Overwrite old frame if buffer is full if not buffer.empty(): try: buffer.get_nowait() - logger.debug(f"[frame_reader] Removed old frame from buffer for camera {camera_id}") except queue.Empty: pass buffer.put(frame) - logger.debug(f"[frame_reader] Added new frame to buffer for camera {camera_id}. Buffer size: {buffer.qsize()}") - - # Short sleep to avoid CPU overuse - time.sleep(0.01) - except cv2.error as e: - logger.error(f"OpenCV error for camera {camera_id}: {e}", exc_info=True) + logging.error(f"OpenCV error for camera {camera_id}: {e}") cap.release() time.sleep(reconnect_interval) retries += 1 if retries > max_retries and max_retries != -1: - logger.error(f"Max retries reached after OpenCV error for camera {camera_id}") + logging.error(f"Max retries reached after OpenCV error for camera {camera_id}") break - logger.info(f"Attempting to reopen RTSP stream after OpenCV error for camera: {camera_id}") cap = cv2.VideoCapture(streams[camera_id]["rtsp_url"]) if not cap.isOpened(): - logger.error(f"Failed to reopen RTSP stream for camera {camera_id} after OpenCV error") + logging.error(f"Failed to reopen RTSP stream for camera {camera_id} after OpenCV error") continue - logger.info(f"Successfully reopened RTSP stream after OpenCV error for camera: {camera_id}") except Exception as e: - logger.error(f"Unexpected error for camera {camera_id}: {str(e)}", exc_info=True) + logging.error(f"Unexpected error for camera {camera_id}: {e}") cap.release() break except Exception as e: - logger.error(f"Error in frame_reader thread for camera {camera_id}: {str(e)}", exc_info=True) - finally: - logger.info(f"Frame reader thread for camera {camera_id} is exiting") - if cap and cap.isOpened(): - cap.release() - - def snapshot_reader(camera_id, snapshot_url, snapshot_interval, buffer, stop_event): - """Frame reader that fetches snapshots from HTTP/HTTPS URL at specified intervals""" - retries = 0 - logger.info(f"Starting snapshot reader thread for camera {camera_id} from {snapshot_url}") - frame_count = 0 - last_log_time = time.time() - - try: - interval_seconds = snapshot_interval / 1000.0 # Convert milliseconds to seconds - logger.info(f"Snapshot interval for camera {camera_id}: {interval_seconds}s") - - while not stop_event.is_set(): - try: - start_time = time.time() - frame = fetch_snapshot(snapshot_url) - - if frame is None: - logger.warning(f"Failed to fetch snapshot for camera: {camera_id}, retry {retries+1}/{max_retries}") - retries += 1 - if retries > max_retries and max_retries != -1: - logger.error(f"Max retries reached for snapshot camera: {camera_id}, stopping reader") - break - time.sleep(min(interval_seconds, reconnect_interval)) - continue - - # Successfully fetched a frame - frame_count += 1 - current_time = time.time() - # Log frame stats every 5 seconds - if current_time - last_log_time > 5: - logger.info(f"Camera {camera_id}: Fetched {frame_count} snapshots in the last {current_time - last_log_time:.1f} seconds") - frame_count = 0 - last_log_time = current_time - - logger.debug(f"Successfully fetched snapshot from camera {camera_id}, shape: {frame.shape}") - retries = 0 - - # Overwrite old frame if buffer is full - if not buffer.empty(): - try: - buffer.get_nowait() - logger.debug(f"[snapshot_reader] Removed old snapshot from buffer for camera {camera_id}") - except queue.Empty: - pass - buffer.put(frame) - logger.debug(f"[snapshot_reader] Added new snapshot to buffer for camera {camera_id}. Buffer size: {buffer.qsize()}") - - # Wait for the specified interval - elapsed = time.time() - start_time - sleep_time = max(interval_seconds - elapsed, 0) - if sleep_time > 0: - time.sleep(sleep_time) - - except Exception as e: - logger.error(f"Unexpected error fetching snapshot for camera {camera_id}: {str(e)}", exc_info=True) - retries += 1 - if retries > max_retries and max_retries != -1: - logger.error(f"Max retries reached after error for snapshot camera {camera_id}") - break - time.sleep(min(interval_seconds, reconnect_interval)) - except Exception as e: - logger.error(f"Error in snapshot_reader thread for camera {camera_id}: {str(e)}", exc_info=True) - finally: - logger.info(f"Snapshot reader thread for camera {camera_id} is exiting") + logging.error(f"Error in frame_reader thread for camera {camera_id}: {e}") async def process_streams(): - logger.info("Started processing streams") + logging.info("Started processing streams") try: while True: start_time = time.time() with streams_lock: current_streams = list(streams.items()) - if current_streams: - logger.debug(f"Processing {len(current_streams)} active streams") - else: - logger.debug("No active streams to process") - for camera_id, stream in current_streams: buffer = stream["buffer"] - if buffer.empty(): - logger.debug(f"Frame buffer is empty for camera {camera_id}") - continue - - logger.debug(f"Got frame from buffer for camera {camera_id}") - frame = buffer.get() - - with models_lock: - model_tree = models.get(camera_id, {}).get(stream["modelId"]) - if not model_tree: - logger.warning(f"Model not found for camera {camera_id}, modelId {stream['modelId']}") - continue - logger.debug(f"Found model tree for camera {camera_id}, modelId {stream['modelId']}") - - key = (camera_id, stream["modelId"]) - persistent_data = persistent_data_dict.get(key, {}) - logger.debug(f"Starting detection for camera {camera_id} with modelId {stream['modelId']}") - updated_persistent_data = await handle_detection( - camera_id, stream, frame, websocket, model_tree, persistent_data - ) - persistent_data_dict[key] = updated_persistent_data - + if not buffer.empty(): + frame = buffer.get() + with models_lock: + model_tree = models.get(camera_id, {}).get(stream["modelId"]) + key = (camera_id, stream["modelId"]) + persistent_data = persistent_data_dict.get(key, {}) + updated_persistent_data = await handle_detection( + camera_id, stream, frame, websocket, model_tree, persistent_data + ) + persistent_data_dict[key] = updated_persistent_data elapsed_time = (time.time() - start_time) * 1000 # ms sleep_time = max(poll_interval - elapsed_time, 0) - logger.debug(f"Frame processing cycle: {elapsed_time:.2f}ms, sleeping for: {sleep_time:.2f}ms") + logging.debug(f"Elapsed time: {elapsed_time}ms, sleeping for: {sleep_time}ms") await asyncio.sleep(sleep_time / 1000.0) except asyncio.CancelledError: - logger.info("Stream processing task cancelled") + logging.info("Stream processing task cancelled") except Exception as e: - logger.error(f"Error in process_streams: {str(e)}", exc_info=True) + logging.error(f"Error in process_streams: {e}") async def send_heartbeat(): while True: @@ -506,19 +187,18 @@ async def detect(websocket: WebSocket): cpu_usage = psutil.cpu_percent() memory_usage = psutil.virtual_memory().percent if torch.cuda.is_available(): - gpu_usage = torch.cuda.utilization() if hasattr(torch.cuda, 'utilization') else None - gpu_memory_usage = torch.cuda.memory_reserved() / (1024 ** 2) + gpu_usage = torch.cuda.memory_allocated() / (1024 ** 2) # MB + gpu_memory_usage = torch.cuda.memory_reserved() / (1024 ** 2) # MB else: gpu_usage = None gpu_memory_usage = None camera_connections = [ { - "subscriptionIdentifier": stream["subscriptionIdentifier"], + "cameraIdentifier": camera_id, "modelId": stream["modelId"], "modelName": stream["modelName"], - "online": True, - **{k: v for k, v in get_crop_coords(stream).items() if v is not None} + "online": True } for camera_id, stream in streams.items() ] @@ -532,225 +212,104 @@ async def detect(websocket: WebSocket): "cameraConnections": camera_connections } await websocket.send_text(json.dumps(state_report)) - logger.debug(f"Sent stateReport as heartbeat: CPU {cpu_usage:.1f}%, Memory {memory_usage:.1f}%, {len(camera_connections)} active cameras") + logging.debug("Sent stateReport as heartbeat") await asyncio.sleep(HEARTBEAT_INTERVAL) except Exception as e: - logger.error(f"Error sending stateReport heartbeat: {e}") + logging.error(f"Error sending stateReport heartbeat: {e}") break async def on_message(): while True: try: msg = await websocket.receive_text() - logger.debug(f"Received message: {msg}") + logging.debug(f"Received message: {msg}") data = json.loads(msg) msg_type = data.get("type") if msg_type == "subscribe": payload = data.get("payload", {}) - subscriptionIdentifier = payload.get("subscriptionIdentifier") + camera_id = payload.get("cameraIdentifier") rtsp_url = payload.get("rtspUrl") - snapshot_url = payload.get("snapshotUrl") - snapshot_interval = payload.get("snapshotInterval") - model_url = payload.get("modelUrl") + model_url = payload.get("modelUrl") # may be remote or local modelId = payload.get("modelId") modelName = payload.get("modelName") - cropX1 = payload.get("cropX1") - cropY1 = payload.get("cropY1") - cropX2 = payload.get("cropX2") - cropY2 = payload.get("cropY2") - - # Extract camera_id from subscriptionIdentifier (format: displayIdentifier;cameraIdentifier) - parts = subscriptionIdentifier.split(';') - if len(parts) != 2: - logger.error(f"Invalid subscriptionIdentifier format: {subscriptionIdentifier}") - continue - - display_identifier, camera_identifier = parts - camera_id = subscriptionIdentifier # Use full subscriptionIdentifier as camera_id for mapping if model_url: with models_lock: - if (camera_id not in models) or (modelId not in models[camera_id]): - logger.info(f"Loading model from {model_url} for camera {camera_id}, modelId {modelId}") - extraction_dir = os.path.join("models", camera_identifier, str(modelId)) + if camera_id not in models: + models[camera_id] = {} + if modelId not in models[camera_id]: + logging.info(f"Loading model from {model_url}") + extraction_dir = os.path.join("models", camera_id, str(modelId)) os.makedirs(extraction_dir, exist_ok=True) # If model_url is remote, download it first. parsed = urlparse(model_url) if parsed.scheme in ("http", "https"): - logger.info(f"Downloading remote .mpta file from {model_url}") - filename = os.path.basename(parsed.path) or f"model_{modelId}.mpta" - local_mpta = os.path.join(extraction_dir, filename) - logger.debug(f"Download destination: {local_mpta}") + local_mpta = os.path.join(extraction_dir, os.path.basename(parsed.path)) local_path = download_mpta(model_url, local_mpta) if not local_path: - logger.error(f"Failed to download the remote .mpta file from {model_url}") - error_response = { - "type": "error", - "subscriptionIdentifier": subscriptionIdentifier, - "error": f"Failed to download model from {model_url}" - } - await websocket.send_json(error_response) + logging.error("Failed to download the remote mpta file.") continue model_tree = load_pipeline_from_zip(local_path, extraction_dir) else: - logger.info(f"Loading local .mpta file from {model_url}") - # Check if file exists before attempting to load - if not os.path.exists(model_url): - logger.error(f"Local .mpta file not found: {model_url}") - logger.debug(f"Current working directory: {os.getcwd()}") - error_response = { - "type": "error", - "subscriptionIdentifier": subscriptionIdentifier, - "error": f"Model file not found: {model_url}" - } - await websocket.send_json(error_response) - continue model_tree = load_pipeline_from_zip(model_url, extraction_dir) if model_tree is None: - logger.error(f"Failed to load model {modelId} from .mpta file for camera {camera_id}") - error_response = { - "type": "error", - "subscriptionIdentifier": subscriptionIdentifier, - "error": f"Failed to load model {modelId}" - } - await websocket.send_json(error_response) + logging.error("Failed to load model from mpta file.") continue - if camera_id not in models: - models[camera_id] = {} models[camera_id][modelId] = model_tree - logger.info(f"Successfully loaded model {modelId} for camera {camera_id}") - logger.debug(f"Model extraction directory: {extraction_dir}") - if camera_id and (rtsp_url or snapshot_url): + logging.info(f"Loaded model {modelId} for camera {camera_id}") + + if camera_id and rtsp_url: with streams_lock: - # Determine camera URL for shared stream management - camera_url = snapshot_url if snapshot_url else rtsp_url - if camera_id not in streams and len(streams) < max_streams: - # Check if we already have a stream for this camera URL - shared_stream = camera_streams.get(camera_url) - - if shared_stream: - # Reuse existing stream - logger.info(f"Reusing existing stream for camera URL: {camera_url}") - buffer = shared_stream["buffer"] - stop_event = shared_stream["stop_event"] - thread = shared_stream["thread"] - mode = shared_stream["mode"] - - # Increment reference count - shared_stream["ref_count"] = shared_stream.get("ref_count", 0) + 1 - else: - # Create new stream - buffer = queue.Queue(maxsize=1) - stop_event = threading.Event() - - if snapshot_url and snapshot_interval: - logger.info(f"Creating new snapshot stream for camera {camera_id}: {snapshot_url}") - thread = threading.Thread(target=snapshot_reader, args=(camera_identifier, snapshot_url, snapshot_interval, buffer, stop_event)) - thread.daemon = True - thread.start() - mode = "snapshot" - - # Store shared stream info - shared_stream = { - "buffer": buffer, - "thread": thread, - "stop_event": stop_event, - "mode": mode, - "url": snapshot_url, - "snapshot_interval": snapshot_interval, - "ref_count": 1 - } - camera_streams[camera_url] = shared_stream - - elif rtsp_url: - logger.info(f"Creating new RTSP stream for camera {camera_id}: {rtsp_url}") - cap = cv2.VideoCapture(rtsp_url) - if not cap.isOpened(): - logger.error(f"Failed to open RTSP stream for camera {camera_id}") - continue - thread = threading.Thread(target=frame_reader, args=(camera_identifier, cap, buffer, stop_event)) - thread.daemon = True - thread.start() - mode = "rtsp" - - # Store shared stream info - shared_stream = { - "buffer": buffer, - "thread": thread, - "stop_event": stop_event, - "mode": mode, - "url": rtsp_url, - "cap": cap, - "ref_count": 1 - } - camera_streams[camera_url] = shared_stream - else: - logger.error(f"No valid URL provided for camera {camera_id}") - continue - - # Create stream info for this subscription - stream_info = { + cap = cv2.VideoCapture(rtsp_url) + if not cap.isOpened(): + logging.error(f"Failed to open RTSP stream for camera {camera_id}") + continue + buffer = queue.Queue(maxsize=1) + stop_event = threading.Event() + thread = threading.Thread(target=frame_reader, args=(camera_id, cap, buffer, stop_event)) + thread.daemon = True + thread.start() + streams[camera_id] = { + "cap": cap, "buffer": buffer, "thread": thread, + "rtsp_url": rtsp_url, "stop_event": stop_event, "modelId": modelId, - "modelName": modelName, - "subscriptionIdentifier": subscriptionIdentifier, - "cropX1": cropX1, - "cropY1": cropY1, - "cropX2": cropX2, - "cropY2": cropY2, - "mode": mode, - "camera_url": camera_url + "modelName": modelName } - - if mode == "snapshot": - stream_info["snapshot_url"] = snapshot_url - stream_info["snapshot_interval"] = snapshot_interval - elif mode == "rtsp": - stream_info["rtsp_url"] = rtsp_url - stream_info["cap"] = shared_stream["cap"] - - streams[camera_id] = stream_info - subscription_to_camera[camera_id] = camera_url - + logging.info(f"Subscribed to camera {camera_id} with modelId {modelId}, modelName {modelName}, URL {rtsp_url}") elif camera_id and camera_id in streams: - # If already subscribed, unsubscribe first - logger.info(f"Resubscribing to camera {camera_id}") - # Note: Keep models in memory for reuse across subscriptions + # If already subscribed, unsubscribe + stream = streams.pop(camera_id) + stream["cap"].release() + logging.info(f"Unsubscribed from camera {camera_id}") + with models_lock: + if camera_id in models and modelId in models[camera_id]: + del models[camera_id][modelId] + if not models[camera_id]: + del models[camera_id] elif msg_type == "unsubscribe": payload = data.get("payload", {}) - subscriptionIdentifier = payload.get("subscriptionIdentifier") - camera_id = subscriptionIdentifier + camera_id = payload.get("cameraIdentifier") + logging.debug(f"Unsubscribing from camera {camera_id}") with streams_lock: if camera_id and camera_id in streams: stream = streams.pop(camera_id) - camera_url = subscription_to_camera.pop(camera_id, None) - - if camera_url and camera_url in camera_streams: - shared_stream = camera_streams[camera_url] - shared_stream["ref_count"] -= 1 - - # If no more references, stop the shared stream - if shared_stream["ref_count"] <= 0: - logger.info(f"Stopping shared stream for camera URL: {camera_url}") - shared_stream["stop_event"].set() - shared_stream["thread"].join() - if "cap" in shared_stream: - shared_stream["cap"].release() - del camera_streams[camera_url] - else: - logger.info(f"Shared stream for {camera_url} still has {shared_stream['ref_count']} references") - - logger.info(f"Unsubscribed from camera {camera_id}") - # Note: Keep models in memory for potential reuse + stream["stop_event"].set() + stream["thread"].join() + stream["cap"].release() + logging.info(f"Unsubscribed from camera {camera_id}") + with models_lock: + if camera_id in models: + del models[camera_id] elif msg_type == "requestState": cpu_usage = psutil.cpu_percent() memory_usage = psutil.virtual_memory().percent if torch.cuda.is_available(): - gpu_usage = torch.cuda.utilization() if hasattr(torch.cuda, 'utilization') else None + gpu_usage = torch.cuda.memory_allocated() / (1024 ** 2) gpu_memory_usage = torch.cuda.memory_reserved() / (1024 ** 2) else: gpu_usage = None @@ -758,11 +317,10 @@ async def detect(websocket: WebSocket): camera_connections = [ { - "subscriptionIdentifier": stream["subscriptionIdentifier"], + "cameraIdentifier": camera_id, "modelId": stream["modelId"], "modelName": stream["modelName"], - "online": True, - **{k: v for k, v in get_crop_coords(stream).items() if v is not None} + "online": True } for camera_id, stream in streams.items() ] @@ -776,47 +334,17 @@ async def detect(websocket: WebSocket): "cameraConnections": camera_connections } await websocket.send_text(json.dumps(state_report)) - - elif msg_type == "setSessionId": - payload = data.get("payload", {}) - display_identifier = payload.get("displayIdentifier") - session_id = payload.get("sessionId") - - if display_identifier: - # Store session ID for this display - if session_id is None: - session_ids.pop(display_identifier, None) - logger.info(f"Cleared session ID for display {display_identifier}") - else: - session_ids[display_identifier] = session_id - logger.info(f"Set session ID {session_id} for display {display_identifier}") - - elif msg_type == "patchSession": - session_id = data.get("sessionId") - patch_data = data.get("data", {}) - - # For now, just acknowledge the patch - actual implementation depends on backend requirements - response = { - "type": "patchSessionResult", - "payload": { - "sessionId": session_id, - "success": True, - "message": "Session patch acknowledged" - } - } - await websocket.send_json(response) - logger.info(f"Acknowledged patch for session {session_id}") - else: - logger.error(f"Unknown message type: {msg_type}") + logging.error(f"Unknown message type: {msg_type}") except json.JSONDecodeError: - logger.error("Received invalid JSON message") + logging.error("Received invalid JSON message") except (WebSocketDisconnect, ConnectionClosedError) as e: - logger.warning(f"WebSocket disconnected: {e}") + logging.warning(f"WebSocket disconnected: {e}") break except Exception as e: - logger.error(f"Error handling message: {e}") + logging.error(f"Error handling message: {e}") break + try: await websocket.accept() stream_task = asyncio.create_task(process_streams()) @@ -824,28 +352,22 @@ async def detect(websocket: WebSocket): message_task = asyncio.create_task(on_message()) await asyncio.gather(heartbeat_task, message_task) except Exception as e: - logger.error(f"Error in detect websocket: {e}") + logging.error(f"Error in detect websocket: {e}") finally: stream_task.cancel() await stream_task with streams_lock: - # Clean up shared camera streams - for camera_url, shared_stream in camera_streams.items(): - shared_stream["stop_event"].set() - shared_stream["thread"].join() - if "cap" in shared_stream: - shared_stream["cap"].release() - while not shared_stream["buffer"].empty(): + for camera_id, stream in streams.items(): + stream["stop_event"].set() + stream["thread"].join() + stream["cap"].release() + while not stream["buffer"].empty(): try: - shared_stream["buffer"].get_nowait() + stream["buffer"].get_nowait() except queue.Empty: pass - logger.info(f"Released shared camera stream for {camera_url}") - + logging.info(f"Released camera {camera_id} and cleaned up resources") streams.clear() - camera_streams.clear() - subscription_to_camera.clear() with models_lock: models.clear() - session_ids.clear() - logger.info("WebSocket connection closed") + logging.info("WebSocket connection closed") diff --git a/app_single.py b/app_single.py new file mode 100644 index 0000000..f0c8266 --- /dev/null +++ b/app_single.py @@ -0,0 +1,366 @@ +from typing import List +from fastapi import FastAPI, WebSocket +from fastapi.websockets import WebSocketDisconnect +from websockets.exceptions import ConnectionClosedError +from ultralytics import YOLO +import torch +import cv2 +import base64 +import numpy as np +import json +import logging +import threading +import queue +import os +import requests +from urllib.parse import urlparse +import asyncio +import psutil + +app = FastAPI() + +models = {} + +with open("config.json", "r") as f: + config = json.load(f) + +poll_interval = config.get("poll_interval_ms", 100) +reconnect_interval = config.get("reconnect_interval_sec", 5) +TARGET_FPS = config.get("target_fps", 10) +poll_interval = 1000 / TARGET_FPS +logging.info(f"Poll interval: {poll_interval}ms") +max_streams = config.get("max_streams", 5) +max_retries = config.get("max_retries", 3) + +# Configure logging +logging.basicConfig( + level=logging.DEBUG, + format="%(asctime)s [%(levelname)s] %(message)s", + handlers=[ + logging.FileHandler("app.log"), + logging.StreamHandler() + ] +) + +# Ensure the models directory exists +os.makedirs("models", exist_ok=True) + +# Add constants for heartbeat +HEARTBEAT_INTERVAL = 2 # seconds +WORKER_TIMEOUT_MS = 10000 + +# Add a lock for thread-safe operations on shared resources +streams_lock = threading.Lock() +models_lock = threading.Lock() + +@app.websocket("/") +async def detect(websocket: WebSocket): + import asyncio + import time + + logging.info("WebSocket connection accepted") + + streams = {} + + # This function is user-modifiable + # Save data you want to persist across frames in the persistent_data dictionary + async def handle_detection(camera_id, stream, frame, websocket, model: YOLO, persistent_data): + try: + highest_conf_box = None + max_conf = -1 + + for r in model.track(frame, stream=False, persist=True): + for box in r.boxes: + box_cpu = box.cpu() + conf = float(box_cpu.conf[0]) + if conf > max_conf and hasattr(box, "id") and box.id is not None: + max_conf = conf + highest_conf_box = { + "class": model.names[int(box_cpu.cls[0])], + "confidence": conf, + "id": box.id.item(), + } + + # Broadcast to all subscribers of this URL + detection_data = { + "type": "imageDetection", + "cameraIdentifier": camera_id, + "timestamp": time.time(), + "data": { + "detections": highest_conf_box if highest_conf_box else None, + "modelId": stream['modelId'], + "modelName": stream['modelName'] + } + } + logging.debug(f"Sending detection data for camera {camera_id}: {detection_data}") + await websocket.send_json(detection_data) + return persistent_data + except Exception as e: + logging.error(f"Error in handle_detection for camera {camera_id}: {e}") + return persistent_data + + def frame_reader(camera_id, cap, buffer, stop_event): + import time + retries = 0 + try: + while not stop_event.is_set(): + try: + ret, frame = cap.read() + if not ret: + logging.warning(f"Connection lost for camera: {camera_id}, retry {retries+1}/{max_retries}") + cap.release() + time.sleep(reconnect_interval) + retries += 1 + if retries > max_retries and max_retries != -1: + logging.error(f"Max retries reached for camera: {camera_id}") + break + # Re-open the VideoCapture + cap = cv2.VideoCapture(streams[camera_id]['rtsp_url']) + if not cap.isOpened(): + logging.error(f"Failed to reopen RTSP stream for camera: {camera_id}") + continue + continue + retries = 0 # Reset on success + if not buffer.empty(): + try: + buffer.get_nowait() # Discard the old frame + except queue.Empty: + pass + buffer.put(frame) + except cv2.error as e: + logging.error(f"OpenCV error for camera {camera_id}: {e}") + cap.release() + time.sleep(reconnect_interval) + retries += 1 + if retries > max_retries and max_retries != -1: + logging.error(f"Max retries reached after OpenCV error for camera: {camera_id}") + break + # Re-open the VideoCapture + cap = cv2.VideoCapture(streams[camera_id]['rtsp_url']) + if not cap.isOpened(): + logging.error(f"Failed to reopen RTSP stream for camera {camera_id} after OpenCV error") + continue + except Exception as e: + logging.error(f"Unexpected error for camera {camera_id}: {e}") + cap.release() + break + except Exception as e: + logging.error(f"Error in frame_reader thread for camera {camera_id}: {e}") + + async def process_streams(): + global models + logging.info("Started processing streams") + persistent_data_dict = {} + try: + while True: + start_time = time.time() + # Round-robin processing + with streams_lock: + current_streams = list(streams.items()) + for camera_id, stream in current_streams: + buffer = stream['buffer'] + if not buffer.empty(): + frame = buffer.get() + with models_lock: + model = models.get(camera_id, {}).get(stream['modelId']) + key = (camera_id, stream['modelId']) + persistent_data = persistent_data_dict.get(key, {}) + updated_persistent_data = await handle_detection(camera_id, stream, frame, websocket, model, persistent_data) + persistent_data_dict[key] = updated_persistent_data + elapsed_time = (time.time() - start_time) * 1000 # in ms + sleep_time = max(poll_interval - elapsed_time, 0) + logging.debug(f"Elapsed time: {elapsed_time}ms, sleeping for: {sleep_time}ms") + await asyncio.sleep(sleep_time / 1000.0) + except asyncio.CancelledError: + logging.info("Stream processing task cancelled") + except Exception as e: + logging.error(f"Error in process_streams: {e}") + + async def send_heartbeat(): + while True: + try: + cpu_usage = psutil.cpu_percent() + memory_usage = psutil.virtual_memory().percent + if torch.cuda.is_available(): + gpu_usage = torch.cuda.memory_allocated() / (1024 ** 2) # Convert to MB + gpu_memory_usage = torch.cuda.memory_reserved() / (1024 ** 2) # Convert to MB + else: + gpu_usage = None + gpu_memory_usage = None + + camera_connections = [ + { + "cameraIdentifier": camera_id, + "modelId": stream['modelId'], + "modelName": stream['modelName'], + "online": True + } + for camera_id, stream in streams.items() + ] + + state_report = { + "type": "stateReport", + "cpuUsage": cpu_usage, + "memoryUsage": memory_usage, + "gpuUsage": gpu_usage, + "gpuMemoryUsage": gpu_memory_usage, + "cameraConnections": camera_connections + } + await websocket.send_text(json.dumps(state_report)) + logging.debug("Sent stateReport as heartbeat") + await asyncio.sleep(HEARTBEAT_INTERVAL) + except Exception as e: + logging.error(f"Error sending stateReport heartbeat: {e}") + break + + async def on_message(): + global models + while True: + try: + msg = await websocket.receive_text() + logging.debug(f"Received message: {msg}") + print(f"Received message: {msg}") + data = json.loads(msg) + msg_type = data.get("type") + + if msg_type == "subscribe": + payload = data.get("payload", {}) + camera_id = payload.get("cameraIdentifier") + rtsp_url = payload.get("rtspUrl") + model_url = payload.get("modelUrl") + modelId = payload.get("modelId") + modelName = payload.get("modelName") + + if model_url: + with models_lock: + if camera_id not in models: + models[camera_id] = {} + if modelId not in models[camera_id]: + print(f"Downloading model from {model_url}") + parsed_url = urlparse(model_url) + filename = os.path.basename(parsed_url.path) + model_filename = os.path.join("models", filename) + # Download the model + response = requests.get(model_url, stream=True) + if response.status_code == 200: + with open(model_filename, 'wb') as f: + for chunk in response.iter_content(chunk_size=8192): + f.write(chunk) + logging.info(f"Downloaded model from {model_url} to {model_filename}") + model = YOLO(model_filename) + if torch.cuda.is_available(): + model.to('cuda') + models[camera_id][modelId] = model + logging.info(f"Loaded model {modelId} for camera {camera_id}") + else: + logging.error(f"Failed to download model from {model_url}") + continue + if camera_id and rtsp_url: + with streams_lock: + if camera_id not in streams and len(streams) < max_streams: + cap = cv2.VideoCapture(rtsp_url) + if not cap.isOpened(): + logging.error(f"Failed to open RTSP stream for camera {camera_id}") + continue + buffer = queue.Queue(maxsize=1) + stop_event = threading.Event() + thread = threading.Thread(target=frame_reader, args=(camera_id, cap, buffer, stop_event)) + thread.daemon = True + thread.start() + streams[camera_id] = { + 'cap': cap, + 'buffer': buffer, + 'thread': thread, + 'rtsp_url': rtsp_url, + 'stop_event': stop_event, + 'modelId': modelId, + 'modelName': modelName + } + logging.info(f"Subscribed to camera {camera_id} with modelId {modelId}, modelName {modelName} and URL {rtsp_url}") + elif camera_id and camera_id in streams: + stream = streams.pop(camera_id) + stream['cap'].release() + logging.info(f"Unsubscribed from camera {camera_id}") + if camera_id in models and modelId in models[camera_id]: + del models[camera_id][modelId] + if not models[camera_id]: + del models[camera_id] + elif msg_type == "unsubscribe": + payload = data.get("payload", {}) + camera_id = payload.get("cameraIdentifier") + logging.debug(f"Unsubscribing from camera {camera_id}") + with streams_lock: + if camera_id and camera_id in streams: + stream = streams.pop(camera_id) + stream['stop_event'].set() + stream['thread'].join() + stream['cap'].release() + logging.info(f"Unsubscribed from camera {camera_id}") + if camera_id in models and modelId in models[camera_id]: + del models[camera_id][modelId] + if not models[camera_id]: + del models[camera_id] + elif msg_type == "requestState": + # Handle state request + cpu_usage = psutil.cpu_percent() + memory_usage = psutil.virtual_memory().percent + if torch.cuda.is_available(): + gpu_usage = torch.cuda.memory_allocated() / (1024 ** 2) # Convert to MB + gpu_memory_usage = torch.cuda.memory_reserved() / (1024 ** 2) # Convert to MB + else: + gpu_usage = None + gpu_memory_usage = None + + camera_connections = [ + { + "cameraIdentifier": camera_id, + "modelId": stream['modelId'], + "modelName": stream['modelName'], + "online": True + } + for camera_id, stream in streams.items() + ] + + state_report = { + "type": "stateReport", + "cpuUsage": cpu_usage, + "memoryUsage": memory_usage, + "gpuUsage": gpu_usage, + "gpuMemoryUsage": gpu_memory_usage, + "cameraConnections": camera_connections + } + await websocket.send_text(json.dumps(state_report)) + else: + logging.error(f"Unknown message type: {msg_type}") + except json.JSONDecodeError: + logging.error("Received invalid JSON message") + except (WebSocketDisconnect, ConnectionClosedError) as e: + logging.warning(f"WebSocket disconnected: {e}") + break + except Exception as e: + logging.error(f"Error handling message: {e}") + break + + try: + await websocket.accept() + task = asyncio.create_task(process_streams()) + heartbeat_task = asyncio.create_task(send_heartbeat()) + message_task = asyncio.create_task(on_message()) + + await asyncio.gather(heartbeat_task, message_task) + except Exception as e: + logging.error(f"Error in detect websocket: {e}") + finally: + task.cancel() + await task + with streams_lock: + for camera_id, stream in streams.items(): + stream['stop_event'].set() + stream['thread'].join() + stream['cap'].release() + stream['buffer'].queue.clear() + logging.info(f"Released camera {camera_id} and cleaned up resources") + streams.clear() + with models_lock: + models.clear() + logging.info("WebSocket connection closed") \ No newline at end of file diff --git a/debug.py b/debug.py new file mode 100644 index 0000000..012ccde --- /dev/null +++ b/debug.py @@ -0,0 +1,143 @@ +import argparse +import os +import cv2 +import time +import logging +import shutil +import threading # added threading +import yaml # for silencing YOLO + +from siwatsystem.pympta import load_pipeline_from_zip, run_pipeline + +# Configure logging +logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s") + +# Silence YOLO logging +os.environ["YOLO_VERBOSE"] = "False" +for logger_name in ["ultralytics", "ultralytics.hub", "ultralytics.yolo.utils"]: + logging.getLogger(logger_name).setLevel(logging.WARNING) + +# Global variables for frame sharing +global_frame = None +global_ret = False +capture_running = False + +def video_capture_loop(cap): + global global_frame, global_ret, capture_running + while capture_running: + global_ret, global_frame = cap.read() + time.sleep(0.01) # slight delay to reduce CPU usage + +def clear_cache(cache_dir: str): + if os.path.exists(cache_dir): + shutil.rmtree(cache_dir) + +def log_pipeline_flow(frame, model_tree, level=0): + """ + Wrapper around run_pipeline that logs the model flow and detection results. + Returns the same output as the original run_pipeline function. + """ + indent = " " * level + model_id = model_tree.get("modelId", "unknown") + logging.info(f"{indent}→ Running model: {model_id}") + + detection, bbox = run_pipeline(frame, model_tree, return_bbox=True) + + if detection: + confidence = detection.get("confidence", 0) * 100 + class_name = detection.get("class", "unknown") + object_id = detection.get("id", "N/A") + + logging.info(f"{indent}✓ Detected: {class_name} (ID: {object_id}, confidence: {confidence:.1f}%)") + + # Check if any branches were triggered + triggered = False + for branch in model_tree.get("branches", []): + trigger_classes = branch.get("triggerClasses", []) + min_conf = branch.get("minConfidence", 0) + + if class_name in trigger_classes and detection.get("confidence", 0) >= min_conf: + triggered = True + if branch.get("crop", False) and bbox: + x1, y1, x2, y2 = bbox + cropped_frame = frame[y1:y2, x1:x2] + logging.info(f"{indent} ⌊ Triggering branch with cropped region {x1},{y1} to {x2},{y2}") + branch_result = log_pipeline_flow(cropped_frame, branch, level + 1) + else: + logging.info(f"{indent} ⌊ Triggering branch with full frame") + branch_result = log_pipeline_flow(frame, branch, level + 1) + + if branch_result[0]: # If branch detection successful, return it + return branch_result + + if not triggered and model_tree.get("branches"): + logging.info(f"{indent} ⌊ No branches triggered") + else: + logging.info(f"{indent}✗ No detection for {model_id}") + + return detection, bbox + +def main(mpta_file: str, video_source: str): + global capture_running + CACHE_DIR = os.path.join(".", ".mptacache") + clear_cache(CACHE_DIR) + logging.info(f"Loading pipeline from local file: {mpta_file}") + model_tree = load_pipeline_from_zip(mpta_file, CACHE_DIR) + if model_tree is None: + logging.error("Failed to load pipeline.") + return + + cap = cv2.VideoCapture(video_source) + if not cap.isOpened(): + logging.error(f"Cannot open video source {video_source}") + return + + # Start video capture in a separate thread + capture_running = True + capture_thread = threading.Thread(target=video_capture_loop, args=(cap,)) + capture_thread.start() + + logging.info("Press 'q' to exit.") + try: + while True: + # Use the global frame and ret updated by the thread + if not global_ret or global_frame is None: + continue # wait until a frame is available + + frame = global_frame.copy() # local copy to work with + + # Replace run_pipeline with our logging version + detection, bbox = log_pipeline_flow(frame, model_tree) + + # Stop if "honda" is detected + if detection and detection.get("class", "").lower() == "toyota": + logging.info("Detected 'toyota'. Stopping pipeline.") + break + + if bbox: + x1, y1, x2, y2 = bbox + cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2) + label = detection["class"] if detection else "Detection" + cv2.putText(frame, label, (x1, y1 - 10), + cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36, 255, 12), 2) + + cv2.imshow("Pipeline Webcam", frame) + if cv2.waitKey(1) & 0xFF == ord('q'): + break + + finally: + # Stop capture thread and cleanup + capture_running = False + capture_thread.join() + cap.release() + cv2.destroyAllWindows() + clear_cache(CACHE_DIR) + logging.info("Cleaned up .mptacache directory on shutdown.") + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Run pipeline webcam utility.") + parser.add_argument("--mpta-file", type=str, required=True, help="Path to the local pipeline mpta (ZIP) file.") + parser.add_argument("--video", type=str, default="0", help="Video source (default webcam index 0).") + args = parser.parse_args() + video_source = int(args.video) if args.video.isdigit() else args.video + main(args.mpta_file, video_source) diff --git a/demoa.mpta b/demoa.mpta new file mode 100644 index 0000000..7471d5d Binary files /dev/null and b/demoa.mpta differ diff --git a/pipeline.log b/pipeline.log new file mode 100644 index 0000000..d3a14c7 --- /dev/null +++ b/pipeline.log @@ -0,0 +1,23 @@ +2025-05-12 18:10:04,590 [INFO] Loading pipeline from local file: demoa.mpta +2025-05-12 18:10:04,610 [INFO] Copied local .mpta file from demoa.mpta to .\.mptacache\pipeline.mpta +2025-05-12 18:10:04,901 [INFO] Extracted .mpta file to .\.mptacache +2025-05-12 18:10:04,905 [INFO] Loading model for node DetectionDraft from .\.mptacache\demoa\DetectionDraft.pt +2025-05-12 18:10:05,083 [INFO] Loading model for node ClassificationDraft from .\.mptacache\demoa\ClassificationDraft.pt +2025-05-12 18:10:08,035 [INFO] Press 'q' to exit. +2025-05-12 18:10:12,217 [INFO] Cleaned up .mptacache directory on shutdown. +2025-05-12 18:13:08,465 [INFO] Loading pipeline from local file: demoa.mpta +2025-05-12 18:13:08,512 [INFO] Copied local .mpta file from demoa.mpta to .\.mptacache\pipeline.mpta +2025-05-12 18:13:08,769 [INFO] Extracted .mpta file to .\.mptacache +2025-05-12 18:13:08,773 [INFO] Loading model for node DetectionDraft from .\.mptacache\demoa\DetectionDraft.pt +2025-05-12 18:13:09,083 [INFO] Loading model for node ClassificationDraft from .\.mptacache\demoa\ClassificationDraft.pt +2025-05-12 18:13:12,187 [INFO] Press 'q' to exit. +2025-05-12 18:13:14,146 [INFO] → Running model: DetectionDraft +2025-05-12 18:13:17,119 [INFO] Cleaned up .mptacache directory on shutdown. +2025-05-12 18:14:25,665 [INFO] Loading pipeline from local file: demoa.mpta +2025-05-12 18:14:25,687 [INFO] Copied local .mpta file from demoa.mpta to .\.mptacache\pipeline.mpta +2025-05-12 18:14:25,953 [INFO] Extracted .mpta file to .\.mptacache +2025-05-12 18:14:25,957 [INFO] Loading model for node DetectionDraft from .\.mptacache\demoa\DetectionDraft.pt +2025-05-12 18:14:26,138 [INFO] Loading model for node ClassificationDraft from .\.mptacache\demoa\ClassificationDraft.pt +2025-05-12 18:14:29,171 [INFO] Press 'q' to exit. +2025-05-12 18:14:30,146 [INFO] → Running model: DetectionDraft +2025-05-12 18:14:32,080 [INFO] Cleaned up .mptacache directory on shutdown. diff --git a/pympta.md b/pympta.md deleted file mode 100644 index ac61f4a..0000000 --- a/pympta.md +++ /dev/null @@ -1,204 +0,0 @@ -# pympta: Modular Pipeline Task Executor - -`pympta` is a Python module designed to load and execute modular, multi-stage AI pipelines defined in a special package format (`.mpta`). It is primarily used within the detector worker to run complex computer vision tasks where the output of one model can trigger a subsequent model on a specific region of interest. - -## Core Concepts - -### 1. MPTA Package (`.mpta`) - -An `.mpta` file is a standard `.zip` archive with a different extension. It bundles all the necessary components for a pipeline to run. - -A typical `.mpta` file has the following structure: - -``` -my_pipeline.mpta/ -├── pipeline.json -├── model1.pt -├── model2.pt -└── ... -``` - -- **`pipeline.json`**: (Required) The manifest file that defines the structure of the pipeline, the models to use, and the logic connecting them. -- **Model Files (`.pt`, etc.)**: The actual pre-trained model files (e.g., PyTorch, ONNX). The pipeline currently uses `ultralytics.YOLO` models. - -### 2. Pipeline Structure - -A pipeline is a tree-like structure of "nodes," defined in `pipeline.json`. - -- **Root Node**: The entry point of the pipeline. It processes the initial, full-frame image. -- **Branch Nodes**: Child nodes that are triggered by specific detection results from their parent. For example, a root node might detect a "vehicle," which then triggers a branch node to detect a "license plate" within the vehicle's bounding box. - -This modular structure allows for creating complex and efficient inference logic, avoiding the need to run every model on every frame. - -## `pipeline.json` Specification - -This file defines the entire pipeline logic. The root object contains a `pipeline` key for the pipeline definition and an optional `redis` key for Redis configuration. - -### Top-Level Object Structure - -| Key | Type | Required | Description | -| ---------- | ------ | -------- | ------------------------------------------------------- | -| `pipeline` | Object | Yes | The root node object of the pipeline. | -| `redis` | Object | No | Configuration for connecting to a Redis server. | - -### Redis Configuration (`redis`) - -| Key | Type | Required | Description | -| ---------- | ------ | -------- | ------------------------------------------------------- | -| `host` | String | Yes | The hostname or IP address of the Redis server. | -| `port` | Number | Yes | The port number of the Redis server. | -| `password` | String | No | The password for Redis authentication. | -| `db` | Number | No | The Redis database number to use. Defaults to `0`. | - -### Node Object Structure - -| Key | Type | Required | Description | -| ------------------- | ------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------- | -| `modelId` | String | Yes | A unique identifier for this model node (e.g., "vehicle-detector"). | -| `modelFile` | String | Yes | The path to the model file within the `.mpta` archive (e.g., "yolov8n.pt"). | -| `minConfidence` | Float | Yes | The minimum confidence score (0.0 to 1.0) required for a detection to be considered valid and potentially trigger a branch. | -| `triggerClasses` | Array | Yes | A list of class names that, when detected by the parent, can trigger this node. For the root node, this lists all classes of interest. | -| `crop` | Boolean | No | If `true`, the image is cropped to the parent's detection bounding box before being passed to this node's model. Defaults to `false`. | -| `branches` | Array | No | A list of child node objects that can be triggered by this node's detections. | -| `actions` | Array | No | A list of actions to execute upon a successful detection in this node. | - -### Action Object Structure - -Actions allow the pipeline to interact with Redis. They are executed sequentially for a given detection. - -#### Action Context & Dynamic Keys - -All actions have access to a dynamic context for formatting keys and messages. The context is created for each detection event and includes: - -- All key-value pairs from the detection result (e.g., `class`, `confidence`, `id`). -- `{timestamp_ms}`: The current Unix timestamp in milliseconds. -- `{uuid}`: A unique identifier (UUID4) for the detection event. -- `{image_key}`: If a `redis_save_image` action has already been executed for this event, this placeholder will be replaced with the key where the image was stored. - -#### `redis_save_image` - -Saves the current image frame (or cropped sub-image) to a Redis key. - -| Key | Type | Required | Description | -| ---------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------- | -| `type` | String | Yes | Must be `"redis_save_image"`. | -| `key` | String | Yes | The Redis key to save the image to. Can contain any of the dynamic placeholders. | -| `expire_seconds` | Number | No | If provided, sets an expiration time (in seconds) for the Redis key. | - -#### `redis_publish` - -Publishes a message to a Redis channel. - -| Key | Type | Required | Description | -| --------- | ------ | -------- | ------------------------------------------------------------------------------------------------------- | -| `type` | String | Yes | Must be `"redis_publish"`. | -| `channel` | String | Yes | The Redis channel to publish the message to. | -| `message` | String | Yes | The message to publish. Can contain any of the dynamic placeholders, including `{image_key}`. | - -### Example `pipeline.json` with Redis - -This example demonstrates a pipeline that detects vehicles, saves a uniquely named image of each detection that expires in one hour, and then publishes a notification with the image key. - -```json -{ - "redis": { - "host": "redis.local", - "port": 6379, - "password": "your-super-secret-password" - }, - "pipeline": { - "modelId": "vehicle-detector", - "modelFile": "vehicle_model.pt", - "minConfidence": 0.6, - "triggerClasses": ["car", "truck"], - "actions": [ - { - "type": "redis_save_image", - "key": "detections:{class}:{timestamp_ms}:{uuid}", - "expire_seconds": 3600 - }, - { - "type": "redis_publish", - "channel": "vehicle_events", - "message": "{\"event\":\"new_detection\",\"class\":\"{class}\",\"confidence\":{confidence},\"image_key\":\"{image_key}\"}" - } - ], - "branches": [] - } -} -``` - -## API Reference - -The `pympta` module exposes two main functions. - -### `load_pipeline_from_zip(zip_source: str, target_dir: str) -> dict` - -Loads, extracts, and parses an `.mpta` file to build a pipeline tree in memory. It also establishes a Redis connection if configured in `pipeline.json`. - -- **Parameters:** - - `zip_source` (str): The file path to the local `.mpta` zip archive. - - `target_dir` (str): A directory path where the archive's contents will be extracted. -- **Returns:** - - A dictionary representing the root node of the pipeline, ready to be used with `run_pipeline`. Returns `None` if loading fails. - -### `run_pipeline(frame, node: dict, return_bbox: bool = False)` - -Executes the inference pipeline on a single image frame. - -- **Parameters:** - - `frame`: The input image frame (e.g., a NumPy array from OpenCV). - - `node` (dict): The pipeline node to execute (typically the root node returned by `load_pipeline_from_zip`). - - `return_bbox` (bool): If `True`, the function returns a tuple `(detection, bounding_box)`. Otherwise, it returns only the `detection`. -- **Returns:** - - The final detection result from the last executed node in the chain. A detection is a dictionary like `{'class': 'car', 'confidence': 0.95, 'id': 1}`. If no detection meets the criteria, it returns `None` (or `(None, None)` if `return_bbox` is `True`). - -## Usage Example - -This snippet, inspired by `pipeline_webcam.py`, shows how to use `pympta` to load a pipeline and process an image from a webcam. - -```python -import cv2 -from siwatsystem.pympta import load_pipeline_from_zip, run_pipeline - -# 1. Define paths -MPTA_FILE = "path/to/your/pipeline.mpta" -CACHE_DIR = ".mptacache" - -# 2. Load the pipeline from the .mpta file -# This reads pipeline.json and loads the YOLO models into memory. -model_tree = load_pipeline_from_zip(MPTA_FILE, CACHE_DIR) - -if not model_tree: - print("Failed to load pipeline.") - exit() - -# 3. Open a video source -cap = cv2.VideoCapture(0) - -while True: - ret, frame = cap.read() - if not ret: - break - - # 4. Run the pipeline on the current frame - # The function will handle the entire logic tree (e.g., find a car, then find its license plate). - detection_result, bounding_box = run_pipeline(frame, model_tree, return_bbox=True) - - # 5. Display the results - if detection_result: - print(f"Detected: {detection_result['class']} with confidence {detection_result['confidence']:.2f}") - if bounding_box: - x1, y1, x2, y2 = bounding_box - cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2) - cv2.putText(frame, detection_result['class'], (x1, y1 - 10), - cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36, 255, 12), 2) - - cv2.imshow("Pipeline Output", frame) - - if cv2.waitKey(1) & 0xFF == ord('q'): - break - -cap.release() -cv2.destroyAllWindows() -``` \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 49ca601..84f45cc 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,5 +5,4 @@ torchvision ultralytics opencv-python websockets -fastapi[standard] -redis \ No newline at end of file +fastapi[standard] \ No newline at end of file diff --git a/siwatsystem/pympta.py b/siwatsystem/pympta.py index f151b55..fc58f3b 100644 --- a/siwatsystem/pympta.py +++ b/siwatsystem/pympta.py @@ -3,228 +3,69 @@ import json import logging import torch import cv2 -import requests import zipfile import shutil -import traceback -import redis -import time -import uuid from ultralytics import YOLO from urllib.parse import urlparse -# Create a logger specifically for this module -logger = logging.getLogger("detector_worker.pympta") - -def load_pipeline_node(node_config: dict, mpta_dir: str, redis_client) -> dict: - # Recursively load a model node from configuration. +def load_pipeline_node(node_config: dict, mpta_dir: str) -> dict: model_path = os.path.join(mpta_dir, node_config["modelFile"]) if not os.path.exists(model_path): - logger.error(f"Model file {model_path} not found. Current directory: {os.getcwd()}") - logger.error(f"Directory content: {os.listdir(os.path.dirname(model_path))}") + logging.error(f"Model file {model_path} not found.") raise FileNotFoundError(f"Model file {model_path} not found.") - logger.info(f"Loading model for node {node_config['modelId']} from {model_path}") + logging.info(f"Loading model {node_config['modelId']} from {model_path}") model = YOLO(model_path) if torch.cuda.is_available(): - logger.info(f"CUDA available. Moving model {node_config['modelId']} to GPU") model.to("cuda") - else: - logger.info(f"CUDA not available. Using CPU for model {node_config['modelId']}") - # Prepare trigger class indices for optimization - trigger_classes = node_config.get("triggerClasses", []) - trigger_class_indices = None - if trigger_classes and hasattr(model, "names"): - # Convert class names to indices for the model - trigger_class_indices = [i for i, name in model.names.items() - if name in trigger_classes] - logger.debug(f"Converted trigger classes to indices: {trigger_class_indices}") + # map triggerClasses names → indices for YOLO + names = model.names # idx -> class name + trigger_names = node_config.get("triggerClasses", []) + trigger_inds = [i for i, nm in names.items() if nm in trigger_names] - node = { + return { "modelId": node_config["modelId"], "modelFile": node_config["modelFile"], - "triggerClasses": trigger_classes, - "triggerClassIndices": trigger_class_indices, + "triggerClasses": trigger_names, + "triggerClassIndices": trigger_inds, "crop": node_config.get("crop", False), - "minConfidence": node_config.get("minConfidence", None), - "actions": node_config.get("actions", []), + "minConfidence": node_config.get("minConfidence", 0.0), "model": model, - "branches": [], - "redis_client": redis_client + "branches": [ + load_pipeline_node(child, mpta_dir) + for child in node_config.get("branches", []) + ] } - logger.debug(f"Configured node {node_config['modelId']} with trigger classes: {node['triggerClasses']}") - for child in node_config.get("branches", []): - logger.debug(f"Loading branch for parent node {node_config['modelId']}") - node["branches"].append(load_pipeline_node(child, mpta_dir, redis_client)) - return node def load_pipeline_from_zip(zip_source: str, target_dir: str) -> dict: - logger.info(f"Attempting to load pipeline from {zip_source} to {target_dir}") os.makedirs(target_dir, exist_ok=True) zip_path = os.path.join(target_dir, "pipeline.mpta") - - # Parse the source; only local files are supported here. parsed = urlparse(zip_source) if parsed.scheme in ("", "file"): - local_path = parsed.path if parsed.scheme == "file" else zip_source - logger.debug(f"Checking if local file exists: {local_path}") - if os.path.exists(local_path): - try: - shutil.copy(local_path, zip_path) - logger.info(f"Copied local .mpta file from {local_path} to {zip_path}") - except Exception as e: - logger.error(f"Failed to copy local .mpta file from {local_path}: {str(e)}", exc_info=True) - return None - else: - logger.error(f"Local file {local_path} does not exist. Current directory: {os.getcwd()}") - # List all subdirectories of models directory to help debugging - if os.path.exists("models"): - logger.error(f"Content of models directory: {os.listdir('models')}") - for root, dirs, files in os.walk("models"): - logger.error(f"Directory {root} contains subdirs: {dirs} and files: {files}") - else: - logger.error("The models directory doesn't exist") + local = parsed.path if parsed.scheme == "file" else zip_source + if not os.path.exists(local): + logging.error(f"Local file {local} does not exist.") return None + shutil.copy(local, zip_path) else: - logger.error(f"HTTP download functionality has been moved. Use a local file path here. Received: {zip_source}") + logging.error("HTTP download not supported; use local file.") return None - try: - if not os.path.exists(zip_path): - logger.error(f"Zip file not found at expected location: {zip_path}") - return None - - logger.debug(f"Extracting .mpta file from {zip_path} to {target_dir}") - # Extract contents and track the directories created - extracted_dirs = [] - with zipfile.ZipFile(zip_path, "r") as zip_ref: - file_list = zip_ref.namelist() - logger.debug(f"Files in .mpta archive: {file_list}") - - # Extract and track the top-level directories - for file_path in file_list: - parts = file_path.split('/') - if len(parts) > 1: - top_dir = parts[0] - if top_dir and top_dir not in extracted_dirs: - extracted_dirs.append(top_dir) - - # Now extract the files - zip_ref.extractall(target_dir) - - logger.info(f"Successfully extracted .mpta file to {target_dir}") - logger.debug(f"Extracted directories: {extracted_dirs}") - - # Check what was actually created after extraction - actual_dirs = [d for d in os.listdir(target_dir) if os.path.isdir(os.path.join(target_dir, d))] - logger.debug(f"Actual directories created: {actual_dirs}") - except zipfile.BadZipFile as e: - logger.error(f"Bad zip file {zip_path}: {str(e)}", exc_info=True) - return None - except Exception as e: - logger.error(f"Failed to extract .mpta file {zip_path}: {str(e)}", exc_info=True) - return None - finally: - if os.path.exists(zip_path): - os.remove(zip_path) - logger.debug(f"Removed temporary zip file: {zip_path}") + with zipfile.ZipFile(zip_path, "r") as z: + z.extractall(target_dir) + os.remove(zip_path) - # Use the first extracted directory if it exists, otherwise use the expected name - pipeline_name = os.path.basename(zip_source) - pipeline_name = os.path.splitext(pipeline_name)[0] - - # Find the directory with pipeline.json - mpta_dir = None - # First try the expected directory name - expected_dir = os.path.join(target_dir, pipeline_name) - if os.path.exists(expected_dir) and os.path.exists(os.path.join(expected_dir, "pipeline.json")): - mpta_dir = expected_dir - logger.debug(f"Found pipeline.json in the expected directory: {mpta_dir}") - else: - # Look through all subdirectories for pipeline.json - for subdir in actual_dirs: - potential_dir = os.path.join(target_dir, subdir) - if os.path.exists(os.path.join(potential_dir, "pipeline.json")): - mpta_dir = potential_dir - logger.info(f"Found pipeline.json in directory: {mpta_dir} (different from expected: {expected_dir})") - break - - if not mpta_dir: - logger.error(f"Could not find pipeline.json in any extracted directory. Directory content: {os.listdir(target_dir)}") - return None - - pipeline_json_path = os.path.join(mpta_dir, "pipeline.json") - if not os.path.exists(pipeline_json_path): - logger.error(f"pipeline.json not found in the .mpta file. Files in directory: {os.listdir(mpta_dir)}") + base = os.path.splitext(os.path.basename(zip_source))[0] + mpta_dir = os.path.join(target_dir, base) + cfg = os.path.join(mpta_dir, "pipeline.json") + if not os.path.exists(cfg): + logging.error("pipeline.json not found in archive.") return None - try: - with open(pipeline_json_path, "r") as f: - pipeline_config = json.load(f) - logger.info(f"Successfully loaded pipeline configuration from {pipeline_json_path}") - logger.debug(f"Pipeline config: {json.dumps(pipeline_config, indent=2)}") - - # Establish Redis connection if configured - redis_client = None - if "redis" in pipeline_config: - redis_config = pipeline_config["redis"] - try: - redis_client = redis.Redis( - host=redis_config["host"], - port=redis_config["port"], - password=redis_config.get("password"), - db=redis_config.get("db", 0), - decode_responses=True - ) - redis_client.ping() - logger.info(f"Successfully connected to Redis at {redis_config['host']}:{redis_config['port']}") - except redis.exceptions.ConnectionError as e: - logger.error(f"Failed to connect to Redis: {e}") - redis_client = None - - return load_pipeline_node(pipeline_config["pipeline"], mpta_dir, redis_client) - except json.JSONDecodeError as e: - logger.error(f"Error parsing pipeline.json: {str(e)}", exc_info=True) - return None - except KeyError as e: - logger.error(f"Missing key in pipeline.json: {str(e)}", exc_info=True) - return None - except Exception as e: - logger.error(f"Error loading pipeline.json: {str(e)}", exc_info=True) - return None + with open(cfg) as f: + pipeline_config = json.load(f) + return load_pipeline_node(pipeline_config["pipeline"], mpta_dir) -def execute_actions(node, frame, detection_result): - if not node["redis_client"] or not node["actions"]: - return - - # Create a dynamic context for this detection event - action_context = { - **detection_result, - "timestamp_ms": int(time.time() * 1000), - "uuid": str(uuid.uuid4()), - } - - for action in node["actions"]: - try: - if action["type"] == "redis_save_image": - key = action["key"].format(**action_context) - _, buffer = cv2.imencode('.jpg', frame) - expire_seconds = action.get("expire_seconds") - if expire_seconds: - node["redis_client"].setex(key, expire_seconds, buffer.tobytes()) - logger.info(f"Saved image to Redis with key: {key} (expires in {expire_seconds}s)") - else: - node["redis_client"].set(key, buffer.tobytes()) - logger.info(f"Saved image to Redis with key: {key}") - # Add the generated key to the context for subsequent actions - action_context["image_key"] = key - elif action["type"] == "redis_publish": - channel = action["channel"] - message = action["message"].format(**action_context) - node["redis_client"].publish(channel, message) - logger.info(f"Published message to Redis channel '{channel}': {message}") - except Exception as e: - logger.error(f"Error executing action {action['type']}: {e}") def run_pipeline(frame, node: dict, return_bbox: bool=False): """ @@ -241,6 +82,26 @@ def run_pipeline(frame, node: dict, return_bbox: bool=False): task = getattr(node["model"], "task", None) # ─── Classification stage ─────────────────────────────────── + # if task == "classify": + # results = node["model"].predict(frame, stream=False) + # dets = [] + # for r in results: + # probs = r.probs + # if probs is not None: + # # sort descending + # idxs = probs.argsort(descending=True) + # for cid in idxs: + # dets.append({ + # "class": node["model"].names[int(cid)], + # "confidence": float(probs[int(cid)]), + # "id": None + # }) + # if not dets: + # return (None, None) if return_bbox else None + + # best = dets[0] + # return (best, None) if return_bbox else best + if task == "classify": # run the classifier and grab its top-1 directly via the Probs API results = node["model"].predict(frame, stream=False) @@ -263,7 +124,6 @@ def run_pipeline(frame, node: dict, return_bbox: bool=False): "confidence": top1_conf, "id": None } - execute_actions(node, frame, det) return (det, None) if return_bbox else det @@ -312,11 +172,9 @@ def run_pipeline(frame, node: dict, return_bbox: bool=False): det2, _ = run_pipeline(sub, br, return_bbox=True) if det2: # return classification result + original bbox - execute_actions(br, sub, det2) return (det2, best_box) if return_bbox else det2 # ─── No branch matched → return this detection ───────────── - execute_actions(node, frame, best_det) return (best_det, best_box) if return_bbox else best_det except Exception as e: diff --git a/test_protocol.py b/test_protocol.py deleted file mode 100644 index 74af7d8..0000000 --- a/test_protocol.py +++ /dev/null @@ -1,125 +0,0 @@ -#!/usr/bin/env python3 -""" -Test script to verify the worker implementation follows the protocol -""" -import json -import asyncio -import websockets -import time - -async def test_protocol(): - """Test the worker protocol implementation""" - uri = "ws://localhost:8000" - - try: - async with websockets.connect(uri) as websocket: - print("✓ Connected to worker") - - # Test 1: Check if we receive heartbeat (stateReport) - print("\n1. Testing heartbeat...") - try: - message = await asyncio.wait_for(websocket.recv(), timeout=5) - data = json.loads(message) - if data.get("type") == "stateReport": - print("✓ Received stateReport heartbeat") - print(f" - CPU Usage: {data.get('cpuUsage', 'N/A')}%") - print(f" - Memory Usage: {data.get('memoryUsage', 'N/A')}%") - print(f" - Camera Connections: {len(data.get('cameraConnections', []))}") - else: - print(f"✗ Expected stateReport, got {data.get('type')}") - except asyncio.TimeoutError: - print("✗ No heartbeat received within 5 seconds") - - # Test 2: Request state - print("\n2. Testing requestState...") - await websocket.send(json.dumps({"type": "requestState"})) - try: - message = await asyncio.wait_for(websocket.recv(), timeout=5) - data = json.loads(message) - if data.get("type") == "stateReport": - print("✓ Received stateReport response") - else: - print(f"✗ Expected stateReport, got {data.get('type')}") - except asyncio.TimeoutError: - print("✗ No response to requestState within 5 seconds") - - # Test 3: Set session ID - print("\n3. Testing setSessionId...") - session_message = { - "type": "setSessionId", - "payload": { - "displayIdentifier": "display-001", - "sessionId": 12345 - } - } - await websocket.send(json.dumps(session_message)) - print("✓ Sent setSessionId message") - - # Test 4: Test patchSession - print("\n4. Testing patchSession...") - patch_message = { - "type": "patchSession", - "sessionId": 12345, - "data": { - "currentCar": { - "carModel": "Civic", - "carBrand": "Honda" - } - } - } - await websocket.send(json.dumps(patch_message)) - - # Wait for patchSessionResult - try: - message = await asyncio.wait_for(websocket.recv(), timeout=5) - data = json.loads(message) - if data.get("type") == "patchSessionResult": - print("✓ Received patchSessionResult") - print(f" - Success: {data.get('payload', {}).get('success')}") - print(f" - Message: {data.get('payload', {}).get('message')}") - else: - print(f"✗ Expected patchSessionResult, got {data.get('type')}") - except asyncio.TimeoutError: - print("✗ No patchSessionResult received within 5 seconds") - - # Test 5: Test subscribe message format (without actual camera) - print("\n5. Testing subscribe message format...") - subscribe_message = { - "type": "subscribe", - "payload": { - "subscriptionIdentifier": "display-001;cam-001", - "snapshotUrl": "http://example.com/snapshot.jpg", - "snapshotInterval": 5000, - "modelUrl": "http://example.com/model.mpta", - "modelName": "Test Model", - "modelId": 101, - "cropX1": 100, - "cropY1": 200, - "cropX2": 300, - "cropY2": 400 - } - } - await websocket.send(json.dumps(subscribe_message)) - print("✓ Sent subscribe message (will fail without actual camera/model)") - - # Listen for a few more messages to catch any errors - print("\n6. Listening for additional messages...") - for i in range(3): - try: - message = await asyncio.wait_for(websocket.recv(), timeout=2) - data = json.loads(message) - msg_type = data.get("type") - print(f" - Received {msg_type}") - if msg_type == "error": - print(f" Error: {data.get('error')}") - except asyncio.TimeoutError: - break - - print("\n✓ Protocol test completed successfully!") - - except Exception as e: - print(f"✗ Connection failed: {e}") - print("Make sure the worker is running on localhost:8000") - -if __name__ == "__main__": - asyncio.run(test_protocol()) \ No newline at end of file diff --git a/worker.md b/worker.md deleted file mode 100644 index c50bae5..0000000 --- a/worker.md +++ /dev/null @@ -1,483 +0,0 @@ -# Worker Communication Protocol - -This document outlines the WebSocket-based communication protocol between the CMS backend and a detector worker. As a worker developer, your primary responsibility is to implement a WebSocket server that adheres to this protocol. - -## 1. Connection - -The worker must run a WebSocket server, preferably on port `8000`. The backend system, which is managed by a container orchestration service, will automatically discover and establish a WebSocket connection to your worker. - -Upon a successful connection from the backend, you should begin sending `stateReport` messages as heartbeats. - -## 2. Communication Overview - -Communication is bidirectional and asynchronous. All messages are JSON objects with a `type` field that indicates the message's purpose, and an optional `payload` field containing the data. - -- **Worker -> Backend:** You will send messages to the backend to report status, forward detection events, or request changes to session data. -- **Backend -> Worker:** The backend will send commands to you to manage camera subscriptions. - -## 3. Dynamic Configuration via MPTA File - -To enable modularity and dynamic configuration, the backend will send you a URL to a `.mpta` file when it issues a `subscribe` command. This file is a renamed `.zip` archive that contains everything your worker needs to perform its task. - -**Your worker is responsible for:** - -1. Fetching this file from the provided URL. -2. Extracting its contents. -3. Interpreting the contents to configure its internal pipeline. - -**The contents of the `.mpta` file are entirely up to the user who configures the model in the CMS.** This allows for maximum flexibility. For example, the archive could contain: - -- AI/ML Models: Pre-trained models for libraries like TensorFlow, PyTorch, or ONNX. -- Configuration Files: A `config.json` or `pipeline.yaml` that defines a sequence of operations, specifies model paths, or sets detection thresholds. -- Scripts: Custom Python scripts for pre-processing or post-processing. -- API Integration Details: A JSON file with endpoint information and credentials for interacting with third-party detection services. - -Essentially, the `.mpta` file is a self-contained package that tells your worker *how* to process the video stream for a given subscription. - -## 4. Messages from Worker to Backend - -These are the messages your worker is expected to send to the backend. - -### 4.1. State Report (Heartbeat) - -This message is crucial for the backend to monitor your worker's health and status, including GPU usage. - -- **Type:** `stateReport` -- **When to Send:** Periodically (e.g., every 2 seconds) after a connection is established. - -**Payload:** - -```json -{ - "type": "stateReport", - "cpuUsage": 75.5, - "memoryUsage": 40.2, - "gpuUsage": 60.0, - "gpuMemoryUsage": 25.1, - "cameraConnections": [ - { - "subscriptionIdentifier": "display-001;cam-001", - "modelId": 101, - "modelName": "General Object Detection", - "online": true, - "cropX1": 100, - "cropY1": 200, - "cropX2": 300, - "cropY2": 400 - } - ] -} -``` - -> **Note:** -> -> - `cropX1`, `cropY1`, `cropX2`, `cropY2` (optional, integer) should be included in each camera connection to indicate the crop coordinates for that subscription. - -### 4.2. Image Detection - -Sent when the worker detects a relevant object. The `detection` object should be flat and contain key-value pairs corresponding to the detected attributes. - -- **Type:** `imageDetection` - -**Payload Example:** - -```json -{ - "type": "imageDetection", - "subscriptionIdentifier": "display-001;cam-001", - "timestamp": "2025-07-14T12:34:56.789Z", - "data": { - "detection": { - "carModel": "Civic", - "carBrand": "Honda", - "carYear": 2023, - "bodyType": "Sedan", - "licensePlateText": "ABCD1234", - "licensePlateConfidence": 0.95 - }, - "modelId": 101, - "modelName": "US-LPR-and-Vehicle-ID" - } -} -``` - -### 4.3. Patch Session - -> **Note:** Patch messages are only used when the worker can't keep up and needs to retroactively send detections. Normally, detections should be sent in real-time using `imageDetection` messages. Use `patchSession` only to update session data after the fact. - -Allows the worker to request a modification to an active session's data. The `data` payload must be a partial object of the `DisplayPersistentData` structure. - -- **Type:** `patchSession` - -**Payload Example:** - -```json -{ - "type": "patchSession", - "sessionId": 12345, - "data": { - "currentCar": { - "carModel": "Civic", - "carBrand": "Honda", - "licensePlateText": "ABCD1234" - } - } -} -``` - -The backend will respond with a `patchSessionResult` command. - -#### `DisplayPersistentData` Structure - -The `data` object in the `patchSession` message is merged with the existing `DisplayPersistentData` on the backend. Here is its structure: - -```typescript -interface DisplayPersistentData { - progressionStage: "welcome" | "car_fueling" | "car_waitpayment" | "car_postpayment" | null; - qrCode: string | null; - adsPlayback: { - playlistSlotOrder: number; // The 'order' of the current slot - adsId: number | null; - adsUrl: string | null; - } | null; - currentCar: { - carModel?: string; - carBrand?: string; - carYear?: number; - bodyType?: string; - licensePlateText?: string; - licensePlateType?: string; - } | null; - fuelPump: { /* FuelPumpData structure */ } | null; - weatherData: { /* WeatherResponse structure */ } | null; - sessionId: number | null; -} -``` - -#### Patching Behavior - -- The patch is a **deep merge**. -- **`undefined`** values are ignored. -- **`null`** values will set the corresponding field to `null`. -- Nested objects are merged recursively. - -## 5. Commands from Backend to Worker - -These are the commands your worker will receive from the backend. - -### 5.1. Subscribe to Camera - -Instructs the worker to process a camera's RTSP stream using the configuration from the specified `.mpta` file. - -- **Type:** `subscribe` - -**Payload:** - -```json -{ - "type": "subscribe", - "payload": { - "subscriptionIdentifier": "display-001;cam-002", - "rtspUrl": "rtsp://user:pass@host:port/stream", - "snapshotUrl": "http://go2rtc/snapshot/1", - "snapshotInterval": 5000, - "modelUrl": "http://storage/models/us-lpr.mpta", - "modelName": "US-LPR-and-Vehicle-ID", - "modelId": 102, - "cropX1": 100, - "cropY1": 200, - "cropX2": 300, - "cropY2": 400 - } -} -``` - -> **Note:** -> -> - `cropX1`, `cropY1`, `cropX2`, `cropY2` (optional, integer) specify the crop coordinates for the camera stream. These values are configured per display and passed in the subscription payload. If not provided, the worker should process the full frame. -> -> **Important:** -> If multiple displays are bound to the same camera, your worker must ensure that only **one stream** is opened per camera. When you receive multiple subscriptions for the same camera (with different `subscriptionIdentifier` values), you should: -> -> - Open the RTSP stream **once** for that camera if using RTSP. -> - Capture each snapshot only once per cycle, and reuse it for all display subscriptions sharing that camera. -> - Capture each frame/image only once per cycle. -> - Reuse the same captured image and snapshot for all display subscriptions that share the camera, processing and routing detection results separately for each display as needed. -> This avoids unnecessary load and bandwidth usage, and ensures consistent detection results and snapshots across all displays sharing the same camera. - -### 5.2. Unsubscribe from Camera - -Instructs the worker to stop processing a camera's stream. - -- **Type:** `unsubscribe` - -**Payload:** - -```json -{ - "type": "unsubscribe", - "payload": { - "subscriptionIdentifier": "display-001;cam-002" - } -} -``` - -### 5.3. Request State - -Direct request for the worker's current state. Respond with a `stateReport` message. - -- **Type:** `requestState` - -**Payload:** - -```json -{ - "type": "requestState" -} -``` - -### 5.4. Patch Session Result - -Backend's response to a `patchSession` message. - -- **Type:** `patchSessionResult` - -**Payload:** - -```json -{ - "type": "patchSessionResult", - "payload": { - "sessionId": 12345, - "success": true, - "message": "Session updated successfully." - } -} -``` - -### 5.5. Set Session ID - -Allows the backend to instruct the worker to associate a session ID with a subscription. This is useful for linking detection events to a specific session. The session ID can be `null` to indicate no active session. - -- **Type:** `setSessionId` - -**Payload:** - -```json -{ - "type": "setSessionId", - "payload": { - "displayIdentifier": "display-001", - "sessionId": 12345 - } -} -``` - -Or to clear the session: - -```json -{ - "type": "setSessionId", - "payload": { - "displayIdentifier": "display-001", - "sessionId": null - } -} -``` - -> **Note:** -> -> - The worker should store the session ID for the given subscription and use it in subsequent detection or patch messages as appropriate. If `sessionId` is `null`, the worker should treat the subscription as having no active session. - -## Subscription Identifier Format - -The `subscriptionIdentifier` used in all messages is constructed as: - -``` -displayIdentifier;cameraIdentifier -``` - -This uniquely identifies a camera subscription for a specific display. - -### Session ID Association - -When the backend sends a `setSessionId` command, it will only provide the `displayIdentifier` (not the full `subscriptionIdentifier`). - -**Worker Responsibility:** - -- The worker must match the `displayIdentifier` to all active subscriptions for that display (i.e., all `subscriptionIdentifier` values that start with `displayIdentifier;`). -- The worker should set or clear the session ID for all matching subscriptions. - -## 6. Example Communication Log - -This section shows a typical sequence of messages between the backend and the worker. Patch messages are not included, as they are only used when the worker cannot keep up. - -> **Note:** Unsubscribe is triggered when a user removes a camera or when the node is too heavily loaded and needs rebalancing. - -1. **Connection Established** & **Heartbeat** - * **Worker -> Backend** - ```json - { - "type": "stateReport", - "cpuUsage": 70.2, - "memoryUsage": 38.1, - "gpuUsage": 55.0, - "gpuMemoryUsage": 20.0, - "cameraConnections": [] - } - ``` -2. **Backend Subscribes Camera** - * **Backend -> Worker** - ```json - { - "type": "subscribe", - "payload": { - "subscriptionIdentifier": "display-001;entry-cam-01", - "rtspUrl": "rtsp://192.168.1.100/stream1", - "modelUrl": "http://storage/models/vehicle-id.mpta", - "modelName": "Vehicle Identification", - "modelId": 201 - } - } - ``` -3. **Worker Acknowledges in Heartbeat** - * **Worker -> Backend** - ```json - { - "type": "stateReport", - "cpuUsage": 72.5, - "memoryUsage": 39.0, - "gpuUsage": 57.0, - "gpuMemoryUsage": 21.0, - "cameraConnections": [ - { - "subscriptionIdentifier": "display-001;entry-cam-01", - "modelId": 201, - "modelName": "Vehicle Identification", - "online": true - } - ] - } - ``` -4. **Worker Detects a Car** - * **Worker -> Backend** - ```json - { - "type": "imageDetection", - "subscriptionIdentifier": "display-001;entry-cam-01", - "timestamp": "2025-07-15T10:00:00.000Z", - "data": { - "detection": { - "carBrand": "Honda", - "carModel": "CR-V", - "bodyType": "SUV", - "licensePlateText": "GEMINI-AI", - "licensePlateConfidence": 0.98 - }, - "modelId": 201, - "modelName": "Vehicle Identification" - } - } - ``` - * **Worker -> Backend** - ```json - { - "type": "imageDetection", - "subscriptionIdentifier": "display-001;entry-cam-01", - "timestamp": "2025-07-15T10:00:01.000Z", - "data": { - "detection": { - "carBrand": "Toyota", - "carModel": "Corolla", - "bodyType": "Sedan", - "licensePlateText": "CMS-1234", - "licensePlateConfidence": 0.97 - }, - "modelId": 201, - "modelName": "Vehicle Identification" - } - } - ``` - * **Worker -> Backend** - ```json - { - "type": "imageDetection", - "subscriptionIdentifier": "display-001;entry-cam-01", - "timestamp": "2025-07-15T10:00:02.000Z", - "data": { - "detection": { - "carBrand": "Ford", - "carModel": "Focus", - "bodyType": "Hatchback", - "licensePlateText": "CMS-5678", - "licensePlateConfidence": 0.96 - }, - "modelId": 201, - "modelName": "Vehicle Identification" - } - } - ``` -5. **Backend Unsubscribes Camera** - * **Backend -> Worker** - ```json - { - "type": "unsubscribe", - "payload": { - "subscriptionIdentifier": "display-001;entry-cam-01" - } - } - ``` -6. **Worker Acknowledges Unsubscription** - * **Worker -> Backend** - ```json - { - "type": "stateReport", - "cpuUsage": 68.0, - "memoryUsage": 37.0, - "gpuUsage": 50.0, - "gpuMemoryUsage": 18.0, - "cameraConnections": [] - } - ``` -## 7. HTTP API: Image Retrieval - -In addition to the WebSocket protocol, the worker exposes an HTTP endpoint for retrieving the latest image frame from a camera. - -### Endpoint - -``` -GET /camera/{camera_id}/image -``` - -- **`camera_id`**: The full `subscriptionIdentifier` (e.g., `display-001;cam-001`). - -### Response - -- **Success (200):** Returns the latest JPEG image from the camera stream. - - `Content-Type: image/jpeg` - - Binary JPEG data. - -- **Error (404):** If the camera is not found or no frame is available. - - JSON error response. - -- **Error (500):** Internal server error. - -### Example Request - -``` -GET /camera/display-001;cam-001/image -``` - -### Example Response - -- **Headers:** - ``` - Content-Type: image/jpeg - ``` -- **Body:** Binary JPEG image. - -### Notes - -- The endpoint returns the most recent frame available for the specified camera subscription. -- If multiple displays share the same camera, each subscription has its own buffer; the endpoint uses the buffer for the given `camera_id`. -- This API is useful for debugging, monitoring, or integrating with external systems that require direct image access.