Use OPC-UA with camera inference¶

Supported for: Instance segmentation – Picking and grading and Multi view instance segmentation - Center picking and grading

Inference results¶

The OPC-UA server provides inference results in a structured format under the CameraInference - CurrentFrame node. Below is a summary of the key elements.

Key nodes¶

• CurrentFrame: Metadata and detections for the latest frame.
• GrabTimestamp: Timestamp when the frame was captured.
• PredictTimestamp: Timestamp when the inference on the frame was completed.
• ResultsFrame: Displays "camera_frame" when no calibration is specified, meaning units are in pixels. If calibration is available, it displays "external_frame," and units match the calibrated scale, for example, meters.
• Detections: List of detected objects in the frame.

Detection details¶

Each detection in the Detections array includes:

• DetectionClass: Class of the detection.
• Confidence: Confidence score of the detection (e.g., 0.98).
• Center: Coordinates of the detection center, calculated using Principal Component Analysis (PCA).
  • If the cameras in the camera configuration selected for camera inference have been calibrated within the project, then the results are expressed in meters (based on the most recent successful calibration for that camera configuration within your project).
  • Otherwise, the results are expressed in pixel coordinates for that camera.
• RotatedBbox: Coordinates of the rotated bounding box encasing the mask in results frame units. The rotation angle (in degrees, not radians) is derived from PCA. The format is [x', y', width, height, angle]. The units are the same as for the Center field.
• Area: Area of the mask. The units are the same as for the Center field.
• View: Identifies which camera produced the detection. For a single-camera setup, this is the default camera. For multiview configurations, this is the camera position name from your camera configuration—the same names you use as views in the label, data, and test centers. See Multiview instance segmentation.

Multiview instance segmentation¶

OPC-UA uses the same output schema for Multi view instance segmentation – Center picking and grading and Instance segmentation – Picking and grading.

• Detections from all cameras are exposed in one Detections list.
• Use View to identify which camera produced each detection.
  • In single-camera configurations, View is default camera.
  • In multiview configurations, View is the configured camera position name (for example, top or side).
  • These values match the camera position names you set when you create camera configurations.
• When calibration is available, Center, RotatedBbox, and Area use the calibration of the camera named in View.

The following example uses placeholder values and shows how detections from multiple views are represented in one list.

[
  {
    "DetectionClass": "Apple",
    "Confidence": 0.95,
    "Center": [0.052, 0.031],
    "RotatedBbox": [0.048, 0.029, 0.014, 0.011, 12.4],
    "Area": 0.000154,
    "View": "top"
  },
  {
    "DetectionClass": "Apple",
    "Confidence": 0.87,
    "Center": [0.041, 0.038],
    "RotatedBbox": [0.039, 0.036, 0.016, 0.012, -8.2],
    "Area": 0.000192,
    "View": "side"
  }
]

Inference and project information¶

When starting a camera inference, by default, the inference takes over the OPC-UA outputs. That behavior can be toggled off in the setup before starting a new inference.

The CameraInference - Info node provides metadata about which inference and project is currently generating OPC-UA outputs. The following fields are relevant:

Key fields¶

• InferenceId: Unique identifier for the inference session.
• InferenceName: Name of the inference session.
• ProjectId: Unique identifier for the project on Robovision AI.
• ProjectName: Name of the project.

Accessing OPC-UA server¶

The OPC-UA server runs on TCP port 4850. For on-premises installations, the server uses the same hostname as your Robovision AI platform.

Robovision AI starts up its own OPC-UA server but does not include a client. You can use a program like UA Expert or use your own client.

Example URL format:

opc.tcp://<hostname>:4850/robovision/opcua/server

If your platform is accessed via example.com, the OPC-UA server URL would be:

opc.tcp://example.com:4850/robovision/opcua/server