visionvision.detect

Object Detect

Run an object detection model on the current screen.

Video tutorial

04

Watch how to configure and run the vision.detect node in a real LaiCai Flow workflow.

Node tutorial: Object Detect

Watch on YouTube

Inputs

5

InputValue typeRequirementAccepted sources

modelIdstringRequiredliteral

All ClassesallClassesWhether every class from the selected model is allowed. Set false to use classNames; false with an empty classNames array selects no classes.boolOptionalliteral

ClassesclassNamesClass allowlist from the selected model when allClasses is false. An empty array then selects no classes.arrayOptionalliteral

minScorenumberRequiredliteral

ROIroiDetection region. Value type: screen-ratio rect {x, y, width, height}; 0..1 maps to the device frame. Smaller ROI can reduce false positives and latency.rectRequiredliteral

Outputs

9

Output pathValue typeCoordinate space

data.bestobject—

data.best.scorenumber—

data.best.modelIdstring—

data.best.classNamestring—

data.best.centerpointscreenRatio

data.best.center.xnumberscreenRatio

data.best.center.ynumberscreenRatio

data.best.rectrectscreenRatio

data.itemsarray—

Runtime support

DesktopSupported

Android AgentSupported

Model compatibility and technical requirements

An ONNX extension alone does not make a model compatible. LaiCai must also understand the model input, output tensor, box coordinates, confidence scores, and class order.

Train a custom detection model

Model file

Use a standard YOLO object-detection model exported to ONNX. PyTorch .pt files, hosted API model IDs, and classification, segmentation, pose, or rotated-box models cannot be imported directly.

.onnxtype: yolo

Input tensor

The model needs one NCHW RGB input with three channels, float32 or float16 values normalized to 0–1. LaiCai preserves aspect ratio with letterbox padding of RGB 114; unresolved dynamic sizes fall back to 640 × 640.

[1, 3, H, W]float32 / float16

Class mapping

Class IDs must start at 0, stay contiguous, and follow the training order. id2label and label2id must agree, and the node className must exactly match a configured label.

id2labellabel2idclassName

Model size and speed

Start with a Nano or Small model for continuous detection. Medium and larger models may improve accuracy but use more memory and run more slowly. Measure the exported model on the actual desktop and Android device.

n / s / m / l / x

Supported output tensors

The model must return exactly one output tensor in one of these layouts. For raw YOLO output, LaiCai applies confidence filtering and NMS.

Output formatAccepted shapeMeaning

raw_yolo_no_objectness[1, 4 + nc, N] / [1, N, 4 + nc]Boxes and class scores without a separate objectness value.

raw_yolo_with_objectness[1, 5 + nc, N] / [1, N, 5 + nc]Boxes, objectness, and class scores.

postprocessed_xyxy[1, N, 6] / [N, 6]Final x1, y1, x2, y2, score, and class ID rows after NMS.

Check before importing

The file is an ONNX object-detection model, not a .pt file or a hosted API identifier.
There is one NCHW RGB input and one supported output tensor.
Input and output use float32 or float16, not INT8 or UINT8 quantization.
The class order and output format are known; single-class models declare outputFormat explicitly.
The model folder contains one .onnx file, or config.json identifies it with modelFile.
Run a current-frame test on desktop, then verify the same model separately on Android Agent.

Example config.json

The labels and outputFormat below are examples. Replace them with the exact class order and tensor format used by your exported model.

{
  "id": "game-objects",
  "name": "Game Objects",
  "type": "yolo",
  "modelFile": "model.onnx",
  "outputFormat": "raw_yolo_no_objectness",
  "id2label": {
    "0": "ore",
    "1": "tree"
  },
  "label2id": {
    "ore": 0,
    "tree": 1
  }
}