Skip to content

Run Ultralytics YOLO on Raspberry Pi with OpenVINO

OpenVINO provides a repeatable deployment layer for running Ultralytics YOLO models on the Arm64 CPU in Raspberry Pi 4 and Raspberry Pi 5. It does not require an external NPU, making it a useful baseline for prototypes, low-frame-rate detection, and systems where cost or hardware availability matters more than maximum throughput.

This guide installs the official Arm64 Python wheels, exports a YOLO model to OpenVINO IR, enables compiled-model caching, runs image and camera inference, and turns the pipeline into a maintainable service.

When OpenVINO is the right choice

Choose this path when:

  • You want to use the Pi CPU without a Coral or Hailo accelerator.
  • You need a standard model-conversion and runtime workflow.
  • Startup time and reproducible deployment matter.
  • The workload can tolerate the performance of a CPU-only pipeline.
  • You may later deploy the same high-level application on other OpenVINO targets.

Raspberry Pi 5 has considerably more CPU headroom than Raspberry Pi 4, but both should use a 64-bit OS. Cooling and power are part of the design for sustained inference.

1. Prepare Raspberry Pi OS

Use current 64-bit Raspberry Pi OS on Raspberry Pi 4 or 5:

1
2
uname -m
python3 --version

Expect aarch64. Update the system and install runtime libraries:

1
2
3
sudo apt update
sudo apt full-upgrade -y
sudo apt install -y python3-venv python3-pip git libglib2.0-0 libgl1

Create a clean virtual environment:

1
2
3
4
python3 -m venv ~/venvs/ov-rpi
source ~/venvs/ov-rpi/bin/activate
python -m pip install --upgrade pip
python -m pip install openvino onnx ultralytics

Verify that OpenVINO sees the CPU plugin:

1
python -c "from openvino import Core; print(Core().available_devices)"

The output should include CPU.

2. Export a small YOLO model

Start with the smallest nano model supported by your installed Ultralytics release. The official 2026 workflow demonstrates YOLO26:

1
2
3
4
5
# export_model.py
from ultralytics import YOLO

model = YOLO("yolo26n.pt")
model.export(format="openvino", imgsz=640)

Run the export:

1
2
python export_model.py
find . -maxdepth 2 -name '*.xml' -o -name '*.bin'

The generated directory contains OpenVINO IR files. Validate the plain export before experimenting with half, int8, dynamic shapes, or embedded NMS.

INT8 expectations on Arm

Do not assume that an INT8 export guarantees faster inference on Raspberry Pi. OpenVINO documents quantised execution on Arm as simulation through floating-point execution for relevant operations. Benchmark the exact release, model, and input shape.

3. Test inference with Ultralytics

Run detection on an image:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# detect_image.py
from ultralytics import YOLO

model = YOLO("yolo26n_openvino_model/")
results = model.predict(
    source="test.jpg",
    imgsz=640,
    conf=0.35,
    device="cpu",
    save=True,
)

for result in results:
    print(result.boxes)

1
python detect_image.py

If memory use or latency is too high, try imgsz=416 or imgsz=320. Changing image size affects small-object accuracy, so validate with representative images rather than a single demo frame.

4. Use OpenVINO Runtime directly

Direct runtime control is useful when you need deterministic startup, caching, and service-level instrumentation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# verify_runtime.py
import openvino as ov
import openvino.properties as props

core = ov.Core()
print("Available devices:", core.available_devices)

core.set_property({props.cache_dir: "./ov_cache"})
model = core.read_model("yolo26n_openvino_model/yolo26n.xml")
compiled = core.compile_model(model, "CPU")

print("Inputs:", compiled.inputs)
print("Outputs:", compiled.outputs)
print("Execution devices:", compiled.get_property("EXECUTION_DEVICES"))

1
2
python verify_runtime.py
du -sh ov_cache

Compiled-model caching reduces repeated startup work after the first run. It is especially valuable for a service that restarts after power loss or automatic updates.

5. Run a camera pipeline

First confirm that the camera works:

1
rpicam-jpeg -o test.jpg

For Python applications, use Picamera2 installed from Raspberry Pi OS:

1
sudo apt install -y python3-picamera2 python3-opencv

Because Picamera2 is installed as a system package, recreate the environment with system-package access if required:

1
2
3
4
5
deactivate
python3 -m venv --system-site-packages ~/venvs/ov-camera
source ~/venvs/ov-camera/bin/activate
python -m pip install --upgrade pip
python -m pip install openvino onnx ultralytics

A simple capture-and-detect loop:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# detect_camera.py
from picamera2 import Picamera2
from ultralytics import YOLO

camera = Picamera2()
camera.configure(
    camera.create_preview_configuration(
        main={"size": (640, 480), "format": "RGB888"}
    )
)
camera.start()

model = YOLO("yolo26n_openvino_model/")

try:
    while True:
        frame = camera.capture_array()
        results = model.predict(
            frame,
            imgsz=416,
            conf=0.35,
            device="cpu",
            verbose=False,
        )
        detections = len(results[0].boxes)
        print(f"detections={detections}")
finally:
    camera.stop()

This intentionally omits a preview window. Establish stable capture and inference first, then add display, tracking, alerts, or recording.

6. Benchmark the complete pipeline

Do not quote a frame rate without recording the conditions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import time

start = time.perf_counter()
iterations = 50

for _ in range(iterations):
    frame = camera.capture_array()
    model.predict(frame, imgsz=416, device="cpu", verbose=False)

elapsed = time.perf_counter() - start
print(f"end-to-end FPS: {iterations / elapsed:.2f}")

Record:

  • Pi model and RAM
  • Raspberry Pi OS and OpenVINO versions
  • YOLO model and input size
  • Confidence threshold and enabled tasks
  • Camera resolution
  • Cooling and CPU temperature
  • End-to-end FPS, not model-only FPS
  • First-run and warm-start latency

Monitor throttling in another terminal:

1
watch -n 1 'vcgencmd measure_temp; vcgencmd get_throttled'

7. Performance tuning

Apply one change at a time:

  1. Use a nano model.
  2. Reduce inference size from 640 to 416 or 320.
  3. Process every second or third camera frame.
  4. Keep capture resolution independent from inference resolution.
  5. Enable OpenVINO's compiled-model cache.
  6. Avoid drawing boxes or encoding video inside the measured inference loop.
  7. Use active cooling on Raspberry Pi 5.

If the application requires high frame rates, multiple models, or a large amount of CPU-side application logic, compare a dedicated accelerator rather than continuing to reduce accuracy.

8. Run as a systemd service

Create /etc/systemd/system/openvino-yolo.service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
[Unit]
Description=OpenVINO YOLO Camera Service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=pi
WorkingDirectory=/home/pi/openvino-yolo
ExecStart=/home/pi/venvs/ov-camera/bin/python detect_camera.py
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Replace the username and paths, then enable it:

1
2
3
sudo systemctl daemon-reload
sudo systemctl enable --now openvino-yolo
journalctl -u openvino-yolo -f

Troubleshooting

No matching distribution found for openvino

Confirm uname -m returns aarch64, use a supported Python version, upgrade pip, and use current Raspberry Pi OS. Old 32-bit or outdated user space will not match current Arm64 wheels.

OpenVINO does not list CPU

Reinstall inside a clean environment and inspect the package:

1
2
python -m pip show openvino
python -c "import openvino; print(openvino.__version__)"

Export succeeds but inference fails

Delete the generated export and cache, then export the plain model without optional precision or dynamic-shape arguments. Confirm that model files are complete and readable.

Camera import fails in the virtual environment

Use --system-site-packages so the environment can access the APT-provided Picamera2 bindings.

Ultralytics models and software have licensing requirements. Review the current AGPL and enterprise terms before embedding YOLO in a proprietary or commercial product.