Author: Meng Chang
This topic presents an example about how to deploy a YOLOX model on a RISC-V development board for object detection.
The content of this topic is derived from the tutorial of LicheePi Module 4A (LPi4A) development board provided by Sipeed. A LPi4A development board uses a TH1520 chip and is equipped with a Xuantie C920 CPU. The software is HHB-onnxruntime customized by T-Head. Based on the onnxruntime project, HHB-onnxruntime supports vector instructions on C920 to optimize execution performance.
In the use case described in this topic, only RISC-V development boards are used, and no additional x86 hosts and other devices are required.
The following sections describe how to:
- Install the Python environment on LPi4A.
- Execute the source code from YOLOX repository on C920.
- Execute ONNX models with HHB-onnxruntime.
The normal open-source model deployment process:
- Configure the basic Python environment on LPi4A.
- Get the YOLOX source code and model.
- Install the Python package that YOLOX depends on.
- Run a demo using HHB-onnxruntime on LPi4A
Configure the basic Python environment
Configure the basic hardware and software
Install the development board and enter the certificate information of the root user after power-on. For more information, see Unboxing.
Make sure that you are connected to the Internet and update the apt source.
apt update
Install the software required for running the demo.
apt install wget git vim
Install the SHL library. SHL provides optimized code for RISC-V vector extension instructions on C920.
wget https://github.com/T-head-Semi/csi-nn2/releases/download/v2.4-beta.1/c920.tar.gz tar xf c920.tar.gz cp c920/lib/* /usr/lib/riscv64-linux-gnu/ -rf
Configure the Python environment
By default, Python V3.11 is installed in the system burned by LPi4A. You can run the following command to check the Python version.
python3 --version
In the following example, Python V3.11 is used. If you want to use another version of Python, specify the version that you want to use in the command when installing dependencies.
Most of the software packages that Python program software depends on can be installed by pip. You can run the following command to install pip.
apt install python3-pip
Before installing other Python packages, install the venv package that can be used to create a Python virtual environment.
apt install python3.11-venv
Create a Python virtual environment and activate it.
cd /root python3 -m venv ort source /root/ort/bin/activate
So far, the basic Python environment is created. Similar to other architectures, pure Python packages can be installed by the pip install command.
Get the YOLOX model
YOLOX is a YOLO-like object detection model with excellent performance.
You can download the source code and model from GitHub.
git clone https://github.com/Megvii-BaseDetection/YOLOX.git cd YOLOX/demo/ONNXRuntime wget https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_s.onnx
Modify the source code
To use HHB-onnxruntime to execute the model, switch to HHB-onnxruntime first. In the onnxruntime sample directory in the source code, modify the beginning content in the demo/ONNXRuntime/onnx_inference.py file and add the fourth and fifth lines of code as shown below.
#!/usr/bin/env python3 # Copyright (c) Megvii, Inc. and its affiliates. +import sys +sys.path.insert(0, "../../") import argparse import os
Use sys.path.insert to specify the search path in the code. Therefore, you do not need to install the YOLOX package from the source code.
Install dependencies
The Python ecology of the RISC-V architecture is still lacking. After the update is released in the future, the dependent packages of YOLOX can be installed directly from the requirements.txt file.
The YOLOX demo relies on multiple Python packages. Therefore, you need to download the precompiled Python packages.
git clone -b python3.11 https://github.com/zhangwm-pt/prebuilt_whl.git cd prebuilt_whl
You can run the following command in sequence to install the dependencies.
pip install numpy-1.25.0-cp311-cp311-linux_riscv64.whl pip install opencv_python-4.5.4+4cd224d-cp311-cp311-linux_riscv64.whl pip install kiwisolver-1.4.4-cp311-cp311-linux_riscv64.whl pip install Pillow-9.5.0-cp311-cp311-linux_riscv64.whl pip install matplotlib-3.7.2.dev0+gb3bd929cf0.d20230630-cp311-cp311-linux_riscv64.whl pip install pycocotools-2.0.6-cp311-cp311-linux_riscv64.whl pip3 install loguru-0.7.0-py3-none-any.whl pip3 install torch-2.0.0a0+gitc263bd4-cp311-cp311-linux_riscv64.whl pip3 install MarkupSafe-2.1.3-cp311-cp311-linux_riscv64.whl pip3 install torchvision-0.15.1a0-cp311-cp311-linux_riscv64.whl pip3 install psutil-5.9.5-cp311-abi3-linux_riscv64.whl pip3 install tqdm-4.65.0-py3-none-any.whl pip3 install tabulate-0.9.0-py3-none-any.whl
In the installation process, other pure Python dependent packages are required and pip automatically downloads them from the official sources.
Install HHB-onnxruntime
HHB-onnxruntime is a transplanted SHL backend (execution provider), and onnxruntime can be reused in SHL for high-performance optimized code on black iron CPUs.
wget https://github.com/zhangwm-pt/onnxruntime/releases/download/riscv_whl/onnxruntime-1.14.1-cp311-cp311-linux_riscv64.whl pip install onnxruntime-1.14.1-cp311-cp311-linux_riscv64.whl
Execute the demo
Execute the onnx_inference.py demo in the examples directory.
python3 onnx_inference.py -m yolox_s.onnx -i ../../assets/dog.jpg -o outdir -s 0.7 --input_shape 640,640
Parameter description:
- -m: specifies the model. In this example, yolox_s that was downloaded is specified as the model.
- -i: specifies the image. In this example, a sample image that comes with the repository is specified.
- -o: specifies the output directory as outdir. The outdir directory will be created during the execution.
- -s: Specifies the detection threshold value as 0.7. Objects with a probability lower than 0.7 will be ignored.
- –input_shape: Specifies the size of the image used for model detection.
Reference result
In this example, the following figure is used. With a threshold value of 0.7, the expected detection result of yolov5 includes a dog and a bicycle.
After the demo is executed, the result image dog.jpg is generated in the outdir directory. Each detected object is marked with a frame in the picture, and the probability is also marked. The result is as follows:
Conclusion
HHB-onnxruntime customized by T-head is used to execute the YOLOX model to complete the object detection task, and the C920 CPU on the RISC-V development board is used to quickly deploy the model.