基于凌蒙派开发板的FastDeploy适配

FastDeploy简介

FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具， 支持云边端部署。提供超过 160+Text，Vision， Speech和跨模态模型开箱即用的部署体验，并实现端到端的推理性能优化。包括 物体检测、字符识别（OCR）、人脸、人像扣图、多目标跟踪系统、NLP、Stable Difussion文图生成、TTS 等几十种任务场景，满足开发者多场景、多硬件、多平台的产业部署需求。

准备工作本文的FastDeploy适配过程需要准备如下：·凌蒙派-RK3568开发板（即需FastDeploy适配的设备终端）·Ubuntu（即建立于虚拟机的Linux编译环境）目前，我已将FastDeploy适配到凌蒙派开发板上，可用于目标检测、人脸检测、人脸识别、人脸对齐、图像分割、OCR等领域，这将大大提高凌蒙派开发板在边缘计算方面的能力。

编译步骤

我们推荐在PC上进行交叉编译（即在Ubuntu进行交叉编译）。

git clone https://github.com/PaddlePaddle/FastDeploy.gitcd FastDeploy# 如果您使用的是develop分支输入以下命令git checkout developmkdir build && cd buildcmake .. -DCMAKE_C_COMPILER=/home/zbc/opt/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc \-DCMAKE_CXX_COMPILER=/home/zbc/opt/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ \-DCMAKE_TOOLCHAIN_FILE=./../cmake/toolchain.cmake \-DTARGET_ABI=arm64 \-DENABLE_ORT_BACKEND=OFF \-DENABLE_RKNPU2_BACKEND=ON \-DENABLE_VISION=ON \-DRKNN2_TARGET_SOC=RK356X \-DCMAKE_INSTALL_PREFIX=${PWD}/fastdeploy-0.0.0make -j12make install

目标检测模型速度表

为了方便大家选择最适合自己的模型，我们选取了目前最流行的几个模型，并整理了模型速度表供大家快速浏览。以下测试速度均为端到端的速度。

Demo演示

FastDeploy提供了统一的接口，可以快速的切换模型，这里以Yolov5为例子，展示如何在凌蒙派RK3568上进行目标检测。

编写代码

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.//// Licensed under the Apache License, Version 2.0 (the "License");// you may not use this file except in compliance with the License.// You may obtain a copy of the License at//// http://www.apache.org/licenses/LICENSE-2.0//// Unless required by applicable law or agreed to in writing, software// distributed under the License is distributed on an "AS IS" BASIS,// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.// See the License for the specific language governing permissions and// limitations under the License.#include "fastdeploy/vision.h"void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {auto option = fastdeploy::RuntimeOption();option.UseRKNPU2();auto format = fastdeploy::RKNN;auto model = fastdeploy::RKYOLOV5(model_file, option,format);auto im = cv::imread(image_file);fastdeploy::DetectionResult res;fastdeploy::TimeCounter tc;tc.Start();if (!model.Predict(im, &res)) {std::cerr << "Failed to predict." << std::endl;return;}auto vis_im = fastdeploy::VisDetection(im, res,0.5);tc.End();tc.PrintInfo("RKYOLOV5 in RKNN");std::cout << res.Str() << std::endl;cv::imwrite("vis_result.jpg", vis_im);std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;}int main(int argc, char* argv[]) {if (argc < 3) {std::cout<< "Usage: infer_demo path/to/model_dir path/to/image run_option, ""e.g ./infer_model ./picodet_model_dir ./test.jpeg"<< std::endl;return -1;}RKNPU2Infer(argv[1], argv[2]);return 0;}

编译代码

# 编译mkdir buildcd buildcmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../fastdeploy-0.0.0make -j4# 下载图片wgethttps://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg# 运行./infer_rkyolov5 ../Model/yolov5-s-relu/yolov5s_relu_tk2_RK356X_i8.rknn./000000014439.jpg

展示结果

输入图片

输出图片