本工程基于复旦微官方提供的PLINHDMI/VPU参考实现进行实现基础环境配置如下硬件平台FMQL30TAI悟净开发板完整版/LITE版传感器SDI摄像头1080P60HzICraft版本v3.36.1工程下载地址FMSH大学计划模型库地址icraft_modelzoo合集详情-来自AIBS · 魔搭社区本文所用具体工程文件如下图所示1. PLIN参考实现数据流2. 硬件互联情况FMQL30TAI悟净开发板完整版FMQL30TAI悟净开发板LITE版3. 编译yolov5模型WIN系统windows系统安装未安装情况下Icraft_Setup_v3.36.1.exeCustomOp_Setup_v3.36.1.exeIcraftMdz_ThirdParty_v0.2.exe将最新生成的yolov5s_plin_ZG.json和yolov5s_plin_ZG.raw手动拷贝至plinhdmi/vpu运行时工程相应目录4. 交叉编译runtime appwin系统下docker环境导入docker环境(base) PS C:\Users\26252\Desktop\plin_demo (base) PS C:\Users\26252\Desktop\plin_demo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (base) PS C:\Users\26252\Desktop\plin_demo docker import .\ubuntu20.04_container.tar ubuntu2004:imported sha256:2cd5e6f8d67d9b8c1560a4f1f71acfbfacb7d6e6e60e4e02d1b6d18a75b220bc (base) PS C:\Users\26252\Desktop\plin_demo docker run --nameubuntu20.04_container -it -v C:\Users\26252\Desktop\plin_demo:/mnt/docker_data ubuntu2004:imported /bin/bash rootf4d7f9b8953e:/# ls bin boot dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var rootf4d7f9b8953e:/# cd /mnt/docker_data/ rootf4d7f9b8953e:/mnt/docker_data# ls CustomOp_3.36.1_onchip.deb CustomOp_Setup_v3.36.1.exe IcraftMdzThirdParty.deb IcraftMdz_ThirdParty_v0.2.exe Icraft_3.36.1_onchip.deb Icraft_Setup_v3.36.1.exe fpai_demo_package_26040701 fpai_demo_package_26040702 ubuntu20.04_container.tar yolov5_7.0 $\346\202\237\345\207\200LITE$\347\211\210_BOOT_25122301_$\345\215\225\350\267\257PLINpHDMI($\345\244\226\347\275\256detpost) $\346\202\237\345\207\200\345\256\214\346\225\264\347\211\210_BOOT_25122301_$\345\215\225\350\267\257PLINpHDMI($\345\244\226\347\275\256detpost) rootf4d7f9b8953e:/mnt/docker_data# rootf4d7f9b8953e:/mnt/docker_data# rootf4d7f9b8953e:/mnt/docker_data# rootf4d7f9b8953e:/mnt/docker_data#安装icraft环境rootf4d7f9b8953e:/mnt/docker_data# dpkg -l | grep icraft rootf4d7f9b8953e:/mnt/docker_data# dpkg -i Icraft_3.36.1_onchip.deb Selecting previously unselected package icraft:arm64. (Reading database ... 19786 files and directories currently installed.) Preparing to unpack Icraft_3.36.1_onchip.deb ... Unpacking icraft:arm64 (3.36.1) ... Setting up icraft:arm64 (3.36.1) ... rootf4d7f9b8953e:/mnt/docker_data# dpkg -i CustomOp_3.36.1_onchip.deb Selecting previously unselected package customop:arm64. (Reading database ... 20227 files and directories currently installed.) Preparing to unpack CustomOp_3.36.1_onchip.deb ... Unpacking customop:arm64 (3.36.1) ... Setting up customop:arm64 (3.36.1) ... rootf4d7f9b8953e:/mnt/docker_data# dpkg -i IcraftMdzThirdParty.deb Selecting previously unselected package icraftmdzthirdparty:arm64. (Reading database ... 20246 files and directories currently installed.) Preparing to unpack IcraftMdzThirdParty.deb ... Unpacking icraftmdzthirdparty:arm64 (0.1.1) ... Setting up icraftmdzthirdparty:arm64 (0.1.1) ... rootf4d7f9b8953e:/mnt/docker_data# rootf4d7f9b8953e:/mnt/docker_data# dpkg -l | grep icraft ii icraft:arm64 3.36.1 arm64 This is Icraft for arm64 ii icraftmdzthirdparty:arm64 0.1.1 arm64 This is IcraftMdzThirdParty for arm64 rootf4d7f9b8953e:/mnt/docker_data#交叉编译PLINHDMI应用程序交叉编译PLINVPU应用程序修改PLINHDMI应用程序yaml配置文件icraft-show确认算子情况5. 板载系统安装Icraft环境6. 运行PLINHDMI应用程序注意root权限根据需要修改yaml配置文件运行程序运行效果视频效果30tai_单路plin_hdmi显示7. 运行PLINVPU应用程序修改配置yaml文件运行程序输出结果打开vpu压缩结果视频效果30tai_单路plin_vpu压缩保存8. 悟净LITE开发板运行PLINHDMI效果两款开发板只有某个管脚约束不同其余完全兼容所以把悟净完整开发板的SD卡系统里保存了上述几个章节的应用程序FAT分区的BOOT.bin替换成LITE版的运行相同程序效果完全一致9. 升级Icraft为v3.39.0版本对于高版本的icraft器支持的算子和模型要更丰富如果针对yolo系列模型3.36和3.39无差异3.39并未发布参考实现但可以复用3.36的runtimeapp代码和位流但win系统版本、交叉编译环境、板载环境三个需要配套升级。9.1 win系统升级icraft3.39卸载icraft3.36安装icraft3.39编译yolov5icraft-show可视化模型结构发现算子op id发生变化将新生成的json和raw上传至开发板系统相关路径下9.2 交叉编译环境升级icraft3.39卸载3.36安装3.39交叉编译PLINHDMI应用程序将交叉编译生成的应用程序上传至开发板相应目录下9.3 板载系统升级icraft3.39卸载 icraft3.36安装 icraft3.399.4 运行PLINHDMI应用程序运行程序报错网络OP ID 发生变化修改配置yaml文件按照9.1节的实现情况进行修改运行程序效果正常10. 固定视频输入HDMI转SDI_IN对于某些特殊场景的实时视频采集与智能分析由于开发场景和应用场景之间的差异摄像头在开发环境无法捕捉到应用场景目标。在此背景下可以采用将视频或图片序列在一台电脑进行播放通过HDMI输出画面进入HDMI转SDI模块接入开发板。该模式也可用于调试和Debug控制变量便于定位分析尤其是精度问题。实时智能分析效果