We use some essential cookies to make our website work.

We use optional cookies, as detailed in our cookie policy, to remember your settings and understand how you use our website.

15 posts • Page 1 of 1
GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 11:19 am

Its impossible to run the any of rpicam-apps hailo related software straight off the bat from a newly written rpi image 64 bit with desktop on a pi5 using the instructions given here https://www.raspberrypi.com/documentati ... ng-started
I get this error

Code: Select all

[HailoRT] [error] CHECK failed - Driver version (4.18.0) is different from library version (4.20.1)
OK so lets uninstall the default rpi-cam-apps package and clone it from the git repo and try buid it with all the hailo stuff enabled... Nope.. meson wont configure the project because it can't find hailort even though we have installed it via apt. So lets clone the hailort repo and try build and install that! Ok that compiled and installed but alas, still no joy because the compiled version of hailort is also at version 4.20.. So let checkout the v4.18 tag and recompile.

This seems to works although

Code: Select all

hailortcli fw-control identify
still give us a warning:

Code: Select all

[HailoRT] [warning] Unsupported firmware operation. Host: 4.18.0, Device: 4.20.0
Well lets try it anyway:

Code: Select all

[0:09:40.412897849] [2359] INFO Camera camera.cpp:1202 configuring streams: (0) 2028x1520-YUV420 (1) 640x640-BGR888 (2) 2028x1520-BGGR_PISP_COMP1
[0:09:40.413268682] [2362] INFO RPI pisp.cpp:1484 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx500@1a - Selected sensor format: 2028x1520-SBGGR10_1X10 - Selected CFE format: 2028x1520-PC1B
[HailoRT] [warning] Unsupported firmware operation. Host: 4.18.0, Device: 4.20.0
[HailoRT] [error] Operation 51 is not allowed when FW version in not supported. Host supported FW version is 4.18.0
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16) - Can't get device extended info
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16)
[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_UNSUPPORTED_FW_VERSION(16)
Failed to create configured infer model, status = 16
HailoRT not ready!
:x :x :x

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 11:29 am

Ok So I have now managed to downgrade the firmware by using:

Code: Select all

 sudo update-alternatives --config hailo8_fw.bin 
trying to the run the demo now give me a seg fault:

Code: Select all

merlin@pi5-ai:~ $ rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov6_inference.json
[0:01:40.801953226] [2005] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+53-29156679
[0:01:40.809499583] [2009] INFO RPI pisp.cpp:720 libpisp version v1.1.0 e7974a156008 27-01-2025 (21:50:51)
[0:01:40.829690697] [2009] INFO RPI pisp.cpp:1179 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx500@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0
Made X/EGL preview window
Postprocessing requested lores: 640x640 BGR888
Reading post processing stage "hailo_yolo_inference"
Reading post processing stage "object_detect_draw_cv"
Mode selection for 2028:1520:12:P
 SRGGB10_CSI2P,2028x1520/0 - Score: 1000
 SRGGB10_CSI2P,4056x3040/0 - Score: 1887
Stream configuration adjusted
[0:01:41.151104867] [2005] INFO Camera camera.cpp:1202 configuring streams: (0) 2028x1520-YUV420 (1) 640x640-BGR888 (2) 2028x1520-RGGB_PISP_COMP1
[0:01:41.151227052] [2009] INFO RPI pisp.cpp:1484 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx500@1a - Selected sensor format: 2028x1520-SRGGB10_1X10 - Selected CFE format: 2028x1520-PC1R
Segmentation fault

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 11:51 am

With reference to the above segfault, running the command through gdb and doing a backtrace gives me this:

Code: Select all

Thread 29 "rpicam-hello" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb9ec9b00 (LWP 2742)]
__memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:87
87	../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory.
(gdb) backtrace 
#0 __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:87
#1 0x00007fffcdf8f7bc in () at /usr/lib/aarch64-linux-gnu/hailo/tappas/post_processes/libyolo_hailortpp_post.so
#2 0x00007fffcdf98638 in filter () at /usr/lib/aarch64-linux-gnu/hailo/tappas/post_processes/libyolo_hailortpp_post.so
#3 0x00007fffd6550bd0 in YoloInference::runInference(unsigned char const*, std::vector<libcamera::Rectangle, std::allocator<libcamera::Rectangle> > const&)
 (this=0x5555555e0420, frame=0x7fffccc74000 <error: Cannot access memory at address 0x7fffccc74000>, scaler_crops=std::vector of length 2, capacity 2 = {...}) at ../post_processing_stages/hailo/hailo_yolo_inference.cpp:257
#4 0x00007fffd6550818 in YoloInference::Process(std::shared_ptr<CompletedRequest>&) (this=0x5555555e0420, completed_request=warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<CompletedRequest*, RPiCamApp::requestComplete(libcamera::Request*)::{lambda(CompletedRequest*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<CompletedRequest*, RPiCamApp::requestComplete(libcamera::Request*)::{lambda(CompletedRequest*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
 std::shared_ptr<CompletedRequest> (use count 1, weak count 0) = {...})
 at ../post_processing_stages/hailo/hailo_yolo_inference.cpp:205
#5 0x00007ffff7e90460 in operator()(CompletedRequestPtr&, std::promise<bool>) const (__closure=0x555556562d98, request=warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<CompletedRequest*, RPiCamApp::requestComplete(libcamera::Request*)::{lambda(CompletedRequest*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
warning: RTTI symbol not found for class 'std::_Sp_counted_deleter<CompletedRequest*, RPiCamApp::requestComplete(libcamera::Request*)::{lambda(CompletedRequest*)#1}, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>'
 std::shared_ptr<CompletedRequest> (use count 1, weak count 0) = {...}, promise=...)
 at ../core/post_processor.cpp:211
#6 0x00007ffff7e90fe0 in std::__invoke_impl<void, PostProcessor::Process(CompletedRequestPtr&)::<lambda(CompletedRequestPtr&, std::promise<bool>)>, std::reference_wrapper<std::shared_ptr<CompletedRequest> >, std::promise<bool> >(std::__invoke_other, struct {...} &&)
 (__f=...) at /usr/include/c++/12/bits/invoke.h:61
#7 0x00007ffff7e90f3c in std::__invoke<PostProcessor::Process(CompletedRequestPtr&)::<lambda(CompletedRequestPtr&, std::promise<bool>)>, std::reference_wrapper<std::shared_ptr<CompletedRequest> >, std::promise<bool> >(struct {...} &&) (__fn=...)
 at /usr/include/c++/12/bits/invoke.h:96
#8 0x00007ffff7e90e68 in std::thread::_Invoker<std::tuple<PostProcessor::Process(CompletedRequestPtr&)::<lambda(CompletedRequestPtr&, std::promise<bool>)>, std::reference_wrapper<std::shared_ptr<CompletedRequest> >, std::promise<bool> > >::_M_invoke<0, 1, 2>(std::_Index_tuple<0, 1, 2>) (this=0x555556562d78) at /usr/include/c++/12/bits/std_thread.h:252
#9 0x00007ffff7e90e0c in std::thread::_Invoker<std::tuple<PostProcessor::Process(CompletedRequestPtr&)::<lambda(CompletedRequestPtr&, std::promise<bool>)>, std::reference_wrapper<std::shared_ptr<CompletedRequest> >, std::promise<bool> > >::operator()(void)
 (this=0x555556562d78) at /usr/include/c++/12/bits/std_thread.h:259
#10 0x00007ffff7e90dec in std::thread::_State_impl<std::thread::_Invoker<std::tuple<PostProcessor::Process(CompletedRequestPtr&)::<lambda(CompletedRequestPtr&, std::promise<bool>)>, std::reference_wrapper<std::shared_ptr<CompletedRequest> >, std::promise<bool> > > >::_M_run(void) (this=0x555556562d70) at /usr/include/c++/12/bits/std_thread.h:210
--Type <RET> for more, q to quit, c to continue without paging--c
#11 0x00007ffff7afe9dc in () at /lib/aarch64-linux-gnu/libstdc++.so.6
#12 0x00007ffff78aee90 in start_thread (arg=0x7fffdcb48187) at ./nptl/pthread_create.c:442
#13 0x00007ffff7917b1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 691
Joined: Mon Apr 25, 2016 10:21 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 11:56 am

I've just tried starting from a fresh sd-card image built with Raspberry Pi Imager and followed the instructions in our getting started guide. This effectively amounted to running

Code: Select all

sudo apt update
sudo apt install hailo-all
Once I rebooted, everything worked as expected, including demos.

Perhaps you can clarify what base OS image you started with? Have you made any other modifications to your system packages other than installing updates via apt - particularly to the libcamera/rpicam-apps libraries?

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 12:07 pm

Hi naushir

This is exactly the same process i went through before I decided to compile rpi-cam apps myself. Exactly as you have it using rpi-imager . I am going to try again on a new card right now.

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 12:33 pm

Nope.

This is not working. The exact steps I followed were:

[*]Use the rpi imager to write a new image (It uses the Nov 2024 release of bookworm.. i.e.. the latest ?)
[*] Once I log in, I do a an update and install hailo-all

This is the contents of my bash history since logging in:

Code: Select all

sudo raspi-config 
sudo apt update
sudo apt install hailo-all
sudo reboot
(I did raspi-config to enable VNC)

I am still getting this error.

Code: Select all

[HailoRT] [error] CHECK failed - Driver version (4.18.0) is different from library version (4.20.0)
I'm assuming this is the hailo driver version mismatched with the hailo libs. If this is the case then I am not sure how you are also not getting the same error from a fresh install.

Are you also running this on a pi5 ?.

The only thing that might be unusual is that I am using the Sony IMX500 camera as the video source. But as far as i can see this is not the reason for the error.

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 691
Joined: Mon Apr 25, 2016 10:21 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 12:38 pm

This does seem very strange. Just to confirm, before you ran the installation command, did you let the system update everything on first boot? I'm running on a Pi 5, and the IMX500 sensor should definitely work.

Can you post the output of

Code: Select all

cat /etc/os-release
and

Code: Select all

uname -a
please?

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 12:47 pm

I'm not sure what you mean by letting the system update everything on the first boot. I generally wait until the system has booted up before logging in and doing the necessary commands.

Code: Select all

merlin@pi5-ai:~ $ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
And

Code: Select all

merlin@pi5-ai:~ $ uname -a
Linux pi5-ai 6.6.51+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt3 (2024年10月08日) aarch64 GNU/Linux

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 691
Joined: Mon Apr 25, 2016 10:21 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 1:01 pm

GrubbyHalo wrote:
Mon Mar 17, 2025 12:47 pm
I'm not sure what you mean by letting the system update everything on the first boot. I generally wait until the system has booted up before logging in and doing the necessary commands.
On first boot, after setting up your username/password, the OS will ask if you want it to run an update cycle to get the latest version of all the libraries and kernel packages. This can also be done manually with:

Code: Select all

sudo apt update && sudo apt full-upgrade
on the command line.
GrubbyHalo wrote:
Mon Mar 17, 2025 12:47 pm

Code: Select all

merlin@pi5-ai:~ $ uname -a
Linux pi5-ai 6.6.51+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.51-1+rpt3 (2024年10月08日) aarch64 GNU/Linux
This is different to my result:

Code: Select all

Linux pi5 6.6.74+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.74-1+rpt1 (2025年01月27日) aarch64 GNU/Linux
I'm not really sure why your kernel is not updated. Perhaps the update cycle on boot has not run on your installation? This can happen if you setup some custom settings in the Raspberry Pi Imager app.

GrubbyHalo
Posts: 46
Joined: Thu Jul 10, 2014 9:33 am

Re: The Hailo AI kit is an absolute mess

Mon Mar 17, 2025 1:02 pm

OK I finally got it working

I did not get a prompt toi= do an upgrade
I had to do a full upgrade

Code: Select all

sudo apt update && sudo apt full-upgrade
I recall doing this earlier but with the command

Code: Select all

sudo apt-get install upgrade
which, of course did not work

I'm assuming that this must have been a driver issue for the hailo.

Anyway.. thank you for your assistance

zoukimando
Posts: 27
Joined: Mon Jan 07, 2013 5:45 pm

Re: The Hailo AI kit is an absolute mess

Sat Nov 15, 2025 11:48 am

I bought the AI HAT + a few months back. Ran through a quick install and didn't get very far. Time constraints stopped me looking too hard. I came back to it yesterday, and Googling I found lots of others who seem to be having issues, and no one actually able to post a working install guide.
I thought "it must be sorted by now so I follow the guide and a fresh sd card with bookworm on and I get to "apt install hailo-all" and this error.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
hailo-all : Depends: hailort (>= 4.20.0) but it is not installable
Depends: hailo-tappas-core (>= 3.31.0) but it is not installable
Depends: rpicam-apps-hailo-postprocess (>= 1.6.0) but it is not installable
Depends: python3-hailort (>= 4.20.0) but it is not installable
E: Unable to correct problems, you have held broken packages."
I see other have the same issue, and no one is posting a solution that I have found.
Is the AI HAT+ not supported?

gordon77
Posts: 9265
Joined: Sun Aug 05, 2012 3:12 pm

Re: The Hailo AI kit is an absolute mess

Sat Nov 15, 2025 1:02 pm

I just tried.

Pi5 with original AI hat ( the one with plug in M2 hailo)
New sd card
FULL BOOKWORM 64 bit install
let it do all updates

followed https://www.raspberrypi.com/documentati ... rs/ai.html
sudo apt install hailo-all
reboot
hailortcli fw-control identify
rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov6_inference.json

all worked fine

zoukimando
Posts: 27
Joined: Mon Jan 07, 2013 5:45 pm

Re: The Hailo AI kit is an absolute mess

Sat Nov 15, 2025 1:08 pm

okay, so the 64 bit version. Thats not anywhere near the top 10 of the offerings in Raspberry Pi imager. Okay I'll give that a go.
Maybe the 64bit detail should be added to the Bookworm recommendation in the guide?

gordon77
Posts: 9265
Joined: Sun Aug 05, 2012 3:12 pm

Re: The Hailo AI kit is an absolute mess

Sat Nov 15, 2025 1:14 pm

zoukimando wrote:
Sat Nov 15, 2025 1:08 pm
okay, so the 64 bit version. Thats not anywhere near the top 10 of the offerings in Raspberry Pi imager. Okay I'll give that a go.
Maybe the 64bit detail should be added to the Bookworm recommendation in the guide?
Now called Legacy.

zoukimando
Posts: 27
Joined: Mon Jan 07, 2013 5:45 pm

15 posts • Page 1 of 1

Return to "Hailo"

AltStyle によって変換されたページ (->オリジナル) /