Re: NUMA Testing
I just wanted to say that I ran this command today and it partially broke my install on my 2GB Pi4. Numerous emulator cores in retropie no longer open and my Secondary Memory Interface gamepad driver that allocates uncached memory no longer functions.
Do you have any idea how to revert back? This didn't happen when I used sudo rpi-update to enable NUMA functionality.
Re: NUMA Testing
That's a full system update, not just a kernel update.GVX64 wrote: ↑Fri Dec 20, 2024 10:26 pmI just wanted to say that I ran this command today and it partially broke my install on my 2GB Pi4. Numerous emulator cores in retropie no longer open and my Secondary Memory Interface gamepad driver that allocates uncached memory no longer functions.
Do you have any idea how to revert back? This didn't happen when I used sudo rpi-update to enable NUMA functionality.
You could try running the retropie setup and update, bringing in any new retropie updates and hope they're fixed.
To go back, you use your backup ;-)
Which if like me you don't have.
You'd need to find and revert whatever update actually broke it.
Re: NUMA Testing
Thanks for the reply.bensimmo wrote: ↑Sat Dec 21, 2024 5:45 amThat's a full system update, not just a kernel update.GVX64 wrote: ↑Fri Dec 20, 2024 10:26 pmI just wanted to say that I ran this command today and it partially broke my install on my 2GB Pi4. Numerous emulator cores in retropie no longer open and my Secondary Memory Interface gamepad driver that allocates uncached memory no longer functions.
Do you have any idea how to revert back? This didn't happen when I used sudo rpi-update to enable NUMA functionality.
You could try running the retropie setup and update, bringing in any new retropie updates and hope they're fixed.
To go back, you use your backup ;-)
Which if like me you don't have.
You'd need to find and revert whatever update actually broke it.
I tried to revert by falling back to an earlier kernel. That didn't work so I then tried uninstalling the newly installed content listed in /var/log/apt/history.log but that just messed things up even more and the Secondary Memory Interface code that I use for my gamepad driver still could not allocate uncached memory.
You're right, the only option remaining is to now fall-back to my last SD card backup image, which is three months old. I am imaging a new SD card now. Oh well, the painful lesson learned is to be more proactive about making backup images BEFORE experimenting with anything new and to just generally be more frequent about making images. This is such an important thing with Linux systems where there is no system restore point.
- dom
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 8472
- Joined: Wed Aug 17, 2011 7:41 pm
Re: NUMA Testing
Had you manually increased cma size? Might be worth trying with that setting disabled.GVX64 wrote: ↑Fri Dec 20, 2024 10:26 pmI just wanted to say that I ran this command today and it partially broke my install on my 2GB Pi4. Numerous emulator cores in retropie no longer open and my Secondary Memory Interface gamepad driver that allocates uncached memory no longer functions.
Do you have any idea how to revert back? This didn't happen when I used sudo rpi-update to enable NUMA functionality.
Post output of "raspinfo | pastebinit" to see if there's any clues in the debug output.
- sf53534535345
- Posts: 1
- Joined: Thu Jan 09, 2025 9:16 pm
Re: NUMA Testing
What about difference in power consumption? Increased?
Having a Pi4 B 4GB and a Pi 4B 8GB I want to test this hack on...
Having a Pi4 B 4GB and a Pi 4B 8GB I want to test this hack on...
Re: NUMA Testing
As there is no voltage change, the relationship between power to rate of work done should be proportional. A computation running 10 percent faster should take 10 percent more power and finish 10 percent sooner.sf53534535345 wrote: ↑Thu Jan 09, 2025 9:19 pmWhat about difference in power consumption? Increased?
Having a Pi4 B 4GB and a Pi 4B 8GB I want to test this hack on...
The above implies there would be no effect on performance/power type efficiency; however, if one adds in the constant power draw of a monitor and other peripherals, then efficiency goes up as the total time to finish the work goes down.
Said another way, a car gets zero miles per gallon when idle at a red light no matter how low the fuel consumption is at idle. At the same time, turning a computer off and on again incurs additional wear and power consumption while rebooting.
Re: NUMA Testing
Hack?, it's part of the main kernel.sf53534535345 wrote: ↑Thu Jan 09, 2025 9:19 pmWhat about difference in power consumption? Increased?
Having a Pi4 B 4GB and a Pi 4B 8GB I want to test this hack on...
You're probably already using it if your Pi is using bookworm and is up to date.
Last edited by bensimmo on Fri Jan 10, 2025 12:50 pm, edited 2 times in total.
- timg236
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 1938
- Joined: Thu Jun 21, 2018 4:30 pm
Re: NUMA Testing
NUMA is automatically enabled in the 'latest' APT firmware. It will soon be in the default / automatic update.
Re: NUMA Testing
Sorry for not following up on this yet. I've now revisited this issue, hoping it would disappear with 6.12 or something, but it's still happening, but only on the Pi5 8GB model as far as I can tell. I've now prepared a small test case here. Needs 64bit Bookworm and a 4K display and might need additional libs:
Code: Select all
apt install ffmpeg libfreetype6 libevent-2.1-7 libgbm1 libegl-mesa0 libao4
systemctl stop lightdm
Last edited by dividuum on Wed Feb 26, 2025 3:14 pm, edited 1 time in total.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted
- dom
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 8472
- Joined: Wed Aug 17, 2011 7:41 pm
Re: NUMA Testing
Using rpi-update bootloader and kernel:
I ran your test and a I see kylie loop for a few seconds on repeat. No obvious flicker.
I'm running on a Pi5 D0 8G. I had booted to console.
Code: Select all
$ vcgencmd bootloader_version && uname -a && rpi-eeprom-config && cat /proc/cmdline && kmsprint
2025年02月12日 10:51:52
version f788aab60121515fe59c37c899186831e6c19585 (release)
timestamp 1739357512
update-time 1740574742
capabilities 0x0000007f
Linux m2hat 6.12.16-v8-16k+ #1859 SMP PREEMPT Mon Feb 24 13:15:44 GMT 2025 aarch64 GNU/Linux
[all]
BOOT_UART=1
BOOT_ORDER=0xf461
NET_INSTALL_AT_POWER_ON=1
reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe cgroup_disable=memory numa_policy=interleave numa=fake=8 system_heap.max_order=0 smsc95xx.macaddr=B8:27:EB:18:1F:17 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000 console=ttyAMA10,115200 console=tty1 root=PARTUUID=5f992c6d-02 rootfstype=ext4 fsck.repair=yes rootwait splash plymouth.ignore-serial-consoles cfg80211.ieee80211_regdom=GB
Connector 0 (33) HDMI-A-1 (connected)
Encoder 0 (32) TMDS
Crtc 2 (92) [email protected] 594.000 3840/176/88/296/+ 2160/8/10/72/+ 60 (60.00) P|D
Plane 4 (105) fb-id: 683 (crtcs: 1 2 3) 0,0 224x36 -> 3362,2068 448x72 (XR24 AR24 AB24 XB24 RG16 BG16 AR15 XR15 RG24 BG24 YU16 YV16 YU24 YV24 YU12 YV12 NV12 NV21 NV16 NV61 P030 XR30 AR30 AB30 XB30 RGB8 BGR8 XR12 AR12 XB12 AB12 BX12 BA12 RX12 RA12)
FB 683 224x36 AR24
Plane 5 (116) fb-id: 685 (crtcs: 1 2 3) 0,0 3840x2160 -> 0,0 3840x2160 (XR24 AR24 AB24 XB24 RG16 BG16 AR15 XR15 RG24 BG24 YU16 YV16 YU24 YV24 YU12 YV12 NV12 NV21 NV16 NV61 P030 XR30 AR30 AB30 XB30 RGB8 BGR8 XR12 AR12 XB12 AB12 BX12 BA12 RX12 RA12)
FB 685 3840x2160 P030
Connector 1 (42) HDMI-A-2 (disconnected)
Encoder 1 (41) TMDS
I'm running on a Pi5 D0 8G. I had booted to console.
Re: NUMA Testing
Ok. I tried to get to that configuration as close as possible:
Code: Select all
pi@raspberrypi:~ $ vcgencmd bootloader_version && uname -a && rpi-eeprom-config && cat /proc/cmdline && kmsprint
2025年02月12日 10:51:52
version f788aab60121515fe59c37c899186831e6c19585 (release)
timestamp 1739357512
update-time 1740577036
capabilities 0x0000007f
Linux raspberrypi 6.12.16-v8-16k+ #1859 SMP PREEMPT Mon Feb 24 13:15:44 GMT 2025 aarch64 GNU/Linux
[all]
BOOT_UART=1
BOOT_ORDER=0xf461
NET_INSTALL_AT_POWER_ON=1
reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe cgroup_disable=memory numa_policy=interleave numa=fake=8 system_heap.max_order=0 smsc95xx.macaddr=D8:3A:DD:A5:9D:A7 vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000 console=ttyAMA10,115200 console=tty1 root=PARTUUID=e7880209-02 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles cfg80211.ieee80211_regdom=GB
Connector 0 (33) HDMI-A-1 (connected)
Encoder 0 (32) TMDS
Crtc 2 (92) [email protected] 594.000 3840/176/88/296/+ 2160/4/10/76/- 60 (60.00) P|D
Plane 4 (105) fb-id: 681 (crtcs: 1 2 3) 0,0 224x36 -> 3362,2068 448x72 (XR24 AR24 AB24 XB24 RG16 BG16 AR15 XR15 RG24 BG24 YU16 YV16 YU24 YV24 YU12 YV12 NV12 NV21 NV16 NV61 P030 XR30 AR30 AB30 XB30 RGB8 BGR8 XR12 AR12 XB12 AB12 BX12 BA12 RX12 RA12)
FB 681 224x36 AR24
Plane 5 (116) fb-id: 685 (crtcs: 1 2 3) 0,0 3840x2160 -> 0,0 3840x2160 (XR24 AR24 AB24 XB24 RG16 BG16 AR15 XR15 RG24 BG24 YU16 YV16 YU24 YV24 YU12 YV12 NV12 NV21 NV16 NV61 P030 XR30 AR30 AB30 XB30 RGB8 BGR8 XR12 AR12 XB12 AB12 BX12 BA12 RX12 RA12)
FB 685 3840x2160 P030
Connector 1 (42) HDMI-A-2 (disconnected)
Encoder 1 (41) TMDS
pi@raspberrypi:~ $
Code: Select all
Revision : d04170
Model : Raspberry Pi 5 Model B Rev 1.0
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted
- dom
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 8472
- Joined: Wed Aug 17, 2011 7:41 pm
Re: NUMA Testing
Just tried on a C1 8G and i see a diagonal lines flicker pattern (top right to bottom left). I'm guessing you are testing with C1.
You can check with:
Code: Select all
$ vclog -m | grep d0
005253.552: Loaded overlay 'bcm2712d0'
It starts with 2712 and ends with either "C1T" or "D0T".
Re: NUMA Testing
Right. It's not loading the 'bcm2712d0' overlay. This Pi is from right after the Pi5 launch, so I would expect it to be the first revision.dom wrote: ↑Wed Feb 26, 2025 1:49 pmJust tried on a C1 8G and i see a diagonal lines flicker pattern (top right to bottom left). I'm guessing you are testing with C1.
You can check with:which won't be present on C1. [..]Code: Select all
$ vclog -m | grep d0 005253.552: Loaded overlay 'bcm2712d0'
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted
Re: NUMA Testing
Will do. Thanks! Would that kernel parameter automatically added by the EEPROM code if necessary?
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted
- dom
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 8472
- Joined: Wed Aug 17, 2011 7:41 pm
Re: NUMA Testing
Yes, assuming this is the fix for your issue (and we don't spot any regressions) then we'll enable it by default through some means
(eeprom bootloader adding it to cmdline if it believes numa is enabled is the likely mechanism).
For info this commit does the same same interleaving to DMA buffers (e.g. allocated for use by hw codec and hvs) as we had been doing for standard user allocations. We didn't include in initial NUMA support as enabling this didn't provide a measurable benefit in benchmarks ran.
However it likely results in sdram accesses that are more efficient for sdram controller (i.e. fewer page opens/closes) for codec and hvs when we run with the new default BANKLOW setting.
I'd be interested if you can see any difference in performance (good or bad) with this change.
In a quick test it seemed to resolve the diagonal glitches (which looked like hvs underruns) for me.
Re: NUMA Testing
Thanks for the explanation. I don't have any real way to measure performance right now (other than it being fast enough, which it still is after this change). I can confirm that this fixes the flickering for me too. That was fast!
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted
Jump to
- Community
- General discussion
- Announcements
- Other languages
- Deutsch
- Español
- Français
- Italiano
- Nederlands
- 日本語
- Polski
- Português
- Русский
- Türkçe
- User groups and events
- Raspberry Pi Official Magazine
- Using the Raspberry Pi
- Beginners
- Troubleshooting
- Advanced users
- Assistive technology and accessibility
- Education
- Picademy
- Teaching and learning resources
- Staffroom, classroom and projects
- Astro Pi
- Mathematica
- High Altitude Balloon
- Weather station
- Programming
- C/C++
- Java
- Python
- Scratch
- Other programming languages
- Windows 10 for IoT
- Wolfram Language
- Bare metal, Assembly language
- Graphics programming
- OpenGLES
- OpenVG
- OpenMAX
- General programming discussion
- Projects
- Networking and servers
- Automation, sensing and robotics
- Graphics, sound and multimedia
- Other projects
- Media centres
- Gaming
- AIY Projects
- Hardware and peripherals
- Camera board
- Compute Module
- Official Display
- HATs and other add-ons
- Device Tree
- Interfacing (DSI, CSI, I2C, etc.)
- Keyboard computers (400, 500, 500+)
- Raspberry Pi Pico
- General
- SDK
- MicroPython
- Other RP2040 boards
- Zephyr
- Rust
- AI Accelerator
- AI Camera - IMX500
- Hailo
- Software
- Raspberry Pi OS
- Raspberry Pi Connect
- Raspberry Pi Desktop for PC and Mac
- Beta testing
- Other
- Android
- Debian
- FreeBSD
- Gentoo
- Linux Kernel
- NetBSD
- openSUSE
- Plan 9
- Puppy
- Arch
- Pidora / Fedora
- RISCOS
- Ubuntu
- Ye Olde Pi Shoppe
- For sale
- Wanted
- Off topic
- Off topic discussion