'Best' language for user Services

Print view

36 posts

pfletch101: Posts: 822; Joined: Sat Feb 24, 2018 4:09 am

Re: 'Best' language for user Services

Quote

Wed Aug 13, 2025 9:04 pm

I started to see declining free memory again - though not as quickly as before - and noticed that restarting one of my services (for unrelated reasons) seemed to have freed up a significant amount of the 'missing' memory. I had a vague recollection, at least partially confirmed by some Googling, that the python requests library, which that service used quite heavily, can leak memory. The suggested 'first-line' way of minimizing this leakage is to close() every returned response after you have finished with it, which I certainly hadn't been doing, since the documentation does not mention the relevant function at all. It will be interesting to see if this has any beneficial effect. It certainly doesn't seem to do any harm!

User avatar DougieLawson: Posts: 43604; Joined: Sun Jun 16, 2013 11:19 pm

Re: 'Best' language for user Services

Quote

Thu Aug 14, 2025 6:46 am

pfletch101 wrote: ↑
Wed Aug 13, 2025 9:04 pm
I started to see declining free memory again

The top (or various flavours of top) command will show you how much memory every task has. Watch that for a while and you may see a growth.

Languages using left-hand whitespace for syntax are ridiculous

DMs sent on Bluesky or by LinkedIn will be answered next month.
Fake doctors - are all on my foes list.

The use of crystal balls and mind reading is prohibited.

pfletch101: Posts: 822; Joined: Sat Feb 24, 2018 4:09 am

Re: 'Best' language for user Services

Quote

Thu Aug 14, 2025 7:31 pm

DougieLawson wrote: ↑
Thu Aug 14, 2025 6:46 am

pfletch101 wrote: ↑
Wed Aug 13, 2025 9:04 pm
I started to see declining free memory again
The top (or various flavours of top) command will show you how much memory every task has. Watch that for a while and you may see a growth.

I have been using top to monitor total usage statistics. It is a bit less straightforward to use it to identify the exact scripts (presumably) responsible - the utility has far too many options - but it may be easier to confirm the source(s) of the problem now I have some active suspects.

User avatar Gavinmc42: Posts: 8346; Joined: Wed Aug 28, 2013 3:31 am

Re: 'Best' language for user Services

Quote

Thu Aug 14, 2025 11:51 pm

A decade ago I had trouble running Python 24/7.
So I learned bash scripting and cron/watch and ran those on TinyCore Linux which I rebooted every night.
Memory leaks went away and they ran for years, some I did not even reboot and had uptimes in the hundreds of days.
Actually TinyCore uses Busybox so is a cut down "Bash".

Some might argue but shell scripting could be "best" as it is usually just there and used for everything else.

I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

peterlite: Posts: 1741; Joined: Sun Apr 17, 2016 4:00 am

Re: 'Best' language for user Services

Quote

Fri Aug 15, 2025 12:40 am

Software makers killing long running programs is as old as programming. The Antikythera computer probably core dumped on occasions. That might be why it was thrown in the ocean. :D

When Microsoft introduced NT, there were servers running for six months without problems. Then someone at Microsoft changed the compiler to not free memory when you used free (). Benchmarks looked faster. But servers died in a few days. Users ended up having to reboot every day.

GTK is the same. GTK has seriously bad memory management. After two days writing a C program, you waste three more days trying to make GTK 3 do what the documentation says it should do then you waste another three days trying to blot out the memory leaks. Or you upgrade your computer from 16 GB to 32 GB just to run a program that should fit in 4 GB.

User avatar Gavinmc42: Posts: 8346; Joined: Wed Aug 28, 2013 3:31 am

Re: 'Best' language for user Services

Quote

Fri Aug 15, 2025 11:58 am

Then someone at Microsoft changed the compiler to not free memory when you used free (). Benchmarks looked faster.

I'm old enough to remember that, it was one of the things that made me dislike MS.
Playing with Windows Server OS was not fun but I was Unix noob and so I started getting Linux books and mags.
It was not until decades later when the Pi came out that I got serious about learning Linux

With cron/watch I can use them to call Python apps that run then shutdown.

The Antikythera computer probably core dumped on occasions

Clickspring is one of my fav maker yt chs, some teenagers are going to steal that thing and send it back in time.
In fact those time traveling teenagers probably explain all of those out of place objects, "hey, lets go see some dinosaurs".
Did they go back and "fix" free() as another joke?

I remember a sifi series in which Terraformers built worlds and did things like embedded signatures into fractal coastlines and fake fossilized dinosaur bones.

Best language - let me ask a friend from the future ;)

I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

swampdog: Posts: 1461; Joined: Fri Dec 04, 2015 11:22 am

Re: 'Best' language for user Services

Quote

Sat Aug 16, 2025 9:21 am

pfletch101 wrote: ↑
Wed Aug 13, 2025 9:04 pm
I started to see declining free memory again - though not as quickly as before - and noticed that restarting one of my services (for unrelated reasons) seemed to have freed up a significant amount of the 'missing' memory. I had a vague recollection, at least partially confirmed by some Googling, that the python requests library, which that service used quite heavily, can leak memory. The suggested 'first-line' way of minimizing this leakage is to close() every returned response after you have finished with it, which I certainly hadn't been doing, since the documentation does not mention the relevant function at all. It will be interesting to see if this has any beneficial effect. It certainly doesn't seem to do any harm!

I wonder why that hasn't been fixed. Maybe it's not a true leak but rather an object which is holding onto an ever increasing buffer?

In C++ terms..

Code: Select all

typedef std::vector<std::string> VecStr;

If we write..

Code: Select all

int main()
{VecStr vs;
 //stuff
}

..then 'vs' will be hanging around until the program terminates. We can make it worse..

Code: Select all

VecStr vs;
int main()
{
 while (!done)	{
 	LoadFile(vs);
 	//stuff
 }

..especially if "stuff" also makes calls which also use 'vs'. All it would take is for the underlying C code being called by python to behave like above to create a non leaky potential memory gobbler. ;-)

Hopefully you've found the culprit in as much it'll run long enough to not have to bother recoding it.

swampdog: Posts: 1461; Joined: Fri Dec 04, 2015 11:22 am

Re: 'Best' language for user Services

Quote

Sat Aug 16, 2025 10:32 am

peterlite wrote: ↑
Fri Aug 15, 2025 12:40 am
Software makers killing long running programs is as old as programming. The Antikythera computer probably core dumped on occasions. That might be why it was thrown in the ocean. :D

When Microsoft introduced NT, there were servers running for six months without problems. Then someone at Microsoft changed the compiler to not free memory when you used free (). Benchmarks looked faster. But servers died in a few days. Users ended up having to reboot every day.

GTK is the same. GTK has seriously bad memory management. After two days writing a C program, you waste three more days trying to make GTK 3 do what the documentation says it should do then you waste another three days trying to blot out the memory leaks. Or you upgrade your computer from 16 GB to 32 GB just to run a program that should fit in 4 GB.

I've always thought it a shame the GUI preceded multitasking in common use. Life would be so much easier if we could fire up our app and have it talk to its GUI thread in a standard fashion. Ditto for the console. The digital research GEM had a gui mode where the display was divided into character cells plus a pixel offset within the cell. We could have had platform agnostic std::gui and std::con with fallback to tty/serial.

What we got instead was callbacks into an app which is failing, by a gui, itself failing. Surely that should have been the other way around?

swampdog: Posts: 1461; Joined: Fri Dec 04, 2015 11:22 am

Re: 'Best' language for user Services

Quote

Mon Aug 18, 2025 9:52 am

pfletch101 wrote: ↑
Wed Aug 13, 2025 9:04 pm
I started to see declining free memory again - though not as quickly as before - and noticed that restarting one of my services (for unrelated reasons) seemed to have freed up a significant amount of the 'missing' memory. I had a vague recollection, at least partially confirmed by some Googling, that the python requests library, which that service used quite heavily, can leak memory. The suggested 'first-line' way of minimizing this leakage is to close() every returned response after you have finished with it, which I certainly hadn't been doing, since the documentation does not mention the relevant function at all. It will be interesting to see if this has any beneficial effect. It certainly doesn't seem to do any harm!

I just noticed my PC is paging. It should never do that. Tons of free ram. Turns out the offending processes are all being driven by python..

Code: Select all

foo@sdu:~/usr/src/QT$ python3 --version
Python 3.10.12

The python on my bookwork rpi5 is currently..

Code: Select all

foo@pi24:~ $ python3 --version
Python 3.11.2

There does indeed seem to be a problem. In the case of my PC, it's virt-manager (kvm/qemu gui) which I leave running 24/7.

User avatar Gavinmc42: Posts: 8346; Joined: Wed Aug 28, 2013 3:31 am

Re: 'Best' language for user Services

Quote

Mon Aug 18, 2025 9:42 pm

Python still got a problem with leakage?
I used to run a Python webserver a decade ago and used top to watch those old 500MB Pi's run out of ram.
Ended up making updating html webpage using Bash with Awk, Grep, Sed.
A big mess of scripts that was a pain to write but which just ran for years.

No apps or libs needed to be installed.

I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

swampdog: Posts: 1461; Joined: Fri Dec 04, 2015 11:22 am

Re: 'Best' language for user Services

Quote

Wed Aug 20, 2025 2:19 am

Here's a thing I didn't know about 'top', it can save its view settings. In order not to permanently affect it, fiddle with HOME..

Code: Select all

foo@pi24:/wrk $ HOME=/tmp top

Now use 'f' to get the display list, use the hints to get "SWAP" into the first position, enabled and set it as the sort item (via arrow keys, space and 's' key from hints). This persists for batch mode but you must be back at the main screen to save the settings and its SHIFT-W rather than 'W', the latter yields a cryptic popup error I've already forgotten.

Code: Select all

foo@pi24:/wrk $ HOME=/tmp top -n 1 -b

I found a bug in rpi bookworm though in that something is not respecting environmental variable "COLUMNS". Seems this workaround suffices..

Code: Select all

foo@pi24:~ $ COLUMNS=$COLUMNS HOME=/tmp top -n 1 -b | head
top - 03:04:45 up 10:19, 5 users, load average: 2.95, 2.69, 2.85
Tasks: 267 total, 4 running, 263 sleeping, 0 stopped, 0 zombie
%Cpu(s): 40.0 us, 60.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st 
MiB Mem : 8059.2 total, 1167.0 free, 6369.6 used, 6166.0 buff/cache 
MiB Swap: 16384.0 total, 15607.1 free, 776.9 used. 1689.6 avail Mem 
 SWAP PID USER PR NI VIRT RES SHR %CPU %MEM TIME+ nTH nMaj nMin nDRT OOMa OOMs COMMAND
 63488 1579 nx 20 0 2368288 56192 14944 0.0 0.7 0:49.95 31 773 84k 0 0 669 nxserver.bin
 26112 1120 root 20 0 536048 44176 27712 0.0 0.5 1:54.71 4 814 15k 0 0 668 Xorg
 22528 1711 foo 0 -20 2349200 66944 20896 0.0 0.8 0:53.82 20 1383 36k 0 0 668 nxnode.bin

I wrote a script to make this easier. It runs the above single batch iteration and sinks any output which isn't paging. Dunno what half those columns mean btw - just that they might be useful..

Code: Select all

foo@pi24:~ $ sys-topswap 
top - 03:05:50 up 10:20, 5 users, load average: 3.23, 2.85, 2.90
Tasks: 272 total, 4 running, 268 sleeping, 0 stopped, 0 zombie
%Cpu(s):100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st 
MiB Mem : 8059.2 total, 1139.7 free, 6427.6 used, 6179.2 buff/cache 
MiB Swap: 16384.0 total, 15608.1 free, 775.9 used. 1631.6 avail Mem 
 SWAP PID PPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ nTH nMaj nMin nDRT OOMa OOMs vMj vMn USED COMMAND
 63488 1579 1 nx 20 0 2368416 56192 14944 S 0.0 0.7 0:50.05 31 775 84k 0 0 669 0 0 119680 nxserver.bin
 26112 1120 1020 root 20 0 536048 44176 27712 S 0.0 0.5 1:54.71 4 814 15k 0 0 668 0 0 70288 Xorg
 22528 1711 1710 foo 0 -20 2349200 66944 20896 S 0.0 0.8 0:53.83 20 1383 36k 0 0 668 0 0 89472 nxnode.bin
 19456 1268 1156 foo 20 0 1175152 30624 20576 S 0.0 0.4 0:07.01 7 386 5427 0 0 668 0 0 50080 lxpanel
 14336 1304 1274 foo 20 0 325984 12496 10448 S 0.0 0.2 0:00.07 2 121 2061 0 0 667 0 0 26832 gtk-nop
 13248 1269 1156 foo 20 0 980784 68368 39408 S 0.0 0.8 0:23.70 6 1429 11k 0 0 668 0 0 81616 pcmanfm
 9728 1154 1136 foo 9 -11 702128 14256 10448 S 0.0 0.2 0:00.59 8 510 2271 0 200 800 0 0 23984 wireplumber
 4608 1740 1711 foo 20 0 1389104 21088 16480 S 0.0 0.3 0:03.44 14 435 3232 0 0 667 0 0 25696 nxrunner.bin
 4096 215389 1 root 20 0 181664 7568 6544 S 0.0 0.1 0:00.11 3 40 644 0 0 666 0 0 11664 cups-browsed
 3584 1725 1 colord 20 0 244880 7472 6448 S 0.0 0.1 0:00.13 3 174 772 0 0 666 0 0 11056 colord
 3072 1137 1136 foo 20 0 105824 3520 1568 S 0.0 0.0 0:00.00 1 0 44 0 100 733 0 0 6592 (sd-pam)
 3072 1262 1156 foo 20 0 226016 13216 9184 S 0.0 0.2 0:00.64 1 181 1390 0 0 666 0 0 16288 openbox
 2560 870 1 polkitd 20 0 311328 6784 5248 S 0.0 0.1 0:00.18 3 174 2676 0 0 666 0 0 9344 polkitd
 2560 1675 1579 nx 20 0 902528 5776 4752 S 0.0 0.1 0:03.20 11 65 624 0 0 666 0 0 8336 nxd
 2048 983 1 root 20 0 250640 7360 5824 S 0.0 0.1 0:00.07 3 169 611 0 0 666 0 0 9408 ModemManager
 2048 1152 1136 foo 9 -11 115920 14768 9248 S 0.0 0.2 0:14.09 3 297 1601 0 200 800 0 0 16816 pipewire
 2048 215388 1 root 20 0 32096 5936 4400 S 0.0 0.1 0:00.04 1 69 498 0 0 666 0 0 7984 cupsd
 1536 1 0 root 20 0 169792 9360 6368 S 0.0 0.1 0:01.41 1 244 7405 0 0 0 0 0 10896 systemd
 1536 426 1 root 20 0 27776 4800 3264 S 0.0 0.1 0:00.17 1 99 1292 0 -1000 0 0 0 6336 systemd-udevd
 1536 899 1 root 20 0 397328 9488 6416 S 0.0 0.1 0:04.80 5 91 2818 0 0 666 0 0 11024 udisksd
 1536 954 1 root 20 0 263808 10288 7728 S 0.0 0.1 0:02.87 3 123 767 0 0 666 0 0 11824 NetworkManager
 1536 1041 1 root 20 0 16832 4816 4816 S 0.0 0.1 0:00.02 1 51 482 0 -1000 0 0 0 6352 sshd
 1536 1136 1 foo 20 0 20608 7472 6448 S 0.0 0.1 0:00.25 1 95 1531 0 100 733 0 0 9008 systemd
 1536 1156 1131 foo 20 0 253200 8128 7104 S 0.0 0.1 0:00.55 3 145 1200 0 0 666 0 0 9664 lxsession
 1024 861 1 root 20 0 240128 7808 5248 S 0.0 0.1 0:00.72 3 266 521 0 0 666 0 0 8832 accounts-daemon
 1024 962 1 root 20 0 17680 4416 3904 S 0.0 0.1 0:00.19 1 31 391 0 0 666 0 0 5440 wpa_supplicant
 1024 1131 1020 root 20 0 162752 7024 5488 S 0.0 0.1 0:00.01 3 114 459 0 0 666 0 0 8048 lightdm
 1024 1153 1136 foo 20 0 87744 3280 3280 S 0.0 0.0 0:00.01 2 9 325 0 200 800 0 0 4304 pipewire
 1024 1155 1136 foo 9 -11 173968 8304 5392 S 0.0 0.1 0:00.36 3 168 1127 0 200 800 0 0 9328 pipewire-pulse
 1024 1248 1136 foo 20 0 238992 5904 5392 S 0.0 0.1 0:00.01 3 55 381 0 200 800 0 0 6928 gvfsd
 1024 1253 1136 foo 20 0 387760 5312 4800 S 0.0 0.1 0:00.00 6 3 247 0 200 800 0 0 6336 gvfsd-fuse
 1024 1266 1156 foo 20 0 181888 9632 8096 S 0.0 0.1 0:00.03 3 143 471 0 0 666 0 0 10656 lxpolkit
 1024 1274 1 foo 20 0 17168 4304 4304 S 0.0 0.1 0:00.00 1 1 214 0 0 666 0 0 5328 systemd-inhibit
 1024 1627 1604 foo 20 0 8384 3712 2688 S 0.0 0.0 0:00.02 1 1 483 0 0 666 0 0 4736 bash
 1024 2833 1041 root 20 0 20080 6912 5376 S 0.0 0.1 0:00.01 1 0 446 0 0 666 0 0 7936 sshd
 1024 2844 2833 foo 20 0 20496 5872 4272 S 0.0 0.1 0:15.41 1 6 162 0 0 666 0 0 6896 sshd
 1024 2845 2844 foo 20 0 8816 3744 2720 S 0.0 0.0 0:00.07 1 0 2092 0 0 666 0 0 4768 bash
 1024 2959 2958 foo 20 0 8272 3216 2704 S 0.0 0.0 0:00.02 1 0 553 0 0 666 0 0 4240 bash
 512 406 1 root 20 0 58832 16992 15968 S 0.0 0.2 0:00.49 1 592 8563 0 -250 500 0 0 17504 systemd-journal
 512 822 1 systemd+ 20 0 91136 5328 4816 S 0.0 0.1 0:00.14 2 3 364 0 0 666 0 0 5840 systemd-timesyn
 512 864 1 root 20 0 13728 4192 3680 S 0.0 0.1 0:00.04 1 15 304 0 0 666 0 0 4704 bluetoothd
 512 882 1 root 20 0 11520 3648 3136 S 0.0 0.0 0:00.03 1 25 266 0 0 666 0 0 4160 smartd
 512 892 1 root 20 0 34352 6912 5376 S 0.0 0.1 0:00.67 1 48 394 0 0 666 0 0 7424 systemd-logind
 512 1339 1136 foo 20 0 501984 10096 7024 S 0.0 0.1 0:00.28 4 57 556 0 200 800 0 0 10608 gvfs-udisks2-vo
 512 1346 1136 foo 20 0 234480 5216 4704 S 0.0 0.1 0:00.00 3 1 253 0 200 800 0 0 5728 gvfs-mtp-volume
 512 1350 1136 foo 20 0 234640 4752 4752 S 0.0 0.1 0:00.00 3 3 246 0 200 800 0 0 5264 gvfs-goa-volume
 512 1354 1136 foo 20 0 313856 5840 5328 S 0.0 0.1 0:00.96 4 8 310 0 200 800 0 0 6352 gvfs-afc-volume
 512 1359 1136 foo 20 0 235968 5728 5216 S 0.0 0.1 0:00.00 3 4 276 0 200 800 0 0 6240 gvfs-gphoto2-vo
 512 1369 1 foo 20 0 157856 5344 4832 S 0.0 0.1 0:00.00 3 0 167 0 0 666 0 0 5856 menu-cached
 512 1373 1248 foo 20 0 313296 6928 5904 S 0.0 0.1 0:00.04 3 81 329 0 200 800 0 0 7440 gvfsd-trash
 512 1604 1 root 20 0 7984 3136 2624 S 0.0 0.0 0:00.00 1 9 302 0 0 666 0 0 3648 login
 512 215407 215388 lp 20 0 18144 3760 2736 S 0.0 0.0 0:00.00 1 1 260 0 0 666 0 0 4272 dbus

Don't take any meaning from the above regards SWAP though: pi24 is busy being hammered compiling something which takes days!

I can post the script but it will require something from github (temporary file/folder shell script code).