Crontab won't run python script

Question

I'm trying to run two simple python scripts from crontab:

crontab file:

@reboot /usr/bin/python3 /home/pi/init_server.py
@reboot /usr/bin/python3 /home/pi/scheduler.py

crontab log:

Aug 12 10:07:11 raspberrypi CRON[423]: (root) CMD (/usr/bin/python3 /home/pi/scheduler.py)
Aug 12 10:07:11 raspberrypi CRON[421]: (root) CMD (/usr/bin/python3 /home/pi/init_server.py)

Running both these files with the same cmd as in the crontab file works fine, but they are not running after a reboot. Any ideas how i can debug this further.

Code from init_server.script:

 import os
 #starts a server on port 4000
 os.system('ruterstop --server')

running a curl command on localhost:4000 doesn't work

---- Update ----

As mentioned in the comments, the server should not be started in python but as a systemctl service.

Another issue i encountered when I added a logfile, is that one of my modules are not found. Even tough it's available when i import it in the shell.

Log from crontab:

Traceback (most recent call last):
 File "/home/pi/scheduler.py", line 7, in <module>
 from yr.libyr import Yr
ModuleNotFoundError: No module named 'yr'

Output from python shell: python shell

• Milliways: without knowing what these scripts do no one can say/.
• Øystein Seel: @Milliways I've added the content of one of the files. Shouldn't the crontab log provide an error message if the python file won't run?
• Dougie: Change ruterstop to /usr/bin/ruterstop or /usr/local/bin/ruterstop (wherever it's been installed). Your script probably can't find that command/program.
• Milliways: Why you you write a python script to run a single OS command? We still don't know what it is or does. Presumably it requires networking, in which case cron is not the way to do it as it runs before networking is up. You need to write a systemd service with a networking dependency.

Answer 1

The most likely reason is that your scripts require a resource or a service that's not yet available when cron is started. This is commonly encountered when running scripts @reboot that use networking resources - as yours seem to do (i.e. curl).

If this is the problem, it's easy enough to fix by running sleep for 15 seconds or so before you invoke you script. cron log files are typically not very exciting to read - they usually don't help debugging. To capture potentially relevant messages to aid debugging, you should redirect the stderr & stdout output from your script to a file for post-mortem review.

You might try this first:

@reboot sleep 15; python3 /home/pi/init_server.py && python3 /home/pi/scheduler.py >> /home/pi/mycronlog.txt 2>&1

A few notes to keep in mind:

1. Full paths for cron commands are an excellent habit. However, /usr/bin is in the PATH for cron. You can see all of cron environment variables by running printenv as a cron job & redirecting the output to a file as shown above.

2. Keep in mind that the default shell for cron is sh - not bash.

3. Using a she-bang as the first line in your Python script may save you the trouble of invoking python3 to run the script; perhaps: #!/usr/bin/env python3

Answer 2

You can add logging to your Python scripts like this:

@reboot /usr/bin/python3 /home/pi/init_server.py >> /home/pi/init_server.log
@reboot /usr/bin/python3 /home/pi/scheduler.py >> /home/pi/scheduler.log

Then check the output of your Python scripts from the command line:
cat /home/pi/init_server.log and cat /home/pi/scheduler.log

From first glance, it seems like ruterstop may need a path prefixing it. Can you try which ruterstop on the command line to see what it is, then put that full path into the os.system call?

Answer 3

Most crontab problems are related to environment variables. Further, most crontab problems result in error messages that get discarded. My recommendation would be to make sure to capture error messages to a place where you can see them, like:

@reboot /usr/bin/python3 /home/pi/init_server.py >/tmp/foo.log 2>&1

and consider adding this command:

@reboot printenv >/tmp/env.log 2>&1

to see what your environment variables are actually set to.

Answer 4

Based on Milliways comment I had to run the server in something else than crontab. After a bit of trial and error I figured out why my modules weren't found, when running them with systemctl.

After installing the necessary dependencies based on this answer I was able to get the it running