Run python script from shell script using crontab

Question

I have a shell script main.sh which runs a python script main.py. When i run the main.sh script manually then the execution is fine (Note: i am running it manually with root user) and the python script works as expected. However when i try the same from crontab which runs the main.sh script every 30 mins then python script is not getting executed.(i have already done chmod u+x main.sh to make it executable)

The crontab given below is also of the root user

my crontab file: */30 * * * * /home/opc/python_scripts/main.sh >> /home/opc/cron.log.

my main.sh file:

#!/bin/bash
output=$(sh /home/opc/python_scripts/url.sh)
echo "$output"
searchstring="url is up, login service is up and oc console is up"
if [[ "$output" == *"$searchstring"* ]] ; then
 echo "URL is up"
 sudo python3 /home/opc/python_scripts/main.py >> /home/opc/pyfile.log
else
 echo "URL is Down"
 sudo python3 /home/opc/python_scripts/disable_crr.py
fi

I know that it is entering the first if condition and it is also printing the "URL is up" statement, i have another log file which records this. When i run it manually then i can see that even python script is running properly but with crontab it gets stuck at echo "URL is up" , after that no execution happens

Things i have done to solve this:

1. i have seen solutions of similar questions like-Calling a python script from shell script cron , Shell Script: Execute a python program from within a shell script, but in all of these questions the user was not able to run the python script from the shell script manually. I am able to run it manually but if i use a cronjob then i am having a problem.

2. when i ran ls -la in the directory where these scripts are located the ownership of these files was like:

main.sh root root 
main.py opc opc

Also i added #!/usr/bin/env python3 at the top of my python script as i had seen this as a solution in a similiar question but did not work, only manually works, not with crontab.

How do i run it with crontab?

• alexzander: have you tried to run the main.sh like this: /home/opc/python_scripts/main.sh >> /home/opc/cron.log, without crontab ?
• aditya_sharma: @alexzander yeah i did, manually i am able to run the file but only crontab is giving me problems.

Answer 1

Did you try to remove sudo?

However, since you have to run each 30 minutes you can also considered to:

• scheduling directly in python using schedule
• using a python library to write your CronTab file
• running on a more isolated enviroment using docker and schedule with crontab the docker run
• finally, if you have access to it, you can consider also using cronjob on a k8s cluster et similia

• aditya_sharma: i did remove sudo and nothing changed, i looked at "schedule" but yeah wanted to use cron as i have never worked with schedule and this is a team project so there are other scritps in the cronjob which depends on this script execution. About k8s and docker, have not worked with them before .

Answer 2

You're attempting to execute the script with sudo, but cron can't prompt you for the password. Since this is root's crontab, you shouldn't need to use sudo.

If you must use sudo in cron, you need to designate those commands to be passwordless through /etc/sudoers. Something along these lines should work.

opc ALL=(root) NOPASSWD: /home/opc/python_scripts/main.py

• aditya_sharma: so yeah, i tried your suggestion but did not affect as it still does not execute. The behavior is still the same. even if i use sudo it does not ask me for password, the thing is this is an instance on Oracle cloud, so when i write sudo it does not ask for a password, if i write sudo su, it directly takes me to root user, never asked for a password. My linux is centos7

Answer 3

So I do not know what the problem was but I solved it.

How:

previously as I had mentioned in my question that I had tried the above two questions separately.

1. Now what I did was, implemented the given answer of Calling a python script from shell script cron in which i only used this part of the answer :

#!/usr/bin/env bash
dirName=`dirname 0ドル`
baseName=`basename 0ドル`
arg1=1ドル
arg2=2ドル
cd ${dirName} && python ./room_wise.py arg1 arg2

2. from the second question Shell Script: Execute a python program from within a shell script, i used João Víctor answer method 2 and replace python with python3.

I do not completely understand what the problem was but yeah when I used both of these solutions together then it started working. Also as other people have mentioned in this post, do not use sudo if you are using root's crontab and otherwise as well as it can lead to authorization issues. This solved it so, if anyone has a better understanding then do add your answers as well.