Problem running Celery task inside Docker container

Question

What I have

I have a container with Django app that executes a Celery task whose purpose is to delete some files that are the media folder, that container is called backend in docker-compose.yml.

Problem

the problem is that the Celery Worker and Celery beat container runs normally and execute the task but does not delete media files. it may be that the problem lies in the volumes mounted on the containers and somehow Celery does not find the media files to be deleted.

docker-compose.yml

version: '3.8'
services:
 # Django app that will upload the files from the admin and that will be deleted by the Celery task.
 backend:
 expose:
 - 8000
 build:
 context: .
 dockerfile: ./Dockerfile
 networks:
 - backend-tier
 depends_on:
 - db
 redis:
 image: library/redis:5.0-alpine
 ports:
 - 6379:6379
 restart: unless-stopped
 networks:
 - backend-tier
 volumes:
 - valor-redis:/data
 worker:
 build:
 context: .
 dockerfile: ./Dockerfile
 command: celery -A config --app=config.celery_app:app worker --loglevel=info
 restart: unless-stopped
 networks:
 - backend-tier
 depends_on:
 - redis
 volumes:
 - ./app_com_co/:/app/app_com_co:Z,cached
 - ./app_com_co/templates/:/app/app_com_co/templates:Z,cached
 # shared volume between worker and backend for media
 - app-media:/app/app_com_co/media
 
 beat:
 build:
 context: .
 dockerfile: ./Dockerfile
 command: celery -A config --app=config.celery_app:app beat --loglevel=info
 restart: unless-stopped
 networks:
 - backend-tier
 depends_on:
 - redis
 volumes:
 - ./app_com_co/:/app/app_com_co:Z,cached
 - ./app_com_co/templates/:/app/app_com_co/templates:Z,cached
 # shared volume between beat and backend for media
 - app-media:/app/app_com_co/media
volumes:
 app-redis:
 driver: local
 app-media:
networks:
 backend-tier:
 driver: bridge

• ItayB: how do those media files should get there in the first place? uploaded from backend container?
• Cristian Flórez: @ItayB Yes, its uploaded from backend container (from the Django admin)... . The Celery task will check every day and find the files that are at a certain date and delete them.
• ItayB: so why didn't you add the app-media volume to the backend container as well?
• Cristian Flórez: you're right, the app-media volume needed to be added to backend to delete the files. one additional question, what kind of volume to apply a named shared volume or a Host volume fit better for this case.
• ItayB: not sure what you are asking, can you elaborate? and I added my answer, if it solves your question please accept that :)

Answer 1

You're missing the app-media volume that needed to be added to backend if the files are uploaded there.