-
Notifications
You must be signed in to change notification settings - Fork 250
Add Kubernetes job runner (#347) #349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Kubernetes job runner (#347) #349
Conversation
DrissiReda
commented
Mar 25, 2021
Since kubernetes is dropping docker, can this allow us to use the kubernetes cluster's container runtime? (Mainly interested in containerd).
@DrissiReda, yes. The Kubernetes Job abstraction is at a higher level than Kubernetes' Container Runtime Interface (CRI), which is where Docker plugs into Kubernetes if you use Docker. And the k3s cluster I used for development and testing of this pull request already doesn't use Docker.
6738e9b to
9cc6cbe
Compare
I've separated better the changes that add support to the entrypoint for trusting CA certs. There are separate trusts for Elasticsearch and for outgoing web services. I have not made a separate pull request for those, but I can. They are possibly of more general interest than the Kubernetes job runner.
9cc6cbe to
15ed398
Compare
OK. I went ahead and split the Elasticsearch authentication and CA cert stuff out into #362. I gave the commits here a hard look, and decided that the story of progression toward functionality they told wasn't something that needed to be preserved. So I collapsed the commits into one, for easier review. The "cruft picked up along the way," which I mentioned at the top of the thread, is gone.
tl-Bruno-Braga
commented
Aug 20, 2021
@jaredjennings any updates on this? I assume that at this point this feature is still not implemented?
@tl-Bruno-Braga, here it sits. Works for me, though I haven't taken it to production yet. I've asked a couple of times on the Discord about it.
This commits my current progress in re-creating the docker-compose pipeline for PeekabooAV in Kubernetes. That includes deployments, services, and hard-coded config files for each step in the pipeline, modeled after what was done in the [pipeline](/sett17/peekabooav-installer/tree/pipeline). The yamls for cortex, and the set-up job, are included, although cortex does currently not work inside of Kubernetes, due to the missing docker runner. There is an open [PR](/TheHive-Project/Cortex/pull/349) and corresponding issue. Except for above mentioned cortex, the pipeline is fully functional. Meaning one can send an email to the postfix_tx deployment, which is then sent to the postfix_rx deployment and then processed by rspamd and Peekaboo. This was tested and developed with microk8s and a single node.
This commits my current progress in re-creating the docker-compose pipeline for PeekabooAV in Kubernetes. That includes deployments, services, and hard-coded config files for each step in the pipeline, modeled after what was done in the [pipeline](/Sett17/PeekabooAV-Installer/tree/pipeline). The yamls for cortex, and the set-up job, are included, although cortex does currently not work inside of Kubernetes, due to the missing docker runner. There is an open [PR](/TheHive-Project/Cortex/pull/349) and corresponding issue. Except for above mentioned cortex, the pipeline is fully functional. Meaning one can send an email to the postfix_tx deployment, which is then sent to the postfix_rx deployment and then processed by rspamd and Peekaboo. This was tested and developed with microk8s and a single node.
15ed398 to
009c997
Compare
michielboekhoff
commented
Sep 27, 2023
I know it's been a while - but I just wanted to say thank you @jaredjennings for creating this PR. I'm disheartened that this hasn't been merged yet, given Kubernetes' popularity. For now, I'm having to maintain a fork just to run Cortex on a managed Kubernetes cluster.
25a727f to
afb3e63
Compare
afb3e63 to
16e795a
Compare
These changes implement a job runner that makes Kubernetes API calls to run Dockerized neurons as Kubernetes Jobs.
There's also a bit of cruft picked up along the way, like enabling Dockerized Cortex to connect to Elasticsearch using HTTPS. Please look out for rookie mistakes: this is my first Scala code.