-
Notifications
You must be signed in to change notification settings - Fork 179
Cortex.cpp: Installation & Uninstallation #1114
-
Goal
- Installer can be updated rapidly (vs. waiting for Package Manager approvals)
- Installer supports Stable, Beta and Nightly versions
Major Decisions
- We will have native installers, as package manager approvals take time
- We will install llama.cpp engine by default, at install time (i.e.
cortex engines init llamacpp) - Use native installers to take advantage of pre-install and post-install hooks (
.exe,.dmg,.deband.appimage)
Out-of-scope (for now)
- "Full Installer" for air-gapped use cases (i.e. unable to pull llama.cpp at install-time)
Related Issues
- feat: Jan has a portable / non install version jan#2643
- Discussion: Cortex.cpp Uninstallation #775
- feat: Jan support Steam Deck jan#3360
- bug: users cannot do a fresh cortex install due to dangling cortex folder (from previous install) #1082
Cortex's Installed Files
Installer:
- Windows
- Format:
.exe - Binary location:
- Stable:
C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe - Beta:
C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exe - Nighty:
C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exe
- Stable:
- Configuration file:
- Stable:
C:\Users\<username>\.cortexrc - Beta:
C:\Users\<username>\.cortexrc-beta - Nighty:
C:\Users\<username>\.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
C:\Users\<username>\.cortexcpp - Beta:
C:\Users\<username>\.cortexcpp-beta - Nighty:
C:\Users\<username>\.cortexcpp-nightly
- Stable:
- Format:
- Linux:
- Format:
.deb - Binary location:
- Stable:
/usr/bin/cortexcpp - Beta:
/usr/bin/cortexcpp-beta - Nighty:
/usr/bin/cortexcpp-nightly
- Stable:
- Configuration file:
- Stable:
/home/<username>/.cortexrc - Beta:
/home/<username>/.cortexrc-beta - Nighty:
/home/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/home/<username>/.cortexcpp - Beta:
/home/<username>/.cortexcpp-beta - Nighty:
/home/<username>/.cortexcpp-nightly
- Stable:
- Format:
- Macos:
- Format:
.pkg - Binary location:
- Stable:
/usr/local/bin/cortexcpp - Beta:
/usr/local/bin/cortexcpp-beta - Nighty:
/usr/local/bin/cortexcpp-nightly
- Stable:
- Configuration file:
- Stable:
/Users/<username>/.cortexrc - Beta:
/Users/<username>/.cortexrc-beta - Nighty:
/Users/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/Users/<username>/.cortexcpp - Beta:
/Users/<username>/.cortexcpp-beta - Nighty:
/Users/<username>/.cortexcpp-nightly
- Stable:
- Format:
Original Post
Motivation
Couple of options:
- via Package managers (brew, winget, launchpad, etc.)
- Pros:
- Easy to update (maybe?) just provide package manager team
- Easy for acquire user? (like just running
brew install cortexcppor something like that)
- Cons:
- Take long time for new version come out
- Hard (or impossible) to choose a place to install binary/files (not entirely sure. Open for discussion)
- self hosted
- Pros:
- Not being constrained by Package managers
- Shorter time to market
- Easy to have multiple variants (stable, beta, nightly)
- Let user decide where to store files. More managable?
- Cons:
- Have to handle upgrade ourselves
- User acquisition is more difficult?
- Hmm, both?
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
Replies: 12 comments 2 replies
-
Installer:
- Windows
- Format:
.exe - Binary location:
- Stable:
C:\Users\<username>\AppData\Local\cortex\cortex.exe - Beta:
C:\Users\<username>\AppData\Local\cortex-beta\cortex-beta.exe - Nighty:
C:\Users\<username>\AppData\Local\cortex-nightly\cortex-nightly.exe
- Stable:
- Configuration file:
- Stable:
C:\Users\<username>\.cortexrc - Beta:
C:\Users\<username>\.cortexrc-beta - Nighty:
C:\Users\<username>\.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
C:\Users\<username>\.cortexcpp - Beta:
C:\Users\<username>\.cortexcpp-beta - Nighty:
C:\Users\<username>\.cortexcpp-nightly
- Stable:
- Format:
- Linux:
- Format:
.deb - Binary location:
- Stable:
/usr/bin/cortex - Beta:
/usr/bin/cortex-beta - Nighty:
/usr/bin/cortex-nightly
- Stable:
- Configuration file:
- Stable:
/home/<username>/.cortexrc - Beta:
/home/<username>/.cortexrc-beta - Nighty:
/home/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/home/<username>/.cortex - Beta:
/home/<username>/.cortexcpp-beta - Nighty:
/home/<username>/.cortexcpp-nightly
- Stable:
- Format:
- Macos:
- Format:
.pkg - Binary location:
- Stable:
/usr/local/bin/cortex - Beta:
/usr/local/bin/cortex-beta - Nighty:
/usr/local/bin/cortex-nightly
- Stable:
- Configuration file:
- Stable:
/Users/<username>/.cortexrc - Beta:
/Users/<username>/.cortexrc-beta - Nighty:
/Users/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/Users/<username>/.cortexcpp - Beta:
/Users/<username>/.cortexcpp-beta - Nighty:
/Users/<username>/.cortexcpp-nightly
- Stable:
- Format:
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 2
-
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 I spot some inconsistencies in your folder/file naming vs #1040
I just edited my comment base on this comment #1040 (comment) from @dan-homebrew
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 I made some changes to your folder/binary layout, based on our earlier discussion #1092
Changes
- CLI command is
cortex, and binary name iscortex(orcortex-beta,cortex-nightly) - Otherwise, our product is
cortexcpp(with versions:cortexcpp-beta,cortexcpp-nightly) - Our convention is to hyphenate
product-version - Data folder is pending discussion on visible vs. hidden folder in Discussion: Cortex.cpp Data Structures #1040
Installer:
- Windows
- Format:
.exe - Binary location:
- Stable:
C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe - Beta:
C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exe - Nighty:
C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exe
- Stable:
- Configuration file:
- Stable:
C:\Users\<username>\.cortexrc - Beta:
C:\Users\<username>\.cortexrc-beta - Nighty:
C:\Users\<username>\.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
C:\Users\<username>\.cortexcpp - Beta:
C:\Users\<username>\.cortexcpp-beta - Nighty:
C:\Users\<username>\.cortexcpp-nightly
- Stable:
- Format:
- Linux:
- Format:
.deb - Binary location:
- Stable:
/usr/bin/cortexcpp - Beta:
/usr/bin/cortexcpp-beta - Nighty:
/usr/bin/cortexcpp-nightly
- Stable:
- Configuration file:
- Stable:
/home/<username>/.cortexrc - Beta:
/home/<username>/.cortexrc-beta - Nighty:
/home/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/home/<username>/.cortexcpp - Beta:
/home/<username>/.cortexcpp-beta - Nighty:
/home/<username>/.cortexcpp-nightly
- Stable:
- Format:
- Macos:
- Format:
.pkg - Binary location:
- Stable:
/usr/local/bin/cortexcpp - Beta:
/usr/local/bin/cortexcpp-beta - Nighty:
/usr/local/bin/cortexcpp-nightly
- Stable:
- Configuration file:
- Stable:
/Users/<username>/.cortexrc - Beta:
/Users/<username>/.cortexrc-beta - Nighty:
/Users/<username>/.cortexrc-nightly
- Stable:
- Data folder (contains engines, models, logs)
- Stable:
/Users/<username>/.cortexcpp - Beta:
/Users/<username>/.cortexcpp-beta - Nighty:
/Users/<username>/.cortexcpp-nightly
- Stable:
- Format:
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 I made some changes to your folder/binary layout, based on our earlier discussion #1092
Changes
CLI command is
cortex, and binary name iscortex(orcortex-beta,cortex-nightly)Our convention is to hyphenate
name-versionData folder is pending discussion on visible vs. hidden folder in Discussion: Cortex.cpp Data Structures #1040
Installer:
Windows
Format:
.exeBinary location:
Stable:
C:\Users\<username>\AppData\Local\cortexcpp\cortex.exeBeta:
C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exeNighty:
C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exeConfiguration file:
Stable:
C:\Users\<username>\.cortexrcBeta:
C:\Users\<username>\.cortexrc-betaNighty:
C:\Users\<username>\.cortexrc-nightlyData folder (contains engines, models, logs)
Stable:
C:\Users\<username>\.cortexcppBeta:
C:\Users\<username>\.cortexcpp-betaNighty:
C:\Users\<username>\.cortexcpp-nightlyLinux:
Format:
.debBinary location:
Stable:
/usr/bin/cortexcppBeta:
/usr/bin/cortexcpp-betaNighty:
/usr/bin/cortexcpp-nightlyConfiguration file:
Stable:
/home/<username>/.cortexrcBeta:
/home/<username>/.cortexrc-betaNighty:
/home/<username>/.cortexrc-nightlyData folder (contains engines, models, logs)
Stable:
/home/<username>/.cortexcppBeta:
/home/<username>/.cortexcpp-betaNighty:
/home/<username>/.cortexcpp-nightlyMacos:
Format:
.pkgBinary location:
Stable:
/usr/local/bin/cortexcppBeta:
/usr/local/bin/cortexcpp-betaNighty:
/usr/local/bin/cortexcpp-nightlyConfiguration file:
Stable:
/Users/<username>/.cortexrcBeta:
/Users/<username>/.cortexrc-betaNighty:
/Users/<username>/.cortexrc-nightlyData folder (contains engines, models, logs)
Stable:
/Users/<username>/.cortexcppBeta:
/Users/<username>/.cortexcpp-betaNighty:
/Users/<username>/.cortexcpp-nightly
Thank you @dan-homebrew
Beta Was this translation helpful? Give feedback.
All reactions
-
🎉 1
-
Task to do here #1030
Beta Was this translation helpful? Give feedback.
All reactions
-
For installer, llamacpp engine will be installed by default
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 Can I clarify my understanding of the Installer, as there are a couple of things that are not aligned from our discussion last week:
Last week's discussion: https://link.excalidraw.com/l/kFY0dI05mm/3rpQK5NI8Xb
Installer Contents
- Cortex will ship with llama.cpp as a default engine
- TensorRT-LLM and ONNX will (for now) be initialized by running
cortex init <engine>
Installation Methods
For the 3 operating systems, my understanding was as follows:
- Windows:
.exe - Mac:
.dmg - Linux: Install script (we would handle
.deband.appimagelater on), in the format of:
curl -fsSL https://get.cortex.so -o get-cortex.sh
From the post above, it seems like we're going directly to .deb and .appImage.
My question: Does the .deb and .appImage wrap the Installer Script? There are definite advantages to having a .deb and .appimage, I just want to know how much more effort it is.
Installation Logic
My understanding is that our Installation logic focuses on two tasks:
- Hardware Detection (e.g. GPU type, and CPU instruction set - e.g. AVX2, AVX512)
- Pulling the correct version of llama.cpp at install time
Alternatives
I am increasingly thinking that it is a better idea for us to pack a larger installer:
- Cortex will be used in offline environments (e.g. USB install), and packing a full installer is better
- We can delete the unused llama.cpp versions during the installation process, to optimize long-term disk usage
- This seems to be the approach taken by the other llama.cpp UIs and libraries (e.g. Ollama, LMStudio)
Beta Was this translation helpful? Give feedback.
All reactions
-
My question: Does the
.deband.appImagewrap the Installer Script? There are definite advantages to having a.deband.appimage, I just want to know how much more effort it is.
This is an example of template structure for .deb file in linux
image
Yes, it wraps bash scripts and allows us to interact with hooks like pre-install, post-install, pre-uninstall, and post-uninstall. It also has installation and uninstallation functions. If we go with a bash script, then we have to write all of the above functions ourselves.
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 Ok, let's go with .deb and .appimage then.
Beta Was this translation helpful? Give feedback.
All reactions
-
Link the final decision for naming here #1092
Beta Was this translation helpful? Give feedback.
All reactions
-
@hiento09 one of our macOS users found Jan files in
~/Library/Preferences/ByHost/jan.ai.app.ShipIt.<UUID>.plist
~/Library/Preferences/jan.ai.app.plist
- Is this something that is automatically cleared by OS when we uninstall on mac
- Will we need to take care of this dangling file.
- File naming wise, is it supposed to be inverse , i.e.
ai.jan.app...
For 3: see examples:
com.microsoft.VSCode.ShipIt.UUID.plist org.whispersystems.signal-desktop.ShipIt.UUID.plist
Beta Was this translation helpful? Give feedback.
All reactions
-
@0xSage , cc @louis-jan
Is this something that is automatically cleared by OS when we uninstall on mac=> YesWill we need to take care of this dangling file.=> No, it's a part of Jan app and is taking care byelectron-builderFile naming wise, is it supposed to be inverse , i.e. ai.jan.app...=> Yes, we can change to whatever name, just notify that change app ID will cause auto-updater crash
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.