-
Notifications
You must be signed in to change notification settings - Fork 179
Spec: Cortex.cpp Logs #1112
-
Goal:
- Clear, detailed and configurable (levels, verbosity) logging
Spec:
- Dependency: Trantor with UDS (multi-process logging)
- Log levels:
trace,info,debug,warning,error,fatal - Log format: TBA
Maintenance:
- Log file rotation: when log file exceeds max size, will roll to new file with the same prefix: .00000.log -> .00001.log.
- Max log file size: 20 Mb.
- Log purging: TBA
TBA
- [TBA] verbose mode
Related
- bug: Multiprocess cannot write log to the same file #1056
- feat: CLI stdout logs are clean and readable #1038
- feat: Configurable Log path for main program and dependencies #1062
- feat: how to monitor cortex inbound requests? #1007
Open Questions:
- What logging library are we using? Stdlib or 3rd party and why?
- What log levels are supported?
- Is logging configurable by user?
- How do you handle log rotation and file size limits?
- Are logs formatted consistently? e.g. timestamp, source of issue, message
- How are we passing through upstream errors, e.g. llamacpp errors
- Where are logs stored?
- Is logging threadsafe? Are there multiple processes writing to the same logs?
- Is there a Cortex command to tail logs?
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 3 comments 2 replies
-
Log library
- we are using trantor for logging feature because:
- It is a Drogon's dependencies, so reuse trantor will save some resources
trantoris non-blocking I/O cross-platform, thread safe, async library.- Support multi streams logger (can chose to write log to different file, stream, ... )
- Support log file rotation and set max file size: when log file exceeds max size, will roll to new file with the same prefix:
<prefix>.00000.log-><prefix>.00001.log. Currently max log file size is 20 Mb. - Support all needed informations: for example
20240903 06:46:55.974670 UTC 1268127 INFO Request 1727: Inference completed - llama_engine.cc:827, it includes time, timezone, process id, log level, message, source code file and line number - Log levels:
traceinfodebugwarningerrorfatal
Configuration
Currently, log files are stored under /path_to_cortex_cpp_binary/logs/ but need to configurable, now we set it as default constant in source code
When this Feature is done, we will read from config to decide where to store log file.
Log upstreams
For upstreams like llama.cpp and tensorrt llm, we have to override log method of those and redirect to right posion as we want, this task is Done for llamacpp and tensorrt llm, refer to this issue
Issues
Trantor is thread safe but it keeps the log file open during the life time of server process. With Linux and MacOS, it works perfectly, but in windows, when a process is holding a file, windows prevents other process to access it. Refer to this Issue
Future features
- Support tail latest logfile
Beta Was this translation helpful? Give feedback.
All reactions
-
- Log rotation is fine, but what about purging. Previously, we've experienced cases when logs explode to xx Gb for users.
- Is there a global log file max
- Is there a max log expiry date, i.e. is there any use in keeping logs older than x Days/Weeks or Months
- How do we denote where the log originated, i.e. llamacpp vs cortex.cpp process?
- Does CLI have verbose / regular logging modes?
Beta Was this translation helpful? Give feedback.
All reactions
-
Also naming wise Im wondering if you should go with - instead of incremental ints
Beta Was this translation helpful? Give feedback.
All reactions
-
- Trantor provide methods to set max number of log file under log folder, Trantor will delete the oldest log file first when the maximum files exceed, but we need to experiment to confirm how it exactly works.
- Trantor doesn't support this feature.
- For now we are using the same log file to log everything (this is configurable), in the log file also contain the file name and line number, -> we can easily find the root cause.
- CLI has
--verboseflag, when this flag is provided, everything will log to terminal instead of files - For naming format the full structure of log file is .-.0000.log
Future change
When this Pr fix: correct path to work with data folder structure #1132 is finished, will move log folder to the data folder (currently, log folder in the same location with binary)
Beta Was this translation helpful? Give feedback.
All reactions
-
Do we log the underlying API requests, e.g. re obj, response
#1007
Beta Was this translation helpful? Give feedback.