Why shouldn't I be using SQLITE instead of the INI format in all cases?

Question

Backstory

In an effort to store partial and full downloads to file in a clean manner, I had utilized QSettings which stores its key value pairs in the INI format in a single file. https://doc.qt.io/qt-5/qsettings.html#Format-enum

After a short while of storing the downloaded file's byte array, the INI grew to over a gigabyte:

-rw-rw-r-- 1 anon anon 9.9K Mar 4 00:39 File.conf
-rw-rw-r-- 1 anon anon 1.2G Mar 4 00:35 Full.conf

The problem with QSettings now, that if I initialize "Full.conf", it takes 14 seconds just to do that, likely trying to parse everything, and calling functions on it is also a slog, I think because it has to rewrite the whole file every time.

Question

• Is this merely an implementation issue, or is the INI format simply fundamentally inappropriate for the storage of large bytearrays?

• If so; do INI files have any real niche purpose or use-case beyond its legacy status?

• I understand that it can store small things, but so can a million other formats. Why not deprecate INI files in favour of something similar such as SQLITE?

• Can't SQLITE after all, do everything the INI is doing, and far more?

• Caleth: How big was the download? Are you removing entries after you no longer need them?
• Doc Brown: "do INI files have any real niche purpose or use-case beyond its legacy status" - INI files or configuration files have still the same purposes as they had decades ago - providing a simple, lightweight way of configuring an application through a human-readable file which can be changed by using a standard text editor (try this with an SQLITE file, good luck). This is neither "niche" nor "legacy", I have no idea why you think it is.
• JimmyJames: @DocBrown +1 Doing a diff between two DBs is also a challenge. I'm not sure about SQLLite but I've found that databases require their data to be writable even when you only want to use them as read-only.
• pjc50: "INI format simply fundamentally inappropriate for the storage of large bytearrays": yes. Use the filesystem for storing a set of large bytearrays called "files".
• Doc Brown: @LieRyan: I think it could be perfectly suitable for what the OP tries. Let me cite this from SQLite - When to use: "SQLite does not compete with client/server databases. SQLite competes with fopen()". For storing the binary data of partial downloads in some block-orientend format, SQLite looks pretty reasonable to me. It may be oversized, of course, maybe a separate intermediate file would be sufficient, one would have to take a more in-depth view into the code.

Answer 1

I think the problem is your definition of application settings.

I think most people would designate your "partial and full downloads" as application state rather than application settings, which would commonly be a small set of configuration information which remains generally static

As to whether databases are better than flat files. Well, they are better suited for several use cases, and probably better suited for the use case described in the question, but this comes for a cost. Such as size on disk/memory of the database engine, non-human readablity of data etc.

"ini" files are usually better suited for small amounts of simple data, such as app settings (= configuration parameters of the application ). So they wouldn't be deprecated in favour of a database, even if in your senario a database might be a better choice.

• Caleth: Databases are more featureful than filesystems, but that isn't "better" in all situations
• JimmyJames: I disagree with the assertion that databases are obviously better than flat files. For many purposes, databases are inferior.
• Doc Brown: @Ewan: I think your answer is generally ok (+1), but by rewording it a little bit you could surely avoid some misunderstandings - that works usually better than telling people they should "read to end". I took the freedom to suggest an edit.
• Doc Brown: @Ewan: you forgot use the @, found your response right now. If you don't like my edit, feel free to revert it. But I hope your response was intended as a joke - if you really prefer a style which can easily lead to misunderstandings, I cannot help you with this.
• Ewan: i think you just proved my point.