[Feature Request] Stop Storing Meta Data In the .md file itself. (SQLite Database?) #174

JKontaro22 commented 2026年06月26日 02:40:19 +02:00

Specifically, I'm talking about the invisible "headers" that get added to files saved in HelixNotes. There are a few reasons why I think this is bad.

image

Reasoning

1. Helix Modified Files Invisibly - Thankfully, I tested this in a folder outside my main notes folder, and I'm glad I did. Helix does not state that it adds these headers as soon as you save the file. And it is not obvious that this data is being written at all. The problem isn't just aesthetics. My markdown notes are used for other programs to reference, and the added extra data for it to ignore or discard caused problems. For AI prompt instructions, the extra metadata adds to the token usage count, which is "fine" if the AI is local, but not if I gotta pay for the tokens. This would have been a very unpleasant surprise.

2. Reduces Portability - Both Obsidian and FOAM (VS code extension) do not add extra data to the files itself. Other people I collaborate use these editors, and will see the extra meta data when they use their editor of choice. Some collaborators I work with only work in plaintext (notepad++, NeoVim). I guarantee this is going to cause problems down the line should I use HelixNotes. I need to be able to work with people who use other editors without friction. I don't want to be the guy who gives people files "with weird garbage at the top".

3. It's Ugly - I think you also agree it's ugly, because you hide it by default when using the WSYWIG editor and the Source editor. Had I not sent a file to another person, it might have been a while before I discovered this quirk because it's hidden so well.

Considerations

SQLite

I don't now exactly how best to store this meta information outside of the file itself, but you could consider an SQLite database embedded into the program. I'm told it's very fast and self contained and has over 1 Trillion installs.

Apparently apple uses it in their "notes" app and Microsoft uses it in their "Sticky notes" app, among other things.

Turso (Written in Rust)

I've noticed that there is a significant portion of this program that is written in Rust. Turso is a Rust rewrite of SQLite. I've heard nice things about it, but don't know if it's any good. It's opensource (MIT) so I don't think mixing the code would be a problem.

Worth considering.

Concluding thoughts

Currently the way metadata is handled is a blocker to me using the app at all, but I like it so much. Most of your app is really well thought out, and it's really pleasant trying to onboard myself. I really think you have a good vision for this. I just selfishly need it to be more of the thing I want specifically.

Specifically, I'm talking about the invisible "headers" that get added to files saved in HelixNotes. There are a few reasons why I think this is bad.  ## Reasoning 1. **Helix Modified Files Invisibly** - Thankfully, I tested this in a folder outside my main notes folder, and I'm glad I did. Helix does not state that it adds these headers as soon as you save the file. And it is not obvious that this data is being written at all. The problem isn't just aesthetics. My markdown notes are used for other programs to reference, and the added extra data for it to ignore or discard caused problems. For AI prompt instructions, the extra metadata adds to the token usage count, which is "fine" if the AI is local, but not if I gotta pay for the tokens. This would have been a very unpleasant surprise. 2. **Reduces Portability** - Both Obsidian and FOAM (VS code extension) do not add extra data to the files itself. Other people I collaborate use these editors, and will see the extra meta data when they use their editor of choice. Some collaborators I work with only work in plaintext (notepad++, NeoVim). I guarantee this is going to cause problems down the line should I use HelixNotes. I need to be able to work with people who use other editors without friction. I don't want to be the guy who gives people files "with weird garbage at the top". 3. **It's Ugly** - I think you also agree it's ugly, because you hide it by default when using the WSYWIG editor **_and_** the Source editor. Had I not sent a file to another person, it might have been a while before I discovered this quirk because it's hidden so well. ## Considerations ### SQLite I don't now exactly how best to store this meta information outside of the file itself, but you could consider an [SQLite](https://sqlite.org/index.html) database embedded into the program. I'm told it's very fast and self contained and has over 1 Trillion installs. Apparently apple uses it in their "notes" app and Microsoft uses it in their "Sticky notes" app, among other things. ### Turso (Written in Rust) I've noticed that there is a significant portion of this program that is written in Rust. [Turso](https://github.com/tursodatabase/turso) is a Rust rewrite of SQLite. I've heard nice things about it, but don't know if it's any good. It's opensource (MIT) so I don't think mixing the code would be a problem. Worth considering. ## Concluding thoughts Currently the way metadata is handled is a blocker to me using the app at all, but I like it so much. Most of your app is really well thought out, and it's really pleasant trying to onboard myself. I really think you have a good vision for this. I just selfishly need it to be more of the thing I want specifically.

davi-jorge-art commented 2026年06月26日 20:57:48 +02:00

I don't know if it's a place have any discussions, but I disagree from start to end. I'm not a developer, maybe just (MAYBE) a "power user" but the "problem" you are rising is exactly the beauty and the core philosophy that a lot others apps and this one are using.
Before the PC's became affordable what has remained from my ideas, references, projects, etc gathered on more than a decade is box full of paper pocket notebooks that I'm sure I will never have the life time to search into .
Then I've became marvelled with the note-taking apps since the "palm era", around 2000.... The idea that I will always have my data reachable and indexed , form the day one I make the first memo till the last ...To have the opportunity easy access and mix ideas from totally different moments of my life....It's the perfect ground for creating new things, understand from where I've came to reach where I'm now....
But it became a enormous frustration when faster than I can follow up, the technology evolved in a way that the notes from my first palm became as useful as the words I've wrote with a pen on my notebooks...The hardware became outdated, alongside the softwares that are abandoned or all of a sudden asking the user to pay a month subscription (on US dollars ...).

The best "Export" of my references are some text files scattered throughout folders ...And it when I was lucky...Sometimes the "export" was only readable by the software that I cannot even install on my new hardware....

To have no database at all, only data wrote on a file that any other app can read and index, with only some modifications and "show" to me with the information that I actually want is the final solution I've thought had appeared more then 25 years ago...

The considerations about "hidden" stuff is not considerable...ALL editing software, after plain text editors, "hide" A LOT of data , sometimes in way no human can read... If I send some edited text file to someone the other person have to open it on the same editor...Because of it was created de PDF files... And even on this case the person that receives the document have at least to have a PDF reader...Nobody sends "raw" text to nobody...Unless it is plain text...It's the idea behind markdown ...Even I , with my bare eyes can understand that the example you've post has been created on 25 , June ...

I have lost a lot of my time explaining the obvious...I don't know your proficiency on software development . What I know is my experience as user... I've tried another app . I've made some notes. Don't liked it. Exported as .md, Put the exported folder on the Helix vault and they appeared on Helix...Simple as that..I can "carry wit me" Helix notes...Just folders on a pendrive, download the software anywhere : Done. And a lot of other benefits far from "database approach"....

I don't know if you will reply to this. I don't know how deep you can go on academic theory.,,,Probably I've made some factual or conceptual mistakes.... But I know by the core content of your "issue" that no matter how logical, theoretical or just reasonable I can refute any of your previous or further arguments you will not see the core mistakes that you are into....Then I'll not read or answer any word you refer to me or about this subject.

By the way , if I write like (削除) this (削除ここまで) there's "hidden" signals before and at the end of the word "this". The "ugly" truth behind you are seen it strikethrough.

I don't know if it's a place have any discussions, but I disagree from start to end. I'm not a developer, maybe just (MAYBE) a "power user" but the "problem" you are rising is exactly the beauty and the core philosophy that a lot others apps and this one are using. Before the PC's became affordable what has remained from my ideas, references, projects, etc gathered on more than a decade is box full of paper pocket notebooks that I'm sure I will never have the life time to search into . Then I've became marvelled with the note-taking apps since the "palm era", around 2000.... The idea that I will always have my data reachable and indexed , form the day one I make the first memo till the last ...To have the opportunity easy access and mix ideas from totally different moments of my life....It's the perfect ground for creating new things, understand from where I've came to reach where I'm now.... But it became a enormous frustration when faster than I can follow up, the technology evolved in a way that the notes from my first palm became as useful as the words I've wrote with a pen on my notebooks...The hardware became outdated, alongside the softwares that are abandoned or all of a sudden asking the user to pay a month subscription (on US dollars ...). The best "Export" of my references are some text files scattered throughout folders ...And it when I was lucky...Sometimes the "export" was only readable by the software that I cannot even install on my new hardware.... To have no database at all, only data wrote on a file that any other app can read and index, with only some modifications and "show" to me with the information that I actually want is the final solution I've thought had appeared more then 25 years ago... The considerations about "hidden" stuff is not considerable...ALL editing software, after plain text editors, "hide" A LOT of data , sometimes in way no human can read... If I send some edited text file to someone the other person have to open it on the same editor...Because of it was created de PDF files... And even on this case the person that receives the document have at least to have a PDF reader...Nobody sends "raw" text to nobody...Unless it is plain text...It's the idea behind markdown ...Even I , with my bare eyes can understand that the example you've post has been created on 25 , June ... I have lost a lot of my time explaining the obvious...I don't know your proficiency on software development . What I know is my experience as user... I've tried another app . I've made some notes. Don't liked it. Exported as .md, Put the exported folder on the Helix vault and they appeared on Helix...Simple as that..I can "carry wit me" Helix notes...Just folders on a pendrive, download the software anywhere : Done. And a lot of other benefits far from "database approach".... I don't know if you will reply to this. I don't know how deep you can go on academic theory.,,,Probably I've made some factual or conceptual mistakes.... But I know by the core content of your "issue" that no matter how logical, theoretical or just reasonable I can refute any of your previous or further arguments you will not see the core mistakes that you are into....Then I'll not read or answer any word you refer to me or about this subject. By the way , if I write like ~this~ there's "hidden" signals before and at the end of the word "this". The "ugly" truth behind you are seen it strikethrough.

JKontaro22 commented 2026年06月26日 22:33:13 +02:00

@davi-jorge-art,

It is a shame you won't read this, but for others, I'd like to clarify a few things based on points raised by davi-jorge-art

To have no database at all, only data wrote on a file that any other app can read and index, with only some modifications and "show" to me with the information that I actually want is the final solution I've thought had appeared more then 25 years ago...

I'm not asking for the markdown files to be put into a database. I only want the things that HelixNotes uses to build on top of .md files put into a database (or some other data structure that makes sense). When HelixNotes adds extra data to .md files that isn't used by any other program, it makes the files fill with "garbage" that would need to be removed if the .md files needed to be moved to another system or interpreted by another program (LLM, etc). The niceties that HelixNotes provides (the Visual view instead of the Source view) are things that are built on top of your files and are bespoke ONLY to HelixNotes.

For example, by default backlinks are not something built into the .md spec. That's something HelixNotes provides. It's really useful, which is why I want to potentially use this software, but that's something HelixNotes needs to manage itself without getting me or my files involved. As soon as HelixNotes starts writing things I don't want into MY files without telling me, that's when it starts putting a burden on me instead of helping me.

As for how this meta data is stored, I don't particularly care how it's stored as long as it's not in MY files. I also don't want a meta data file hidden but placed next to my files (eg. MyNote.md.helix in the same directory as the note.).

The considerations about "hidden" stuff is not considerable...ALL editing software, after plain text editors, "hide" A LOT of data , sometimes in way no human can read... If I send some edited text file to someone the other person have to open it on the same editor...Because of it was created de PDF files... And even on this case the person that receives the document have at least to have a PDF reader...Nobody sends "raw" text to nobody...Unless it is plain text...It's the idea behind markdown ...

I am not asking for my markdown (.md) files to be turned into binary files like a .PDF, nor am I asking for these theoretical files to be stored in a database. I need to be able to send my .md file without the extra bits HelixNotes puts in there that nobody else wants just because it's required by my editor.

The metadata HelixNotes Writes is in plain text.

Even I , with my bare eyes can understand that the example you've post has been created on 25 , June ...
I have lost a lot of my time explaining the obvious...I don't know your proficiency on software development . What I know is my experience as user...

I have been using Markdown personally and professionally for 8 years. I understand the power of raw text. It is the reason I write things in markdown. That picture is from when I was doing my testing yesterday (2026年06月25日), and discovered what HelixNotes was doing to my files. HelixNotes will also add those lines with no modifications if you attempt to save an unedited file. I had to open that .md file in a different text editor (Notepad++) to see that any text was written at all. Just to be clear Notepad++ is a plaintext editor, also capable of writing markdown files.

The reason I don't use Notepad++ for writing my markdown is because it doesn't support backlinks.

By the way , if I write like (削除) this (削除ここまで) there's "hidden" signals before and at the end of the word "this". The "ugly" truth behind you are seen it strikethrough.

You are confusing the Rendering of markdown text in a browser with the "raw text". I am very much aware of how things like Bold and (削除) Strikethrough (削除ここまで) work when they are redered by an HTML markdown interpreter. The thing you fail to understand is that rending the strikethrough as you see it is done By the browser, not the .md files themselves, nor a plain text editor. If you are implying that the browser is hiding the raw text from you, I guess you are sort of correct, but that doesn't mean I don't have to use ~~symbols~~ to write things using (削除) Strikethrough (削除ここまで).

My point still stands, the cataloging of backlinks is handled by HelixNote, and it should not place a burden on me or my files for Helix note to function properly. Pretending the burden isn't there is the problem.

@davi-jorge-art, It is a shame you won't read this, but for others, I'd like to clarify a few things based on points raised by davi-jorge-art > To have no database at all, only data wrote on a file that any other app can read and index, with only some modifications and "show" to me with the information that I actually want is the final solution I've thought had appeared more then 25 years ago... I'm not asking for the markdown files to be put into a database. I only want the things that HelixNotes uses to build on top of .md files put into a database (or some other data structure that makes sense). When HelixNotes adds extra data to .md files that isn't used by any other program, it makes the files fill with "garbage" that would need to be removed if the .md files needed to be moved to another system or interpreted by another program (LLM, etc). The niceties that HelixNotes provides (the Visual view instead of the Source view) are things that are built on top of your files and are bespoke **ONLY to HelixNotes**. For example, by default backlinks are not something built into the .md spec. That's something HelixNotes provides. It's really useful, which is why I want to potentially use this software, but that's something HelixNotes needs to manage itself without getting me or my files involved. As soon as HelixNotes starts writing things I don't want into **MY** files ***without telling me***, that's when it starts putting a burden on me instead of helping me. As for how this meta data is stored, I don't particularly care how it's stored as long as it's **not in MY files**. I also don't want a meta data file hidden but placed next to my files (eg. MyNote.md.helix in the same directory as the note.). >The considerations about "hidden" stuff is not considerable...ALL editing software, after plain text editors, "hide" A LOT of data , sometimes in way no human can read... If I send some edited text file to someone the other person have to open it on the same editor...Because of it was created de PDF files... And even on this case the person that receives the document have at least to have a PDF reader...Nobody sends "raw" text to nobody...Unless it is plain text...It's the idea behind markdown ... I am not asking for my markdown (.md) files to be turned into binary files like a .PDF, nor am I asking for these theoretical files to be stored in a database. I need to be able to send my .md file without the extra bits HelixNotes puts in there that nobody else wants just because it's required by my editor. The metadata HelixNotes Writes is in plain text. > Even I , with my bare eyes can understand that the example you've post has been created on 25 , June ... > I have lost a lot of my time explaining the obvious...I don't know your proficiency on software development . What I know is my experience as user... I have been using Markdown personally and professionally for 8 years. I understand the power of raw text. It is the reason I write things in markdown. That picture is from when I was doing my testing yesterday (2026年06月25日), and discovered what HelixNotes was doing to my files. HelixNotes will also add those lines with no modifications if you attempt to save an unedited file. I had to open that .md file in a different text editor (Notepad++) to see that any text was written at all. Just to be clear Notepad++ is a plaintext editor, also capable of writing markdown files. The reason I don't use Notepad++ for writing my markdown is because it doesn't support backlinks. > By the way , if I write like ~~this~~ there's "hidden" signals before and at the end of the word "this". The "ugly" truth behind you are seen it strikethrough. You are confusing the **Rendering** of markdown text in a browser with the "raw text". I am very much aware of how things like **Bold** and ~~Strikethrough~~ work when they are redered by an HTML markdown interpreter. The thing you fail to understand is that rending the strikethrough as you see it is done **By the browser**, not the .md files themselves, nor a plain text editor. If you are implying that the browser is hiding the raw text from you, I guess you are sort of correct, but that doesn't mean I don't have to use \~~symbols~~ to write things using ~~Strikethrough~~. My point still stands, the cataloging of backlinks is handled **by HelixNote**, and it should not place a burden on me or my files for Helix note to function properly. Pretending the burden isn't there is the problem.

ArkHost commented 2026年06月26日 23:48:00 +02:00

Going to pass on this. That's standard YAML frontmatter, not a HelixNotes invention (Obsidian uses it too). It's the interoperable choice, not lock-in.

A SQLite database would do the opposite of what you want: your .md files would lose their metadata the second you move, sync, or back them up as plain files, and a database can corrupt where plain text can't. Metadata living in the file, human-readable, is the whole point. It's what lets your notes outlive the app. That's a deliberate design decision, and it's staying.

Closing this issue.

Going to pass on this. That's standard YAML frontmatter, not a HelixNotes invention (Obsidian uses it too). It's the interoperable choice, not lock-in. A SQLite database would do the opposite of what you want: your .md files would lose their metadata the second you move, sync, or back them up as plain files, and a database can corrupt where plain text can't. Metadata living in the file, human-readable, is the whole point. It's what lets your notes outlive the app. That's a deliberate design decision, and it's staying. Closing this issue.