-
Notifications
You must be signed in to change notification settings - Fork 14
Triadic Memory implementation in Odin #7
-
Hi,
I found the Triadic Memory idea interesting, so I implemented a version of it in Odin (a C-replacement language).
It follows the C version for the most part. So far it only includes a basic test of the temporal memory, more to come.
Check it out here
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 1 comment 3 replies
-
That's really cool! Please send a pull request, and I'll include the Odin version in the main repository...
Beta Was this translation helpful? Give feedback.
All reactions
-
Hi again, thanks for accepting the PR. I have some additional updates.
I found that the triadic memory itself works quite well, but the temporal memory (at least as it was in the C version) doesn't work very well. So, I made my own variant. It is available here.. It seems to predict much better overall. It doesn't OR together SDR's at all, instead it auto-encodes both the input and the context (previous hidden) into a new hidden state. The M2 module is similar to before. I also found that the "early stop" on the learning leads to decreased performance, so I removed it and capped the triadic memory values to prevent overflow. It still uses random anchor SDRs, which works for now but should probably be improved to handle local sensitivity better.
The new main.odin loads a text file and runs this new temporal memory through it, character by character. It then generates more text. It works, although not to the level of my AI system (AOgmaNeo) quite yet (which can remember 80,000 digits of pi). Still, it was fun to experiment with, and I look forward to experimenting more with it soon.
Beta Was this translation helpful? Give feedback.
All reactions
-
I think what you're doing in your variant is building some kind of a linked list. With this, you can store any sequence, only limited by memory capacity. Can it make a prediction starting somewhere in the middle of a sequence?
Beta Was this translation helpful? Give feedback.
All reactions
-
I haven't tested it yet on that specifically, but I don't see why not
Beta Was this translation helpful? Give feedback.