Popequer

Popequer is a way to organize your digital life around a text-based, entry-based, personal knownledge management system.

Built to be flexible, one of the initial goal is to have a way to edit and contribute to wikidata.org using text syntax.

But the goal is to be able to use it later as a collaborative knowledge management system to eventually replace wikibase.

This project is currently at the very begining with only the parser for now.

The popequer project currently have a unique mono-repo.

Components

• The popequer data entry language parser (PDEL)
  • Parse an entry to an intermediate Entry representation (IER)
• The wikidata interface:
  • The wikidata to PDEL translator lib
  • The parsed PDEL to wikidata translator lib
  • The normal wikidata client
  • The wikidata CLI editor that use the libs
• The Database/Indexer:
  • Directory crawler, markdown parser and indexer
  • Convertion from IER to a rich, validated and linked database
  • The query interpretor (that use an adaptation of the PDEL for the query language)
  • Indexer CLI
  • Query CLI
• User Front-end
  • Calendar view
  • Events/History view (Activity view)
  • Adaptative view (for any kind of relational and general database)
• Convertion utils from and to common formats (CSV, JSON, etc)
• Import/Export interface for specific uses
  • CalDAV interface
  • CardDAV interface
  • YouTube interface
  • E-Mail interface (IMAP)
  • File system interface
  • Any kind of interface to foreign software to pull activity data
• Popequer notebook day-to-day CLI helper

External components:

• Android app
• Desktop visualizer

Quick reference

To only run one test with stdout print not hidden:

cargo test parse::test_parse::test_parse_entry_claim -- --nocapture --exact

The PDEL Parser

We will introduce a new markup language specification or format The purpose is to ease and simplify the creation, edition and management of wikibase-like items This boils down to being able to quickly make claims using human usable identifier or key Instead of remembering for each wikibase property an Id, we introduce a mapping between the key and the property id We will also introduce simple syntaxic sugar to prevent redundancy in our syntax It's a simpler form of the notation3 syntax which aims to a readable RDF syntax We aims to keep as close as possible to the RDF models but with specificities of wikibase in mind To put it in a nutshell, we want to create a language to edit wikibase with the same ease and practicality of the wikibase web interface.

The popequer database and data model

Compatible with the wikibase, the data model is relatively simple and allow to easily model any kind of knowledge.

The database must be design to allow for graph-like query search. Like: what are the friends of my friend? We could use neo4j but it's not very embededd. We could study what makes neo4j interesting as a graph database.

Ideally I would like to use an embeded graph-database

All datetime are UTC only (at least internally).

TODO

• most basic database

  • write: encode popequer bunch of objects into binary file
  • read: decode binaries and have a way to view them
  • only a index on ID field to quickly find later
  • no SQL

• basic query CLI feature

  • search function that take O(n) on pure entries with basic filter language using claim property

• implement auto_labels function for Human, Events and things...

• add macro or shortcut

• handle one language quick labels as (name:, description:, alias:)

• !labels shortcut expansion

• function to take a ParsedEntry with details and turn it to a Entry struct removing every intermediate ParseOutput

• handle recursive entries definition.

• handle entry header shortcut like @Human, or @Event, or @Account, @Attendance, used to skip the use of is: claim

• fix macro cursor offset, but how?

• parse_extractor: add more errors

• parse_extractor: add test for Geo function

• indexing: add entry mutation (to add new ID) on new entry (have the indexer add a ID after first discovery of the items)

  • the requirement for that was to have the whole syntaxical tree available in the parse output of an entry
  • we now have that, we can now safely add in text the claim: id: E000001, that way we can identify each item later for update

• adapter: add vcard contacts export

• easy link to wikidata

• advanced db features

  • construct custom indexs (eg. index for geocoords)

Definition of the MVP

There are two versions of the MVP

MVP for personal use with neovim

• Rust? treesitter extension that use the same codebase would be great
• Rust LSP server on a notebook
  • Implement LSP "GoTo" feature to resolve references
  • Implement LSP
  • Provide completion of properties AND items name
• Neovim plugin to add all the useful stuff
• Basic CLI util
  • launch an indexation and update of the DB
  • get the status

MVP for wikidata use with neovim

Wikidata editor

Getting started as a user

use the indexer

cargo run --bin indexer -- -s ./examples/notebook/ -i ./.tmp/notebook_internals/

2023年08月10日

placement de l'ids, les labels

reflexions sur le placement de l'id, du nom et de la description

dans un notebook classique, on va avoir classiquement seulement qu'un seul language donc c'est un peu superflut d'avoir le

il faut un moyen de ne pas avoir a preciser {en: , fr:} etc

pour l'instant on fait sans, car en vrai avec un snippet ça devrais le faire

moyen pour passer d'un objet ParsedEntry a Entry tout cours sans les wrapper de parsing

soit une macro qui reconstruit un Entry depuis un ParseEntry avec des references

interactions avec git

notre indexer va devoir demander a git les dates de modification de tel ou tel partie de texte corespondant a tel ou tel partie de texte a partir des dates de commits.

2023年11月13日

So okay, now with the !labels macro we have name, description, alias properties which are specials So we can in theory delete all the special labels function that parsed the claims

2023年11月20日

Okay now we have a way to take a Parsed Entry with complicated cursor attributes to a "Pure Entry" that only care about the abstractions.

Now another issue with macro expansion

if you have

{
 shit: !somemacro_expand_long(length: 500),
 another_code: 10
}

then it expand into

{
 shit: (
 50000000000000...0000000000000
 ),
 another_code: 10
}

the claim another_code: 10 is offset by 2 lines

possible fixes and workaround:

• fix with dummy whitespaces
• have some way to parse the macro at the same time as the others components

so for example, parse into claims