Declarative DSLs

This library presents declarative DSLs for Janet, based around dataframes (hashmaps with indexed values {:key [1 2 3] :key2 ["v" "a" "l"]}.) The tests surface many example programs.

Query Syntax

This SQL-inspired query syntax can access hashmaps, arrays, strings and dataframes:

• (df/select :from (range 10) :where odd? :where (partial < 5)) => @[7 9]
• (df/select :from "my favorite animal is..." :where |(has-value? "aeiou" $)) returns: "aoieaiai"
• (df/print-as-table (df/select :from {:a 1 :b 2} :cross {:b 2 :c 3})) returns:

key_right value_right key_left value_left
--------- ----------- -------- ----------
b 2 b 2 
c 3 b 2 
b 2 a 1 
c 3 a 1 

An example program:

(importdeclarative-dsls/dataframes:asdf)(defpeople(df/dataframe:name:age:job))(df/dataframe?people)(df/insert!{:name"Bob":age30:job"Developer"}:intopeople)(df/insert!{:name"Alice":age27:job"Sales"}:intopeople)(df/update!:set{:job"Engineer"}:where|(=($:job)"Developer"):frompeople)(df/save-csvpeople"people.csv":sep"\\t")(defpeople2(df/load-csv"people.csv":sep"\\t"))(->people2df/dataframe->rowsdf/rows->dataframedf/print-as-table)

Prints:

job age name 
-------- --- -----
Engineer 30 Bob 
Sales 27 Alice

Select also has :cross for cartesian products, :join, :exclude, :sort ... :by, :union, :distinct, :limit...

Other useful functions: populate-df!, delete!, dataframe, dataframe?, make-insert-func

The v (for vectorize) macro lets you:

(importdeclarative-dsls/dataframes:asdf)(df/v+[123]1[123]1)# returns: [4 6 8](df/v+1{:column[123]:key[123]})# returns: {:column @[2 3 4] :key @[2 3 4]}(df/v*[123][[111][122][123]])# returns: @[@[1 1 1] @[2 4 4] @[3 6 9]]

Carried away, I implemented window functions etc. in data-analysis.janet which don't quite fit the theme of declarative-dsls:

(importdeclarative-dsls/dataframes:asdf)(importdeclarative-dsls/data-analysis:asdf)# This file requires a separate import(defraw-transaction-log# long form@{:store@["A""A""A""B""B""A""A""B""B""B"]:month@["jan""jan""feb""jan""jan""feb""feb""feb""feb""feb"]:amount@[100502008020150906040100]})# Rank stores by total revenue(defby-store(df/aggregateraw-transaction-log:store{:revenue[:amountsum]}))(df/window!by-store:placedf/rank:revenue)# Make store x month grid (like for a report)(defmonthly(df/aggregate(df/column-combineraw-transaction-log:store-x-month[:store:month]|(string($0)"-"($1))):store-x-month{:total[:amountsum]}))# In our next example, df/aggregate and df/window! # will transform this chart:revenueproductregionqtyline------------------------------30widgeteast10basic48gadgeteast4basic21widgetwest7basic50gizmoeast2premium72gadgetwest6basic15widgetwest5basic25gizmoeast1premium36gadgetwest3basic(defby-region(df/aggregatepriced:region# rank regional revenues{:units[:qtysum]:revenue[:revenuesum]}))(df/window!by-region:rankdf/rank:revenue)rankregionunitsrevenue----------------------1west211442east17153

Botom Up Logic via l/fixpoint!

This datalog inspired DSL stores knowledge as facts/triples in a knowledge base dataframe like {:subject :predicate :object} stored at (dyn :logic-kb) set with with-kb or set-kb!. Symbols starting with ? are logic variables, _ is wildcard. (doc an-explanation-of-logic.janet) explains the thought process of much of this and the next from the REPL. Strings and bare symbols are coerced into keywords.

(importdeclarative-dsls/logic:asdl)(defkb(dl/new-kb))(dl/with-kbkb(dl/assert-facts!["alice""parent""bob"]["bob""parent""carol"]["carol""parent""dave"])# ancestor: direct parent OR parent of an ancestor(dl/defrelancestor[?x?z](or("parent"?x?z)(and("parent"?x?mid)(ancestor?mid?z))))(dl/print-as-table(dl/query[?ancestor?descendent](ancestor?ancestor?descendent)))(dl/assert-facts!["dave""parent""alice"])# recursive cycles now(dl/print-as-table(dl/query[?ancestor?descendent](ancestor?ancestor?descendent))))

datalog.janet exposes these symbols: dl/query dl/goal-relation, dl/fixpoint!, dl/new-kb, dl/fact?

If you import datalog.janet or minikanren.janet without (or before) dataframes.janet, 5 dataframe functions will be imported print-as-table select, populate-df!, rows->dataframe and dataframe->rows. Similarly, these from logic.janet are also exposed: set-kb!, with-kb, current-kb, assert-facts!, defrel, retract!, assert-rules!

Top Down Logic a la miniKanren

This DSL works on the same knowledge base and API as above, but top down via lazy mk/run with lazy interleaving. It currently lacks sufficient documentation/example tests.

minikanren.janet exposes these symbols: mk/fail, mk/dataframe->stream, mk/defgoal, mk/run, mk/once, mk/disj, mk/project, mk/absento, mk/==, mk/!=, mk/fresh, mk/stringo, mk/symbolo, mk/relation , mk/conj, mk/datalog->minikanren, mk/numbero, mk/succeed and mk/conde

Glossary

• relation/rel - named array set (Datalog/Prolog "predicate")
• goal - func transforming df (var bindings) to df satisfying something
• fact - array in a relation without variables
• rule - head (conclusion) and body (premise)
• fixpoint - state where applying rules doesn't make new facts
• term - an atom's args
• atom -
• entry - @{:columns :edb :seen ...} bindings

Further info about Packages

These functions add data: l/assert-rules!, l/declare-relation!, l/assert-facts!, df/populate-df!, df/insert!, df/update! and df/rows->dataframe

TODO: split :db to :facts (for the asserted+derived facts) and :relations in the kb

TODO: determine naming conventions

TODO: migrating most uses of "predicate" to "relation"

So "predicate" suggests lispy odd? string? type funcs. Prolog also used "functor" and "table" could even work

Old Docs:

As Datalog lends itself to eager columnar representation, the core approaches Datalog with some miniKanren thrown on top (which uses on lazy streams (fibers) and rows.)

tl;dr: Make a knowledge base with new-kb set it with set-kb! or with-kb then populate it with assert-facts! and defrel, which you an then query

(use ./logic) (def kb (new-kb)) (with-kb kb (assert-facts! ["alice" :parent "bob"]) (defrel ancestor [?x ?z] (or (:parent ?x ?z) (and (:parent ?x ?mid) (ancestor ?mid ?z)))) (query [?x] (ancestor ?x "bob")))

Core (Datalog-ish) concepts:

Knowledge base (KB) stores knowledge as facts/triples in a dataframe like {:subject :predicate :object} keys, where predicate (relationship) and object (target/value). For example: {:subject "Bob" :predicate :friend-of :object "Jack"} The current-kb is at (dyn :logic-kb) set with (with-kb kb) for scoped use (set-kb! kb). (current-kb) returns it.

Symbols starting with ? are logic variables, _ is the wildcard _ which gets ignored. The engine looks for values for the logic variables making the goals true

Call relations like: (name s o) where bare symbol names become keywords. (strings or keywords stay the same), ?var is used as a logic variable/predicate, or evaluated if it's an unquoted ,expr. Args follow the same rules: ?x and _ are logic vars, bare symbols are Janet bindings looked up at call time, :keyword, "string" and numbers are constants, and ,expr unquotes.

Goals are search criteria e.g. (friend-of ?x "Bob") searches the DB for every 1st value where the 2nd and 3rd parts are true, i.e. Bob's friends. Goals are relational operations taking a set of variable substitutions and returning new substitutions.

Rules derive new facts from existing ones. For example:

(defrel good-pet? [?x]
 (is ?x "pet")
 (not (has ?x "flea")))

fixpoint! applies all rules to the DB to derive/hydrate all facts, recursively

walk-goal-dsl is the heart of this engine

Body clauses use (name s o) form, where inputs are: - bare-symbol - keyword predicate (:bare-symbol) - :keyword-name - no change (:parent) - ?logic-var - variable predicate - ,name - dynamic predicate

Args follow compile-term rules:

• ?x/_ - logic vars
• bare-symbol - Janet bindings (e.g. a function's params)
• literals - constants
• ,expr - unquotes

miniKanren concepts:

• Top-down, goal-directed search with lazy interleaving (while fixpoint! is bottom-up)
• Goals are (fn [binding] -> stream-of-subs)
• Streams are Janet fibers yielding substitution maps
• == / != are used in term unification
• fresh / conde for scoped variable introduction and fair disjunction
• run returns n results of a query (or all with *)
• triple for miniKanren goals
• project escapes to Janet on walked variables for e.g. math operations