Coder's Basilisk

CBAS logo

Coder's Basilisk (or "CBAS"/"Seabass") is my magnum opus, and my research project contributing to the Hackable Compiler Infrastructure Project.

Documentation (WIP): https://gek.codeberg.page/coders_basilisk_docs/

A tool with no equals

CBAS is a structured, high level language compiler which can be hacked and modified in ways that other metacompilation systems like Racket cannot.

How it works

Structurally

Coder's Basilisk is, at its very bottom, a C-like language.

A program is a combination of global variables, data statements (equivalent to const arrays), functions, and structs.

You can think of this list of global variables, data statements, functions, and structs as the "AST". These form the semantic structure of the program.

Within this AST, you have symbols and types which are "codegen" or "noexport" and those which are not.

Codegen and noexport symbols and types only exist while the compiler itself is running. The other symbols and types are implicitly defined as being for the "target" - a final compiled binary with no pieces of the compiler in it.

Procedurally

Lexing and Preprocessing

The source file(s) are lexed and preprocessed. The end result is a series of "lexer tokens".

These lexer tokens are just instances of a struct in a giant linked list.

Each instance represents a tiny piece of the source code.

(You can look up "Lexer tokens" online if you don't understand what this means)

Parsing

This is where things begin to get interesting.

Before I can explain what Coder's Basilisk does, I need to explain what a C compiler would do.

Imagine a traditional C recursive descent parser (see note 1). At global scope, you parse either declaration/predeclaration of a function or variable, a struct/typedef, etc.

As you parse these definitions, they get added to global arrays. You might have an array of functions, an array of variables, and an array of struct definitions, for instance.

After parsing is done, you do semantic analysis. All the functions are checked to see if they are actually syntactically and semantically correct. This is where you get fancy type error warnings and whatnot.

Then you move onto IR/Codegen. The C compiler would pass the parsed AST to an IR generator...

Do you get that? Good. Let's talk about what's different with Coder's Basilisk.

Coder's basilisk changes this system in several distinct ways:

1. Immediately after a function body is parsed, semantic analysis happens on it in isolation.

2. After semantic analysis is done, if the function is declared to be codegen, it will be compiled immediately into an executable format. References to predeclared but not defined functions and variables are embedded within this executable format, so that predeclaration works.

3. The parser can be made to directly call user-defined compiletime code. This is done through "parsehooks".

here is a tiny toy example:

fn codegen predecl callme;
fn codegen parsehook_demo:
	callme();
end
fn codegen callme:
	__builtin_puts("Hello from Parse Time!");
end
@demo
@demo
@demo
fn codegen codegen_main:
	__builtin_puts("Hi!")
end

The invocations of demo here are mid-parse. That's important.

4. In addition to parsehooks, there are also "validator hooks". These come in two flavors: Pre and post validator hooks. These will be explained later.

5. Compiletime code can recursively call the parser. That means compiletime code can ask the parser to parse a global variable declaration, struct declaration, or function declaration. These are automatically added to the global symbol table.

This can also be done piecewise - compiletime code can ask the parser to parse a type or expression and return the AST subtree to the caller as a live, editable data structure.

6. After parsing is done, instead of an internal code generation pipeline being called, the compiler calls a user-defined compiletime function called codegen_main. This function - like all compiletime code - has complete and unrestricted read/write access to the full compilation unit's AST. This editable AST includes the codegen symbols and noexport types.

Note 1: (Here is a prompt you can ask a chatbot to teach you recursive descent parsing: Hey claude/chatgpt, please write a tiny toy recursive descent parser calculator for me in plain C which I can compile and run immediately.

I want it to parse unsigned integers only. It should handle the operators *, /, +, and - with proper precedence. Infix notation.

The parser should receive text from stdin (just line-buffered input) and for each line, attempt to parse an expression.

If parsing fails, just print an error message.

Write comments in the code explaining how it works and what the functions are doing.

Explain afterward in your own words how this demonstrates recursive descent parsing, and what specifically "recursive descent parsing" is, using your example. ).

Parsehooks

Parsehooks, are the bread and butter of Coder's Basilisk's metaprogramming.

They are compiletime functions that the parser calls during its execution.

Parsehooks may read and modify the lexer token stream, access the parsed AST, and even read and modify live parser state. This is true of all compiletime code.

Validator Hooks

Pre-validator hooks are called before semantic analysis, type checking, and several other key AST operations done by the compiler's semantic analyzer.

This allows compiletime code to perform arbitrary AST transformations and process any custom AST nodes into finalized core CBAS AST nodes.

Post-validator hooks are called after this process is finished, but before codegen symbols are compiled into an executable format. Their purpose is mainly to enforce constraints. That's a fancy word for "rule you're not supposed to break in a programming language."

If you wanted, for instance, to make sure that a global variable was never modified, you could implement a validator hook which checked every place it was used and make sure no value was ever assigned to it, nor its pointer ever used.

Compiletime Code has complete control

codegen code in Coder's Basilisk is not like Rust Procedural Macros, or Racket meta-layers.

codegen code in Coder's Basilisk is meant to be a logical extension of the compiler itself - a complete compiler plugin API.

It can...

1. Get pointers to the compiler's own internal variables and read/modify their values, changing compilation state arbitrarily.

2. Access live parser state - the set of scopes the parser is inside of, for instance. This, combined with 3, allows macros to access the full compilation context of their invocation site in the very same way the compiler's own source code can.

3. Access the Full AST. This includes BOTH codegen and non-codegen symbols and types. Everything is there.

(2 and 3 are notably absent even in languages like Racket, which claim to be metaprogramming languages.)

4. Recursively call the main parser. This was mentioned before - you can ask the main parser to call itself again. Yes, the main parser is re-entrant.

5. Read and write arbitrary files. This is the main venue for parsing external source code and emitting arbitrary code out.

This means, you can...

1. Completely abandon the main parser and just parse an entirely new language. You can even write your own full languages with their own lexers using code strings (A special kind of string literal which works well for holding code, it won't look green in your text editor and allows nesting).

This language can then FFI perfectly with core Coder's Basilisk or anything else written for it. You can make it a metaprogramming language too, if you want.

2. Write tiny little tools. Domain specific languages, template programming systems, you name it. Notably, it can do this with full contextual awareness.

Your macros can enumerate the full AST and live parser state. This is not possible in Rust or Racket.

3. Write your own code generation. codegen_main is programmable. If you make a language inside compiletime CBAS, then you can now target every language ever.

4. Write your own constraint enforcement, static analysis, etc. Have you ever thought Rust's borrow checker could use a little work? Here's your chance. Go for it.

From a language development perspective

Coder's Basilisk is a direct iteration on FORTH's incremental compilation and dictionary model, applied to a C-like programming language.

My Gift to you

Coder's Basilisk is CC0 - see the LICENSE file. That means there are no attribution requirements, there are no burdensome license terms.

Coder's Basilisk is an unencumbered gift to humanity.

Dare not look into the eyes of the Basilisk
For behind his gaze dwells the power to create and destroy
The power to kill and the power to make alive

I have a discord server for those who are interested in the project:

https://discord.gg/PkejAd6RXk

But you don't have to join. This project is Public Domain (CC0). Just steal it!

A toolkit for composable, modular, tiny tools

Coder's Basilisk's design puts it in a strange place between being a compiler construction toolkit and a traditional systems programming language. It's both a directly useful tool, and a tiny little universe of weird programming language thinamabobs.

Why Programming Languages are important

They're not just how we talk to computers. They're the structure and substrate of how we reason about logic.

Programming languages are very similar to mathematical notation in some sense - a programmer who learns to reason in terms of a programming language's structure can reason in terms of its elements.

This helps a programmer solve problems in their mind.

Humans become better thinkers because programming languages implement new features.

Computer Science at large is misdirected

The current focus of computer science is to run stochastic gradient descent algorithms to try to train a digital god inside a data center.

This is more than just a massive waste of resources. It is an abomination to reason.

Programming language design has not only fallen by the wayside, but lost its way.

This is my philosophy:

First, we must have power. Then, we can do everything else.

There is nothing I know of like Coder's Basilisk.

Not even Lisp, Nim, or Forth come close.

I use it for my programming projects.

Every single file in this repository resides entirely within the public domain.

It is my gift to you. Enjoy it.

How do I get started?

First, build and test the compiler.

cc -O3 *.c -o cbas
# Read version printout
./cbas -v
# basic featuretest - test the compiletime language
./cbas tests/pass/featuretest.cbas
./cbas tests2/dirlist.cbas
cc auto_out.c -o dirlist -lm -pthread
# Run the dirlist test program, should function sort of like 'ls' or 'DIR'
./dirlist .
# if all this worked your compiler is working

put 'cbas' somewhere on your PATH (like /usr/bin/) so you can invoke it by name like gcc.

To make a new project, create a folder that looks like this:

MYPROJECT/
	library/
	.cbas_cfg
	main.cbas

You can edit .cbas_cfg to change where system includes (#include<>) directives look for files.

From there, edit main.cbas, write other .hbas and .cbas files, set up your build system, and get going!

But how do I learn the language?

The Core of Coder's Basilisk is a small structured C-like language.

If you can understand C, you can understand that.

Once you have mastered the basic syntax, you can move onto using its more advanced features.

My suggestion for the curious:

Read the existing documentation and example programs.

See the docs folder.

What license/permissions does this project have?

Everything here is public domain.

Can I yell any louder "Please steal my code"?

ADDITIONAL CREDITS

https://codeberg.org/elotonsotilas

• Made a docs builder system, responsible for letting me make https://gek.codeberg.page/coders_basilisk_docs/

• Used Claude Code to create AI Generated examples under examples/AI-generated/

• Had the idea for nomangle.hbas in stdmeta

Various compatibility fixes - JustAntom