Ryan Culpepper <ryanc@racket-lang.org>
This library provides an S-expression notation for a subset of SQL. It provides forms that produce statements (as opaque values rather than strings) that can be used directly with Racket’s db library. It also provides macros and functions for creating and manipulating SQL ASTs.
This library complements the db library. The database library manages connecting to databases and executing queries; this library helps construction of the queries to execute.
We’ll start by going through the examples Introduction to Using Database Connections using this library’s SQL notation instead.
First we create a temporary table to play around with:
Let’s take a look at the statements that just went by:
#:columns[ninteger#:not-null][dvarchar])(sql-statement "... not in current dialect ...")
(sql-statement "INSERT INTO the_numbers (n, d) VALUES (0, 'nothing')")
Now let’s add another row, using “computed” values rather than literals. We can use unquote (or ,) in a scalar expression position to insert a Racket value:
Let’s look at that last statement:
(sql-statement
"INSERT INTO the_numbers (n, d) VALUES (?, ?)"
1
"the loneliest number")
The unquote d expressions turned into parameter placeholders, and the statement stores their values separately. Strangely, the placeholders appear as ?, and PostgreSQL doesn’t understand ? placeholders; they should have been 1γγ« and 2γγ«. But the statement seems to have worked. What’s going on?
(sql-statement "INSERT INTO the_numbers (n, d) VALUES (1,γγ« 2γγ«)" 1 "the loneliest number")
We can introduce placeholders explicitly (although unquote is usually more convenient). An explicit placeholder is written ?, regardless of the dialect. The parameters are given in the query call as usual:
eval:2:0: insert: cannot use both placeholders and unquoted
values
in: (insert #:into the_numbers #:set (n (unquote 3)) (d
?))
You can, of course, mix constant literals and unquote s (or placeholders).
SELECT statements are constructed similarly, and they follow the same rules regarding parameters. The statements work the same with all of the query operations.
(rows-result
'(((name . "n") (typeid . 23) (type-size . 4) (type-mod . -1))
((name . "d") (typeid . 1043) (type-size . -1) (type-mod . -1)))
'(#(0 "nothing") #(2 "company")))
'(#(0 "nothing") #(2 "company"))
'#(0 "nothing")
'("nothing" "company")
"nothing, company"
There are S-expression notations for many common SQL operators and expression forms. See SQL Scalar Expressions for details.
The macros in this section create statement values suitable for passing to the query functions of the db library. These statement values satisfy the db library’s statement? predicate. They are different from the statement ASTs produced by statement-qq .
The printing of a statement value is controlled by (current-sql-dialect ), but the code it generates when passed to a query function is determined by the dialect of the connection the query is performed on.
(sql-statement "SELECT a, b, c FROM mytable WHERE (a > 10)")
(sql-statement "INSERT INTO mytable (a, b, c) VALUES (1, 2, 3)")
(sql-statement
"CREATE TABLE numbers (n integer NOT NULL, t text, PRIMARY KEY (n))")
syntax
( insert #:intotable-nameassign-clausemaybe-conflict)
syntax
( update table-nameassign-clausemaybe-where)
syntax
( delete #:fromtable-namemaybe-where)
(sql-statement "SELECT a, b, c FROM mytable WHERE (a > 10)")
(sql-statement "INSERT INTO mytable (a, b, c) VALUES (1, 2, 3)")
(sql-statement
"INSERT INTO mytable SELECT a, b, c FROM other_table WHERE (d IS NOT NULL)")
Changed in version 1.1 of package sql: Added #:or-ignore for insert .
syntax
#:columnscolumn-def...maybe-constraints)#:asstatement)syntax
statement)
(sql-statement
"CREATE TABLE numbers (n integer NOT NULL, t text, PRIMARY KEY (n))")
Changed in version 1.1 of package sql: Added #:if-not-exists option.
procedure
( sql-statement? v)→boolean?
v:any/c
procedure
( sql-statement->string statement[dialect])→string?
statement:sql-statement?
an S-expression syntax,
an AST type predicate, and
a quasiquotation macro to produce AST values from the S-expression syntax.
procedure
( sql-ast->string ast[dialect])→string?
A name is either an unqualified identifier or an identifier qualified with another name, which depending on its usage might represent a catalog, schema, table, range variable, etc.
Concrete SQL has both unquoted and quoted identifiers. Different SQL environments (eg, database backends) have different restrictions on unquoted identifiers, regarding illegal characters and reserved words. Most (but not all) systems also apply some case-folding rule to unquoted identifiers (eg, PostgreSQL converts to lowercase, some others convert to uppercase).
Similarly, this library has both “tagged” and “untagged” notations for identifiers and names. Untagged identifiers are written as raw symbols; they are short and convenient, but they run the risk of confusion with operators and special symbols used by this library. Examples of special symbols include select, as, and from. Examples of identifiers containing operator characters include hello-goodbye and first/last. These identifiers must be written in tagged form.
(Ident:symbol)Unquoted if possible; case-folded and quoted according the SQL dialect’s rules if symbol is a reserved word or contains illegal characters.
(Ident:MyTable);MyTable(Ident:Select);"SELECT"(Ident:a+b.c);"a+b.c"
(Ident:string)Always quoted without case-folding.
(Ident:"MyTable");"MyTable"(Ident:"Select");"Select"(Ident:"x1.$!!");"x1.$!!"
(Name:name...+)Qualified name; each name except the last qualifies the name to its right.
(Name:xyz);x.y.z(Name:xy.z);x.y.z(Name:x(Ident:y.z));x."y.z"
symbolMust not be a special symbol; otherwise an error is raised.
Equivalent to (Ident:symbol) if symbol contains no dot (.) characters and matches the pattern #px"^(?:\\p{L}|_)(?:\\p{L}|\\p{N}|[_$])*$"—that is, a letter or underscore followed by zero or more letters, numbers, underscores, and dollar signs.
If symbol consists of dot-separated parts satisfying the rule above, it is equivalent to (Name:part... ).
MyTable;MyTablex.y.z;x.y.zx.select.as;x."SELECT"."AS"
Because case-folding behavior is system-dependent, it is wisest to either always quote a given name or never quote it.
syntax
( ident-qq ident)
procedure
( ident-ast? v)→boolean?
v:any/c
"MyTable"
"MyTable"
"\"MyTable\""
"\"SELECT\""
"\"SELECT\""
"\"Select\""
"\"A+B.C\""
eval:17:0: ident-qq: special symbol cannot be used as
untagged identifier
at: select
in: (ident-qq select)
parsing context:
while parsing Ident
term: select
location: eval:17:0
eval:18:0: ident-qq: illegal character in untagged
identifier
at: a+b.c
in: (ident-qq a+b.c)
parsing context:
while parsing Ident
term: a+b.c
location: eval:18:0
"x.y.z"
"x.y.z"
"x.y.z"
"x.\"SELECT\".\"AS\""
A scalar expression is either a name, a literal integer or string value, or an application of some function or operator. Note: not every kind of expression is supported in every SQL dialect.
(existstable-expr)Produces an EXISTS expression:
(exists(select 1#:fromt));EXISTS (SELECT 1 FROM t)
(inscalar-expr#:fromtable-expr)(inscalar-expr#:valuesscalar-expr...)There are two forms of IN expression, one for table expressions and one for lists of scalar expressions:
(inx#:values123);x IN (1, 2, 3)
(case[scalar-exprscalar-expr]...maybe-else)(case#:ofscalar-expr[scalar-exprscalar-expr]...maybe-else)There are two forms of CASE expression, one like Racket’s cond and the other like Racket’s case :
(compare-operatorscalar-expr#:sometable-expr)(compare-operatorscalar-expr#:alltable-expr)Produces an “all-or-any” comparison between a scalar (or row) expression and a table expression.
(namescalar-expr...)Represents an ordinary function call; no arity checking is done.(coalescexyz);coalesce(x, y, z)
table-exprRepresents a subquery; the query must return at most one row.
(select y#:fromys#:where(x= 0));(SELECT y FROM ys WHERE x = 0)
syntax
( scalar-expr-qq scalar-expr)
procedure
( scalar-expr-ast? v)→boolean?
v:any/c
"mytable.mycolumn"
"42"
"'Salutations'"
"'a ''tricky'' string'"
"log((1 - p))"
"((x > 10) AND (x < 55))"
"coalesce(x, y, z)"
"CAST('2015-03-15' AS \"DATE\")"
"EXTRACT(\"YEAR\" FROM dob)"
"(mytable.mycolumn IS NULL)"
"(ph_num LIKE '555-____')"
"(lname || ', ' || fname)"
(operator/specialscalar-expr...)This function-like syntax is used to represent uses of SQL operators, standard SQL functions that don’t use ordinary function-call notation, and a few other special cases, listed below.
The CAST and EXTRACT special functions:
(cast"2015-03-15"DATE);CAST('2015-03-15' AS DATE)(cast"123"(NUMERIC50));CAST('123' AS NUMERIC(5, 0))(extractYEARdob);EXTRACT(YEAR FROM dob)
Note that as above, types and fields are written as “scalar expressions”, in a mild abuse of syntax.
The OVERLAY, POSITION, and SUBSTRING functions:
(overlay"abc""z"21);OVERLAY('abc' PLACING 'z' FROM 2 FOR 1)(position"c""abc");POSITION('c' IN 'abc)
The TRIM function is written using one of the following variants:
(trim-leading"z""zzabc");TRIM(LEADING 'z' FROM 'zzabc')(trim-trailing"z""abczz");TRIM(TRAILING 'z' FROM 'abczz')(trim-both"z""zzabczz");TRIM(BOTH 'z' FROM 'zzabczz')
The syntax COUNT(*) can be written as follows:
(count-all);COUNT(*)
The chaining arithmetic operators +, -, *, and /:
(+ 1234);1 + 2 + 3 + 4
The chaining infix logical operators AND and OR:
The chaining infix binary operator || can be written as \|\| or as ||; the latter reads as the empty symbol.
(||lname", "fname);lname || ', ' || fname(\|\|lname", "fname);lname || ', ' || fname
Any identifier consisting of only characters in ~!@#%^&*-_=+|<>?/ is considered a non-chaining infix binary operator:
(%&!12);1 %&! 2
The following operators:
(is-nullx);x IS NULL(is-not-nullx);x IS NOT NULL(is-truex);x IS TRUE(is-not-truex);x IS NOT TRUE(is-falsex);x IS FALSE(is-not-falsex);x IS NOT FALSE(is-unknownx);x IS UNKNOWN(is-not-unknownx);x IS NOT UNKNOWN(collatexutf8);x COLLATE utf8(between-and5110);5 BETWEEN 1 AND 10(distinct-fromxy);x DISTINCT FROM y(like"abc""a%");'abc' LIKE 'a%'(ilike"aBC""ab_");'aBC' ILIKE 'ab_'(similar-to"abc""(a|z)%");'abc' SIMILAR TO '(a|z)%'
Field selection is written as a regular identifier (or *) prefixed by a dot.
(.citystate);(state).city(.*table1);(table1).*(.*);*
Row constructors (the ROW syntax is a PostgreSQL extension):
(%row123);(1, 2, 3)(row123);ROW(1, 2, 3)
Arrays and array indexing (PostgreSQL extension):
(%array123);ARRAY[1, 2, 3](%refx1);(x)[1](%refx123);(x)[1,2,3]
There are two variants of scalar-expr that enable the construction of parameterized queries. The first is the placeholder, written ? (regardless of the notation used by the database the query is to be sent to). The second is the unquote form, which is equivalent to inserting a placeholder and also providing the expression as a query parameter.
Note: Due to limitations in the db library, unquote parameters and ordinary placeholders cannot be mixed in the same statement.
The resulting statement can be used with parameters thus:
(query-value c(select a#:frommytable#:where(= b?))10)
Using the unquote form eliminates the need to keep track of positional parameters; instead, the parameter value is written as a Racket expression within the statement. It is automatically translated to SQL code containing placeholders.
(sql-statement "SELECT a FROM mytable WHERE (b = ?)" 10)
The resulting statement must be called without additional parameters:
(query-value c(select a#:frommytable#:where(= b,b-param)))
Note that placeholder syntax varies between SQL dialects. We can see the code a statement produces for a specific dialect by setting the current-sql-dialect parameter:
"SELECT a FROM mytable WHERE (b = 1γγ«)"
A table reference is either a reference to a defined table (or view) or a computed table with a name or named components. A table expression can be formed using join and set operations.
Note: in the final variant of table-ref, the table-expr must be a join table expression, specifically.
syntax
( table-ref-qq table-ref)
procedure
( table-ref-ast? v)→boolean?
v:any/c
"supplier"
"supplier AS s"
"supplier INNER JOIN part USING (supply_id)"
syntax
( table-expr-qq table-expr)
procedure
( table-expr-ast? v)→boolean?
v:any/c
A statement is one of the four standard DML statements or a WITH statement that combines them with one or more common table expressions.
Select
A select-statement can contain each kind of select-clause at most once. The clauses can occur in any order. If the first form of select-statement is used (that is, with the initial select-items), then the #:values clause may not also be used.
Changed in version 1.2 of package sql: Added distinct-clause
Insert
Changed in version 1.1 of package sql: Added #:or-ignore.
Update
Delete
With
Different database systems place different restrictions on what kinds of statements and table expressions they allow within WITH statements.
Changed in version 1.6 of package sql: Allowed table-expr for right-hand side.
syntax
( statement-qq statement)
procedure
( statement-ast? v)→boolean?
v:any/c
"SELECT a, b, c FROM mytable WHERE (a > 10)"
"INSERT INTO mytable (a, b, c) VALUES (1, 2, 3)"
syntax
( select-item-qq select-item)
procedure
( select-item-ast? v)→boolean?
v:any/c
A select-item AST can be a list of select-item ASTs, representing multiple items to be spliced into the SELECT statement.
Added in version 1.6 of package sql.
Changed in version 1.1 of package sql: Added #:if-not-exists option
for create-table.
Changed in version 1.4: Added action-decl for foreign-key
and the #:default column-option for column-def.
parameter
( current-sql-dialect )→(or/c symbol? #f)
This parameter does not affect statement (sql-statement? ) values used with connection query methods; generation of SQL code for a query method automatically uses the dialect associated with the connection the query is performed on.
This library allows the dynamic composition of statements and the injection of SQL text using the following extensions to the SQL grammar.
Warning: Never use the INJECT forms to include SQL computed from an untrusted source. Use placeholders or unquote parameters instead; see Placeholders and Unquoted Parameters.
Changed in version 1.2 of package sql: Added Ident:AST and Name:AST.
Changed in version 1.6: Added SelectItem:AST. See note under
select-item-ast? about lists.
procedure
( make-ident-ast s)→ident-ast?
Added in version 1.2 of package sql.
"MyTable"
"\"MyTable\""
"\"SELECT\""
"\"A+B.C\""
procedure
( make-name-ast s)→name-ast?
If s is a symbol, it is parsed according to the symbol case of the name nonterminal (see SQL Names and Identifiers).
If s is a name-ast? , it is returned unchanged.
If s is a list, each component is parsed a name-ast? and the components are joined.
Added in version 1.2 of package sql.
"x"
"x.y.z"
"x.y.a.b"
procedure
v:any/c
Equivalent to (scalar-expr-qq ,value).
Added in version 1.2 of package sql.
procedure
( make-values*-table-expr-ast l-rows)→table-expr-ast?
Note that all of the rows (i.e. the inner lists) must be of the same length.
Added in version 1.3 of package sql.
#:intoproduce#:columnsnamenumtype#:from(TableExpr:ASTitem)(insert-produce(produce-item"bananas"5"organic")(produce-item"cranberries"50"canned"))))"INSERT INTO produce (name, num, type) VALUES (?, ?, ?), (?, ?, ?), (?, ?, ?)"