Currently, grammar.y is out of sync with the actual grammar implemented by the zig compiler (in std/zig/tokenizer and std/zig/parser.zig).
These are the places where grammar.y and zig diverges, found with the improved parser from https://github.com/ziglang/zig-spec/pull/42, using files in src and lib/std in the zig repository.
doc-comments not allowed in top-level comptime and test declaration.
/// doc-comment for comptime.comptime{vara=1;_=a;}/// doc-comment for test.test{}
Saturating arithmetic is not supported by grammar.y.
Mixed doc-comment and line-comment is not supported by grammar.y.
// A doc-comment followed by a line-comment is not supported by grammar.y.constS=struct{//! doc/// doc// doca:i32,};
NOTE: I found mixing doc-comment and line-comment confusing, and autodoc doesn't not handle them correctly.
Examples:
std/mem.zig:3760
/// Force an evaluation of the expression; this tries to prevent/// the compiler from optimizing the computation away even if the/// result eventually gets discarded.// TODO: use @declareSideEffect() when it is available - https://github.com/ziglang/zig/issues/6168pubfndoNotOptimizeAway(val:anytype)void{
/// This relocation is meaningful only when the machine type is ARM or Thumb./// The base relocation applies the 32-bit address of a symbol across a consecutive MOVW/MOVT instruction pair.// ARM_MOV32 = 5,/// This relocation is only meaningful when the machine type is RISC-V./// The base relocation applies to the high 20 bits of a 32-bit absolute address.// RISCV_HIGH20 = 5,/// Reserved, must be zero.RESERVED=6,
conststd=@import("std");constexpect=std.testing.expect;constSliceTypeA=externstruct{len:usize,ptr:[*]u32,};constSliceTypeB=externstruct{ptr:[*]SliceTypeA,len:usize,};constAnySlice=union(enum){a:SliceTypeA,b:SliceTypeB,c:[]constu8,d:[]AnySlice,};fnwithSwitch(any:AnySlice)usize{returnswitch(any){// With `inline else` the function is explicitly generated// as the desired switch and the compiler can check that// every possible case is handled.inlineelse=>|slice|slice.len,};}test"inline else"{varany=AnySlice{.c="hello"};tryexpect(withSwitch(any)==5);}
Currently, `grammar.y` is out of sync with the actual grammar implemented by the `zig` compiler (in `std/zig/tokenizer` and `std/zig/parser.zig`).
These are the places where `grammar.y` and `zig` diverges, found with the improved parser from https://github.com/ziglang/zig-spec/pull/42, using files in `src` and `lib/std` in the `zig` repository.
- [ ] doc-comments not allowed in top-level `comptime` and `test` declaration.
```zig
/// doc-comment for comptime.
comptime {
var a = 1;
_ = a;
}
/// doc-comment for test.
test {}
```
- [ ] Saturating arithmetic is not supported by `grammar.y`.
```zig
test {
var a: isize = 10;
a +|= 1;
a -|= 1;
a *|= 1;
a <<|= 1;
const b: isize = a +| 1;
const c: isize = b -| 1;
const d: isize = c *| 1;
const e: isize = d <<| 1;
_ = e;
}
```
- [ ] Mixed `doc-comment` and `line-comment` is not supported by `grammar.y`.
```zig
// A doc-comment followed by a line-comment is not supported by grammar.y.
const S = struct {
//! doc
/// doc
// doc
a: i32,
};
```
**NOTE**: I found mixing `doc-comment` and `line-comment` confusing, and `autodoc` doesn't not handle them correctly.
Examples:
- std/mem.zig:3760
```zig
/// Force an evaluation of the expression; this tries to prevent
/// the compiler from optimizing the computation away even if the
/// result eventually gets discarded.
// TODO: use @declareSideEffect() when it is available - https://github.com/ziglang/zig/issues/6168
pub fn doNotOptimizeAway(val: anytype) void {
```
See: https://ziglang.org/documentation/master/std/#root;mem.doNotOptimizeAway.
- std/coff.zig:354
```zig
/// This relocation is meaningful only when the machine type is ARM or Thumb.
/// The base relocation applies the 32-bit address of a symbol across a consecutive MOVW/MOVT instruction pair.
// ARM_MOV32 = 5,
/// This relocation is only meaningful when the machine type is RISC-V.
/// The base relocation applies to the high 20 bits of a 32-bit absolute address.
// RISCV_HIGH20 = 5,
/// Reserved, must be zero.
RESERVED = 6,
```
See https://ziglang.org/documentation/master/std/#root;coff.BaseRelocationType.
- [ ] New `addrspace` keyword.
Commit: ziglang/zig@ccc7f9987 (Address spaces: addrspace(A) parsing)
Date: 2021年09月14日
```
test {
const y: *allowzero align(8) addrspace(.generic) const volatile u32 = undefined;
_ = y;
}
```
- [ ] Inline switch prong not supported by `grammar.y`.
Commit: ziglang/zig@b4d81857f (stage1+2: parse inline switch cases)
Date: 2022年02月13日
```zig
const std = @import("std");
const expect = std.testing.expect;
const SliceTypeA = extern struct {
len: usize,
ptr: [*]u32,
};
const SliceTypeB = extern struct {
ptr: [*]SliceTypeA,
len: usize,
};
const AnySlice = union(enum) {
a: SliceTypeA,
b: SliceTypeB,
c: []const u8,
d: []AnySlice,
};
fn withSwitch(any: AnySlice) usize {
return switch (any) {
// With `inline else` the function is explicitly generated
// as the desired switch and the compiler can check that
// every possible case is handled.
inline else => |slice| slice.len,
};
}
test "inline else" {
var any = AnySlice{ .c = "hello" };
try expect(withSwitch(any) == 5);
}
```
- [ ] New packed `struct` syntax.
Commit: ziglang/zig@6249a24e8 (stage2: integer-backed packed structs)
Date: 2022年02月23日
```zig
pub const AbsolutePointerModeAttributes = packed struct(u32) {
supports_alt_active: bool,
supports_pressure_as_z: bool,
_pad: u30 = 0,
};
```
@Vexu, in order to check the PEG parser I wrote a tool (in Go, since it was more convenient) that downloads up to 1000 repositories using the GitHub API, and check each .zig file with the check_parser.sh script.
test{varwindow=trycapy.Window.init();trywindow.set(capy.Column(.{},.{capy.Label(.{.text="Balls with attraction and friction"}),capy.Label(.{}).bind("text",totalEnergyFormat),capy.Align(.{},&canvas),}));}
There are two tabs before capy.Align(.{}, &canvas),
The culprit is probably the source file size: 20.6 MB
@Vexu, in order to check the *PEG* parser I wrote a tool (in *Go*, since it was more convenient) that downloads up to 1000 repositories using the GitHub API, and check each `.zig` file with the `check_parser.sh` script.
For now I found these issues:
- [ ] A Windows style line ending is not supported
```zig
const std = @import("std")^M
```
https://github.com/ziglibs/zig-lv2/blob/master/examples/fifths/fifths.zig
```
check https://github.com/ziglibs/zig-lv2/blob/main/examples/fifths/fifths.zig : error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/346899503/examples/fifths/fifths.zig: zig: 0, grammar: 1
<stdin>:1: syntax error
const std = @import("std");^M
^
```
- [ ] Official grammar does not support the UTF-8 BOM
https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig
```
check https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig : error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/368337113/src/gl_es_2v0.zig: zig: 0, grammar: 1
<stdin>:1: syntax error
<feff>//
^
```
NOTE: `<feff>` is Vim *bomb*.
- [ ] Alignment using tabs is not supported
```zig
test {
var window = try capy.Window.init();
try window.set(capy.Column(.{}, .{
capy.Label(.{ .text = "Balls with attraction and friction" }),
capy.Label(.{ })
.bind("text", totalEnergyFormat),
capy.Align(.{}, &canvas),
}));
}
```
There are two tabs before `capy.Align(.{}, &canvas),`
https://github.com/capy-ui/capy/blob/master/examples/balls.zig
```
check https://github.com/capy-ui/capy/blob/master/examples/balls.zig : error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/351056766/examples/balls.zig: zig: 0, grammar: 1
<stdin>:58: syntax error
capy.Align(.{}, &canvas),
^
```
- [ ] PEG parser crash
https://github.com/marlersoft/zigwin32/blob/master/win32/everything.zig
```
check marlersoft/zigwin32:everything.zig: error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/333504729/win32/everything.zig: zig: 0, grammar: 139
```
The culprit is probably the source file size: 20.6 MB
Found other issues:
- [ ] Not sure where is the problem
```zig
pub const DmaTuple = struct { DmaController(0), DmaController(1), DmaController(2), DmaController(3) };
```
https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig
```
check https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig : error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/464707106/src/core/bus/dma.zig: zig: 0, grammar: 1
<stdin>:8: syntax error
pub const DmaTuple = struct { DmaController(0), DmaController(1), DmaController(2), DmaController(3) };
^
```
- [ ] Official grammar does not report an error for "binary operator `xxx` has whitespace on one side, but not the other"
```zig
const a = 1 +2;
```
https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig
```
check https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig : error: exit status 1
running 1 tests...
FAIL: /data/cache/zig-package-index/207880856/wapc.zig: zig: 1, grammar: 0
```
Not sure if this is responsibility of the PEG grammar.
Here is the full report: https://gist.github.com/perillo/c33e79e6d95efabaf302ef3c5d5907ad
Thank you for taking care of the grammar. A few observations:
I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with *, like this?
ascii_char_not_nl_slash_squote rule specifies a weird range (see multiple dashes) [000円-011円013円-046円-050円-133円135円-177円] - could it be clarified?
Thank you for taking care of the grammar. A few observations:
* I see that `ContainerDeclarations` is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with `*`, like this?
```diff
diff --git a/grammar/grammar.y b/grammar/grammar.y
index ec08094..c5c47be 100644
--- a/grammar/grammar.y
+++ b/grammar/grammar.y
@@ -1,13 +1,12 @@
Root <- skip container_doc_comment? ContainerMembers eof
# *** Top level ***
-ContainerMembers <- ContainerDeclarations (ContainerField COMMA)* (ContainerField / ContainerDeclarations)
+ContainerMembers <- ContainerDeclaration* (ContainerField COMMA)* (ContainerField / ContainerDeclaration*)
-ContainerDeclarations
- <- TestDecl ContainerDeclarations
- / ComptimeDecl ContainerDeclarations
- / doc_comment? KEYWORD_pub? Decl ContainerDeclarations
- /
+ContainerDeclaration
+ <- TestDecl
+ / ComptimeDecl
+ / doc_comment? KEYWORD_pub? Decl
```
* `ascii_char_not_nl_slash_squote` rule specifies a weird range (see multiple dashes) `[000円-011円013円-046円-050円-133円135円-177円]` - could it be clarified?
I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with *
Seconded. Also IMHO container_doc_comment? should be part of ContainerMembers so the container declaration expression needs not repeat it:
ascii_char_not_nl_slash_squote rule specifies a weird range (see multiple dashes) [000円-011円013円-046円-050円-133円135円-177円] - could it be clarified?
It skips 012円 (newline), 047円 (single quote) and 134円 (backslash). Perhaps the name should be clarified that it's backslash instead of slash and sort them in order? BTW the dash in the middle was a typo and fixed in GH-52.
> I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with `*`
Seconded. Also IMHO `container_doc_comment?` should be part of ContainerMembers so the container declaration expression needs not repeat it:
ContainerDeclAuto <- ContainerDeclType LBRACE container_doc_comment? ContainerMembers RBRACE
> `ascii_char_not_nl_slash_squote` rule specifies a weird range (see multiple dashes) `[000円-011円013円-046円-050円-133円135円-177円]` - could it be clarified?
It skips `012円` (newline), `047円` (single quote) and `134円` (backslash). Perhaps the name should be clarified that it's backslash instead of slash and sort them in order? BTW the dash in the middle was a typo and fixed in GH-52.
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
ziglang/zig-spec#43
Loading...
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?