15
8
Fork
You've already forked zig-spec
2

Update grammar.y #43

Open
opened 2022年12月26日 11:20:22 +01:00 by perillo · 12 comments
perillo commented 2022年12月26日 11:20:22 +01:00 (Migrated from github.com)
Copy link

Currently, grammar.y is out of sync with the actual grammar implemented by the zig compiler (in std/zig/tokenizer and std/zig/parser.zig).

These are the places where grammar.y and zig diverges, found with the improved parser from https://github.com/ziglang/zig-spec/pull/42, using files in src and lib/std in the zig repository.

  • doc-comments not allowed in top-level comptime and test declaration.

    /// doc-comment for comptime.comptime{vara=1;_=a;}/// doc-comment for test.test{}
  • Saturating arithmetic is not supported by grammar.y.

    test{vara:isize=10;a+|=1;a-|=1;a*|=1;a<<|=1;constb:isize=a+|1;constc:isize=b-|1;constd:isize=c*|1;conste:isize=d<<|1;_=e;}
  • Mixed doc-comment and line-comment is not supported by grammar.y.

    // A doc-comment followed by a line-comment is not supported by grammar.y.constS=struct{//! doc/// doc// doca:i32,};

    NOTE: I found mixing doc-comment and line-comment confusing, and autodoc doesn't not handle them correctly.

    Examples:

    • std/mem.zig:3760

      /// Force an evaluation of the expression; this tries to prevent/// the compiler from optimizing the computation away even if the/// result eventually gets discarded.// TODO: use @declareSideEffect() when it is available - https://github.com/ziglang/zig/issues/6168pubfndoNotOptimizeAway(val:anytype)void{

      See: https://ziglang.org/documentation/master/std/#root;mem.doNotOptimizeAway.

    • std/coff.zig:354

      /// This relocation is meaningful only when the machine type is ARM or Thumb./// The base relocation applies the 32-bit address of a symbol across a consecutive MOVW/MOVT instruction pair.// ARM_MOV32 = 5,/// This relocation is only meaningful when the machine type is RISC-V./// The base relocation applies to the high 20 bits of a 32-bit absolute address.// RISCV_HIGH20 = 5,/// Reserved, must be zero.RESERVED=6,

      See https://ziglang.org/documentation/master/std/#root;coff.BaseRelocationType.

  • New addrspace keyword.

    Commit: ziglang/zig@ccc7f9987 (Address spaces: addrspace(A) parsing)
    Date: 2021年09月14日

    test {
     const y: *allowzero align(8) addrspace(.generic) const volatile u32 = undefined;
     _ = y;
    }
    
  • Inline switch prong not supported by grammar.y.

    Commit: ziglang/zig@b4d81857f (stage1+2: parse inline switch cases)
    Date: 2022年02月13日

    conststd=@import("std");constexpect=std.testing.expect;constSliceTypeA=externstruct{len:usize,ptr:[*]u32,};constSliceTypeB=externstruct{ptr:[*]SliceTypeA,len:usize,};constAnySlice=union(enum){a:SliceTypeA,b:SliceTypeB,c:[]constu8,d:[]AnySlice,};fnwithSwitch(any:AnySlice)usize{returnswitch(any){// With `inline else` the function is explicitly generated// as the desired switch and the compiler can check that// every possible case is handled.inlineelse=>|slice|slice.len,};}test"inline else"{varany=AnySlice{.c="hello"};tryexpect(withSwitch(any)==5);}
  • New packed struct syntax.

    Commit: ziglang/zig@6249a24e8 (stage2: integer-backed packed structs)
    Date: 2022年02月23日

    pubconstAbsolutePointerModeAttributes=packedstruct(u32){supports_alt_active:bool,supports_pressure_as_z:bool,_pad:u30=0,};
Currently, `grammar.y` is out of sync with the actual grammar implemented by the `zig` compiler (in `std/zig/tokenizer` and `std/zig/parser.zig`). These are the places where `grammar.y` and `zig` diverges, found with the improved parser from https://github.com/ziglang/zig-spec/pull/42, using files in `src` and `lib/std` in the `zig` repository. - [ ] doc-comments not allowed in top-level `comptime` and `test` declaration. ```zig /// doc-comment for comptime. comptime { var a = 1; _ = a; } /// doc-comment for test. test {} ``` - [ ] Saturating arithmetic is not supported by `grammar.y`. ```zig test { var a: isize = 10; a +|= 1; a -|= 1; a *|= 1; a <<|= 1; const b: isize = a +| 1; const c: isize = b -| 1; const d: isize = c *| 1; const e: isize = d <<| 1; _ = e; } ``` - [ ] Mixed `doc-comment` and `line-comment` is not supported by `grammar.y`. ```zig // A doc-comment followed by a line-comment is not supported by grammar.y. const S = struct { //! doc /// doc // doc a: i32, }; ``` **NOTE**: I found mixing `doc-comment` and `line-comment` confusing, and `autodoc` doesn't not handle them correctly. Examples: - std/mem.zig:3760 ```zig /// Force an evaluation of the expression; this tries to prevent /// the compiler from optimizing the computation away even if the /// result eventually gets discarded. // TODO: use @declareSideEffect() when it is available - https://github.com/ziglang/zig/issues/6168 pub fn doNotOptimizeAway(val: anytype) void { ``` See: https://ziglang.org/documentation/master/std/#root;mem.doNotOptimizeAway. - std/coff.zig:354 ```zig /// This relocation is meaningful only when the machine type is ARM or Thumb. /// The base relocation applies the 32-bit address of a symbol across a consecutive MOVW/MOVT instruction pair. // ARM_MOV32 = 5, /// This relocation is only meaningful when the machine type is RISC-V. /// The base relocation applies to the high 20 bits of a 32-bit absolute address. // RISCV_HIGH20 = 5, /// Reserved, must be zero. RESERVED = 6, ``` See https://ziglang.org/documentation/master/std/#root;coff.BaseRelocationType. - [ ] New `addrspace` keyword. Commit: ziglang/zig@ccc7f9987 (Address spaces: addrspace(A) parsing) Date: 2021年09月14日 ``` test { const y: *allowzero align(8) addrspace(.generic) const volatile u32 = undefined; _ = y; } ``` - [ ] Inline switch prong not supported by `grammar.y`. Commit: ziglang/zig@b4d81857f (stage1+2: parse inline switch cases) Date: 2022年02月13日 ```zig const std = @import("std"); const expect = std.testing.expect; const SliceTypeA = extern struct { len: usize, ptr: [*]u32, }; const SliceTypeB = extern struct { ptr: [*]SliceTypeA, len: usize, }; const AnySlice = union(enum) { a: SliceTypeA, b: SliceTypeB, c: []const u8, d: []AnySlice, }; fn withSwitch(any: AnySlice) usize { return switch (any) { // With `inline else` the function is explicitly generated // as the desired switch and the compiler can check that // every possible case is handled. inline else => |slice| slice.len, }; } test "inline else" { var any = AnySlice{ .c = "hello" }; try expect(withSwitch(any) == 5); } ``` - [ ] New packed `struct` syntax. Commit: ziglang/zig@6249a24e8 (stage2: integer-backed packed structs) Date: 2022年02月23日 ```zig pub const AbsolutePointerModeAttributes = packed struct(u32) { supports_alt_active: bool, supports_pressure_as_z: bool, _pad: u30 = 0, }; ```
perillo commented 2022年12月26日 18:55:16 +01:00 (Migrated from github.com)
Copy link

@Vexu, the grammar still fails with some source file in the zig project.

Using the script https://gist.github.com/perillo/a1a731d234b425597a70412a7343b772, these are the errors:

compiler source

./parse.sh ../../zig/src/**/*.zig:

parsing 128 files...
FAIL: ../../zig/src/arch/wasm/CodeGen.zig: 1
<stdin>:661: syntax error
/// 
FAIL: ../../zig/src/link/SpirV.zig: 1
<stdin>:19: syntax error
// 

std source

./parse.sh ../../zig/lib/std/**/*.zig:

parsing 479 files...
FAIL: ../../zig/lib/std/coff.zig: 1
<stdin>:356: syntax error
 /// 
FAIL: ../../zig/lib/std/comptime_string_map.zig: 1
<stdin>:99: syntax error
 const KV = struct { [
FAIL: ../../zig/lib/std/fmt/parse_float/convert_hex.zig: 1
<stdin>:3: syntax error
FAIL: ../../zig/lib/std/http/method.zig: 1
<stdin>:4: syntax error
// 
FAIL: ../../zig/lib/std/meta.zig: 1
<stdin>:118: syntax error
 const EnumKV = struct { [

In the previous list I forgot to add the new tuple type declaration.

Thanks.

@Vexu, the grammar still fails with some source file in the `zig` project. Using the script https://gist.github.com/perillo/a1a731d234b425597a70412a7343b772, these are the errors: ## compiler source `./parse.sh ../../zig/src/**/*.zig`: ``` parsing 128 files... FAIL: ../../zig/src/arch/wasm/CodeGen.zig: 1 <stdin>:661: syntax error /// FAIL: ../../zig/src/link/SpirV.zig: 1 <stdin>:19: syntax error // ``` ## std source `./parse.sh ../../zig/lib/std/**/*.zig`: ``` parsing 479 files... FAIL: ../../zig/lib/std/coff.zig: 1 <stdin>:356: syntax error /// FAIL: ../../zig/lib/std/comptime_string_map.zig: 1 <stdin>:99: syntax error const KV = struct { [ FAIL: ../../zig/lib/std/fmt/parse_float/convert_hex.zig: 1 <stdin>:3: syntax error FAIL: ../../zig/lib/std/http/method.zig: 1 <stdin>:4: syntax error // FAIL: ../../zig/lib/std/meta.zig: 1 <stdin>:118: syntax error const EnumKV = struct { [ ``` In the previous list I forgot to add the new tuple type declaration. - [ ] New tuple type declaration Commit: ziglang/zig@30eb2a175 ```zig const KV = struct { []const u8, isize }; ``` Thanks.
Member
Copy link

Done in 3aefd846ff

Done in 3aefd846ffb1dae997c6a715f81b4003a9072364
perillo commented 2022年12月27日 14:55:02 +01:00 (Migrated from github.com)
Copy link

@Vexu, found more errors when checking the zig test suite:

Compiler tests

./check_parser.sh ../../zig/test/**/*.zig:

running 1456 tests...
FAIL: ../../zig/test/behavior/decltest.zig: zig: 0, grammar: 1
<stdin>:5: syntax error
test t
FAIL: ../../zig/test/behavior/tuple_declarations.zig: zig: 0, grammar: 1
<stdin>:13: syntax error
 const T = struct { comptime u32 a
FAIL: ../../zig/test/cases/compile_errors/Issue_6823_dont_allow_._to_be_followed_by_.zig: zig: 1, grammar: 0
FAIL: ../../zig/test/cases/compile_errors/alignment_of_enum_field_specified.zig: zig: 0, grammar: 1
<stdin>:3: syntax error
 b a
FAIL: ../../zig/test/cases/compile_errors/attempted_double_ampersand.zig: zig: 1, grammar: 0
FAIL: ../../zig/test/cases/compile_errors/chained_comparison_operators.zig: zig: 1, grammar: 0
FAIL: ../../zig/test/cases/compile_errors/invalid_empty_unicode_escape.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 const a = '\u{}
FAIL: ../../zig/test/cases/compile_errors/invalid_exponent_in_float_literal-1.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 0x1.0p1ab
FAIL: ../../zig/test/cases/compile_errors/invalid_exponent_in_float_literal-2.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 0x1.0p50F
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-10.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0__
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-11.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0e-1__
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-12.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1_x
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-13.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 0x_
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-14.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 0x0.0_p
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-2.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1_.
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-3.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 0.0_;
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-4.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0e_
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-5.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0e+_
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-6.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0e-_
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-7.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1.0e-1_;
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-9.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: f128 = 1__
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-1.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: u128 = 10_;
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-2.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: u128 = 0b0010_;
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-3.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: u128 = 0o0010_;
FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-4.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 var bad: u128 = 0x0010_;
FAIL: ../../zig/test/cases/compile_errors/missing_digit_after_base.zig: zig: 0, grammar: 1
<stdin>:2: syntax error
 const x = @as(usize, -0x)
FAIL: ../../zig/test/cases/compile_errors/reject_extern_function_definitions_with_body.zig: zig: 1, grammar: 0
FAIL: ../../zig/test/cases/double_ampersand.0.zig: zig: 1, grammar: 0

Most of the errors are invalid literals; however an interesting case is:

pubfnthe_add_function(a:u32,b:u32)u32{returna+b;}testthe_add_function{if(the_add_function(1,2)!=3)unreachable;}

decltest was added in ziglang/zig@3bbe6a28e.

Thanks

@Vexu, found more errors when checking the `zig` test suite: ## Compiler tests `./check_parser.sh ../../zig/test/**/*.zig`: ``` running 1456 tests... FAIL: ../../zig/test/behavior/decltest.zig: zig: 0, grammar: 1 <stdin>:5: syntax error test t FAIL: ../../zig/test/behavior/tuple_declarations.zig: zig: 0, grammar: 1 <stdin>:13: syntax error const T = struct { comptime u32 a FAIL: ../../zig/test/cases/compile_errors/Issue_6823_dont_allow_._to_be_followed_by_.zig: zig: 1, grammar: 0 FAIL: ../../zig/test/cases/compile_errors/alignment_of_enum_field_specified.zig: zig: 0, grammar: 1 <stdin>:3: syntax error b a FAIL: ../../zig/test/cases/compile_errors/attempted_double_ampersand.zig: zig: 1, grammar: 0 FAIL: ../../zig/test/cases/compile_errors/chained_comparison_operators.zig: zig: 1, grammar: 0 FAIL: ../../zig/test/cases/compile_errors/invalid_empty_unicode_escape.zig: zig: 0, grammar: 1 <stdin>:2: syntax error const a = '\u{} FAIL: ../../zig/test/cases/compile_errors/invalid_exponent_in_float_literal-1.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 0x1.0p1ab FAIL: ../../zig/test/cases/compile_errors/invalid_exponent_in_float_literal-2.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 0x1.0p50F FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-10.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0__ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-11.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0e-1__ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-12.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1_x FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-13.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 0x_ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-14.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 0x0.0_p FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-2.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1_. FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-3.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 0.0_; FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-4.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0e_ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-5.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0e+_ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-6.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0e-_ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-7.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1.0e-1_; FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_float_literal-9.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: f128 = 1__ FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-1.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: u128 = 10_; FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-2.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: u128 = 0b0010_; FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-3.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: u128 = 0o0010_; FAIL: ../../zig/test/cases/compile_errors/invalid_underscore_placement_in_int_literal-4.zig: zig: 0, grammar: 1 <stdin>:2: syntax error var bad: u128 = 0x0010_; FAIL: ../../zig/test/cases/compile_errors/missing_digit_after_base.zig: zig: 0, grammar: 1 <stdin>:2: syntax error const x = @as(usize, -0x) FAIL: ../../zig/test/cases/compile_errors/reject_extern_function_definitions_with_body.zig: zig: 1, grammar: 0 FAIL: ../../zig/test/cases/double_ampersand.0.zig: zig: 1, grammar: 0 ``` Most of the errors are invalid literals; however an interesting case is: ```zig pub fn the_add_function(a: u32, b: u32) u32 { return a + b; } test the_add_function { if (the_add_function(1, 2) != 3) unreachable; } ``` `decltest` was added in ziglang/zig@3bbe6a28e. Thanks
Member
Copy link

Done in 11ed032665, I'm not sure whether this grammar should allow valid numbers or not so I didn't touch them.

Done in 11ed032665fca9c78144c11993cfe229d21a5bc0, I'm not sure whether this grammar should allow valid numbers or not so I didn't touch them.
perillo commented 2022年12月27日 16:07:45 +01:00 (Migrated from github.com)
Copy link

In addition to number literals, the zig parser seems to be able to do additional analysis compared to grammar.y, in order to handle ambiguous cases.

./check_parser.sh ../../zig/test/**/*.zig | grep "zig: 1":

FAIL: ../../zig/test/cases/compile_errors/Issue_6823_dont_allow_._to_be_followed_by_.zig: zig: 1, grammar: 0
 var sequence = "repeat".*** 10;
FAIL: ../../zig/test/cases/compile_errors/attempted_double_ampersand.zig: zig: 1, grammar: 0
 if (a && b) {
 return 1234;
 }
FAIL: ../../zig/test/cases/compile_errors/chained_comparison_operators.zig: zig: 1, grammar: 0
 return 1 < value < 1000
FAIL: ../../zig/test/cases/compile_errors/reject_extern_function_definitions_with_body.zig: zig: 1, grammar: 0
 extern "c" fn definitelyNotInLibC(a: i32, b: i32) i32 {
 return a + b;
 }
FAIL: ../../zig/test/cases/double_ampersand.0.zig: zig: 1, grammar: 0
 pub const a = if (true && false) 1 else 2;
In addition to number literals, the zig parser seems to be able to do additional analysis compared to `grammar.y`, in order to handle ambiguous cases. `./check_parser.sh ../../zig/test/**/*.zig | grep "zig: 1"`: ``` FAIL: ../../zig/test/cases/compile_errors/Issue_6823_dont_allow_._to_be_followed_by_.zig: zig: 1, grammar: 0 var sequence = "repeat".*** 10; FAIL: ../../zig/test/cases/compile_errors/attempted_double_ampersand.zig: zig: 1, grammar: 0 if (a && b) { return 1234; } FAIL: ../../zig/test/cases/compile_errors/chained_comparison_operators.zig: zig: 1, grammar: 0 return 1 < value < 1000 FAIL: ../../zig/test/cases/compile_errors/reject_extern_function_definitions_with_body.zig: zig: 1, grammar: 0 extern "c" fn definitelyNotInLibC(a: i32, b: i32) i32 { return a + b; } FAIL: ../../zig/test/cases/double_ampersand.0.zig: zig: 1, grammar: 0 pub const a = if (true && false) 1 else 2; ```
Member
Copy link

Those shouldn't be in the grammar IMO.

Those shouldn't be in the grammar IMO.
perillo commented 2022年12月27日 16:45:16 +01:00 (Migrated from github.com)
Copy link

Do you think they should be documented in the Zig Language Reference?

Do you think they should be documented in the Zig Language Reference?
Member
Copy link

I don't think it's strictly necessary but if you want to mention it somewhere then go ahead.

I don't think it's strictly necessary but if you want to mention it somewhere then go ahead.
perillo commented 2023年01月09日 16:59:21 +01:00 (Migrated from github.com)
Copy link

@Vexu, in order to check the PEG parser I wrote a tool (in Go, since it was more convenient) that downloads up to 1000 repositories using the GitHub API, and check each .zig file with the check_parser.sh script.

For now I found these issues:

  • A Windows style line ending is not supported

    conststd=@import("std")^M

    https://github.com/ziglibs/zig-lv2/blob/master/examples/fifths/fifths.zig

    check https://github.com/ziglibs/zig-lv2/blob/main/examples/fifths/fifths.zig : error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/346899503/examples/fifths/fifths.zig: zig: 0, grammar: 1
    <stdin>:1: syntax error
    const std = @import("std");^M
     ^
    
  • Official grammar does not support the UTF-8 BOM

    https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig

    check https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig : error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/368337113/src/gl_es_2v0.zig: zig: 0, grammar: 1
    <stdin>:1: syntax error
    <feff>//
    ^
    

    NOTE: <feff> is Vim bomb.

  • Alignment using tabs is not supported

    test{varwindow=trycapy.Window.init();trywindow.set(capy.Column(.{},.{capy.Label(.{.text="Balls with attraction and friction"}),capy.Label(.{}).bind("text",totalEnergyFormat),capy.Align(.{},&canvas),}));}

    There are two tabs before capy.Align(.{}, &canvas),

    https://github.com/capy-ui/capy/blob/master/examples/balls.zig

    check https://github.com/capy-ui/capy/blob/master/examples/balls.zig : error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/351056766/examples/balls.zig: zig: 0, grammar: 1
    <stdin>:58: syntax error
    		capy.Align(.{}, &canvas),
    ^
    
  • PEG parser crash

    https://github.com/marlersoft/zigwin32/blob/master/win32/everything.zig

    check marlersoft/zigwin32:everything.zig: error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/333504729/win32/everything.zig: zig: 0, grammar: 139
    

    The culprit is probably the source file size: 20.6 MB

@Vexu, in order to check the *PEG* parser I wrote a tool (in *Go*, since it was more convenient) that downloads up to 1000 repositories using the GitHub API, and check each `.zig` file with the `check_parser.sh` script. For now I found these issues: - [ ] A Windows style line ending is not supported ```zig const std = @import("std")^M ``` https://github.com/ziglibs/zig-lv2/blob/master/examples/fifths/fifths.zig ``` check https://github.com/ziglibs/zig-lv2/blob/main/examples/fifths/fifths.zig : error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/346899503/examples/fifths/fifths.zig: zig: 0, grammar: 1 <stdin>:1: syntax error const std = @import("std");^M ^ ``` - [ ] Official grammar does not support the UTF-8 BOM https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig ``` check https://github.com/MasterQ32/zero-graphics/blob/master/src/gl_es_2v0.zig : error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/368337113/src/gl_es_2v0.zig: zig: 0, grammar: 1 <stdin>:1: syntax error <feff>// ^ ``` NOTE: `<feff>` is Vim *bomb*. - [ ] Alignment using tabs is not supported ```zig test { var window = try capy.Window.init(); try window.set(capy.Column(.{}, .{ capy.Label(.{ .text = "Balls with attraction and friction" }), capy.Label(.{ }) .bind("text", totalEnergyFormat), capy.Align(.{}, &canvas), })); } ``` There are two tabs before `capy.Align(.{}, &canvas),` https://github.com/capy-ui/capy/blob/master/examples/balls.zig ``` check https://github.com/capy-ui/capy/blob/master/examples/balls.zig : error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/351056766/examples/balls.zig: zig: 0, grammar: 1 <stdin>:58: syntax error capy.Align(.{}, &canvas), ^ ``` - [ ] PEG parser crash https://github.com/marlersoft/zigwin32/blob/master/win32/everything.zig ``` check marlersoft/zigwin32:everything.zig: error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/333504729/win32/everything.zig: zig: 0, grammar: 139 ``` The culprit is probably the source file size: 20.6 MB
perillo commented 2023年01月09日 18:40:03 +01:00 (Migrated from github.com)
Copy link

Found other issues:

  • Not sure where is the problem

    pubconstDmaTuple=struct{DmaController(0),DmaController(1),DmaController(2),DmaController(3)};

    https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig

    check https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig : error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/464707106/src/core/bus/dma.zig: zig: 0, grammar: 1
    <stdin>:8: syntax error
    pub const DmaTuple = struct { DmaController(0), DmaController(1), DmaController(2), DmaController(3) };
     ^
    
  • Official grammar does not report an error for "binary operator xxx has whitespace on one side, but not the other"

    consta=1+2;

    https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig

    check https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig : error: exit status 1
    running 1 tests...
    FAIL: /data/cache/zig-package-index/207880856/wapc.zig: zig: 1, grammar: 0
    

    Not sure if this is responsibility of the PEG grammar.

Here is the full report: https://gist.github.com/perillo/c33e79e6d95efabaf302ef3c5d5907ad

Found other issues: - [ ] Not sure where is the problem ```zig pub const DmaTuple = struct { DmaController(0), DmaController(1), DmaController(2), DmaController(3) }; ``` https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig ``` check https://github.com/paoda/zba/blob/main/src/core/bus/dma.zig : error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/464707106/src/core/bus/dma.zig: zig: 0, grammar: 1 <stdin>:8: syntax error pub const DmaTuple = struct { DmaController(0), DmaController(1), DmaController(2), DmaController(3) }; ^ ``` - [ ] Official grammar does not report an error for "binary operator `xxx` has whitespace on one side, but not the other" ```zig const a = 1 +2; ``` https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig ``` check https://github.com/wapc/wapc-guest-zig/blob/master/wapc.zig : error: exit status 1 running 1 tests... FAIL: /data/cache/zig-package-index/207880856/wapc.zig: zig: 1, grammar: 0 ``` Not sure if this is responsibility of the PEG grammar. Here is the full report: https://gist.github.com/perillo/c33e79e6d95efabaf302ef3c5d5907ad
anatol commented 2023年01月23日 18:00:25 +01:00 (Migrated from github.com)
Copy link

Thank you for taking care of the grammar. A few observations:

  • I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with *, like this?
diff --git a/grammar/grammar.y b/grammar/grammar.y
index ec08094..c5c47be 100644
--- a/grammar/grammar.y
+++ b/grammar/grammar.y
@@ -1,13 +1,12 @@
 Root <- skip container_doc_comment? ContainerMembers eof
 
 # *** Top level ***
-ContainerMembers <- ContainerDeclarations (ContainerField COMMA)* (ContainerField / ContainerDeclarations)
+ContainerMembers <- ContainerDeclaration* (ContainerField COMMA)* (ContainerField / ContainerDeclaration*)
 
-ContainerDeclarations
- <- TestDecl ContainerDeclarations
- / ComptimeDecl ContainerDeclarations
- / doc_comment? KEYWORD_pub? Decl ContainerDeclarations
- /
+ContainerDeclaration
+ <- TestDecl
+ / ComptimeDecl
+ / doc_comment? KEYWORD_pub? Decl
  • ascii_char_not_nl_slash_squote rule specifies a weird range (see multiple dashes) [000円-011円013円-046円-050円-133円135円-177円] - could it be clarified?
Thank you for taking care of the grammar. A few observations: * I see that `ContainerDeclarations` is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with `*`, like this? ```diff diff --git a/grammar/grammar.y b/grammar/grammar.y index ec08094..c5c47be 100644 --- a/grammar/grammar.y +++ b/grammar/grammar.y @@ -1,13 +1,12 @@ Root <- skip container_doc_comment? ContainerMembers eof # *** Top level *** -ContainerMembers <- ContainerDeclarations (ContainerField COMMA)* (ContainerField / ContainerDeclarations) +ContainerMembers <- ContainerDeclaration* (ContainerField COMMA)* (ContainerField / ContainerDeclaration*) -ContainerDeclarations - <- TestDecl ContainerDeclarations - / ComptimeDecl ContainerDeclarations - / doc_comment? KEYWORD_pub? Decl ContainerDeclarations - / +ContainerDeclaration + <- TestDecl + / ComptimeDecl + / doc_comment? KEYWORD_pub? Decl ``` * `ascii_char_not_nl_slash_squote` rule specifies a weird range (see multiple dashes) `[000円-011円013円-046円-050円-133円135円-177円]` - could it be clarified?
McSinyx commented 2024年04月03日 11:11:55 +02:00 (Migrated from github.com)
Copy link

I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with *

Seconded. Also IMHO container_doc_comment? should be part of ContainerMembers so the container declaration expression needs not repeat it:

ContainerDeclAuto <- ContainerDeclType LBRACE container_doc_comment? ContainerMembers RBRACE

ascii_char_not_nl_slash_squote rule specifies a weird range (see multiple dashes) [000円-011円013円-046円-050円-133円135円-177円] - could it be clarified?

It skips 012円 (newline), 047円 (single quote) and 134円 (backslash). Perhaps the name should be clarified that it's backslash instead of slash and sort them in order? BTW the dash in the middle was a typo and fixed in GH-52.

> I see that ContainerDeclarations is declared with right-hand recursion and an additional empty rule. Would it be more readable to refactor the rule and replace it with `*` Seconded. Also IMHO `container_doc_comment?` should be part of ContainerMembers so the container declaration expression needs not repeat it: ContainerDeclAuto <- ContainerDeclType LBRACE container_doc_comment? ContainerMembers RBRACE > `ascii_char_not_nl_slash_squote` rule specifies a weird range (see multiple dashes) `[000円-011円013円-046円-050円-133円135円-177円]` - could it be clarified? It skips `012円` (newline), `047円` (single quote) and `134円` (backslash). Perhaps the name should be clarified that it's backslash instead of slash and sort them in order? BTW the dash in the middle was a typo and fixed in GH-52.
alexrp unlocked this conversation 2025年12月13日 01:58:40 +01:00
Sign in to join this conversation.
No Branch/Tag specified
master
sync
disorganized-spec-chunks
No results found.
Labels
Clear labels
bounty
https://ziglang.org/news/announcing-donor-bounties
bug
Observed behavior contradicts documented or intended behavior.
contributor-friendly
This issue is limited in scope and/or knowledge of project internals.
downstream
An issue with a third-party project that uses this project.
enhancement
Solving this issue will likely involve adding new logic or components to the codebase.
infra
An issue related to project infrastructure, e.g. continuous integration.
optimization
A task to improve performance and/or resource usage.
question
No questions on the issue tracker; use a community space instead.
regression
Something that used to work in a previous version stopped working
upstream
An issue with a third-party project that this project uses.
use case
Describes a real use case that is difficult or impossible, but does not propose a solution.
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ziglang/zig-spec#43
Reference in a new issue
ziglang/zig-spec
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?