lang::parser

lang/src/parser.mo

Self-hosted Monad grammar parser. Split into modules for maintainability.

defdecls_parser#
pub def decls_parser (input : String) : ParseResult (List Decl)
defdecls_parser_located#
pub def decls_parser_located (input : String) : ParseResult (List Decl)
defdecls_parser_located_cfg#
pub def decls_parser_located_cfg (env : CfgEnv) (input : String) : ParseResult (List Decl)

decls_parser_located under a feature set: declarations whose #[cfg] is false under env are dropped before lowering.

structLocatedDecls#
pub struct LocatedDecls {
    decl_list : List Decl,
    spans : List SourceRange,
}

A located parse, as both things a consumer needs: the lowered declarations, and one SourceRange per declaration IN THE SAME ORDER.

SourceRange rather than a fresh type because lang/types.mo already has exactly this shape and -- until this milestone -- had no user at all: nothing outside its own declaration mentioned it. Reusing it keeps one vocabulary for "a span in a file" across the parser, the checker and the language server, and Location.offset is there for any consumer that wants byte arithmetic back.

path is Option.none throughout: the parser is handed text, never a filename. A consumer that knows the file fills it in.

No Parse* type appears here, per the section comment above.

defdecls_parser_located_with_ranges#
pub def decls_parser_located_with_ranges (input : String) : ParseResult LocatedDecls

decls_parser_located's sibling: the same located parse, plus one SourceRange per declaration.

A SEPARATE entry point on purpose. The ranges cost a second resolve_offsets_in_file pass over the declaration spans, and the production parse must not pay for a feature only navigation reads. Nothing here changes decls_parser_located, locate_and_lower or build_loc_table: decl_list is built by calling exactly those, so a caller can swap one entry point for the other and the terms do not move.

The ranges are PRE-EXPANSION, deliberately. parse_all_decls runs expand_decls after lowering and this does not, because an outline should show the declarations the user wrote rather than the ones a macro generated -- and because expansion changes the declaration COUNT, which would break the one-per-declaration alignment below.

That alignment is positional and rests on lower_parse_decls being a 1:1 map (see collect_decl_spans's own comment for the full statement of the contract and the test that pins it).

defbuild_loc_table#
pub def build_loc_table (whole_file : String) (ds : List ParseDecl) : HashMap String Location

Resolve every span in the parse tree to a position, keyed back by start_rem so lowering needs no arithmetic per node.

resolve_offsets_in_file wants ASCENDING absolute offsets, and this comment used to claim pre-order collection delivers them: a span's start_rem is a remaining length, so larger means earlier, and a pre-order walk visits nodes left to right. That is wrong, and the counterexample is every infix expression: a + b parses to app (app (+) a) b, so pre-order reaches the operator node -- whose span starts at the + -- before the operand a that precedes it in the source. lang/types.mo inverts at span index 244 of 1469.

It cost 164x on the located parse, because resolve_offsets_in_file silently fell back to rescanning the whole file per offset. That function now sorts instead; see its own doc comment for the measurements. Nothing here needs to change, but do not re-derive the false invariant.

defrems_to_offsets#
pub def rems_to_offsets (total : I64) (rems : List I64) (acc : List I64) : List I64
defrekey_by_rem#
pub def rekey_by_rem (total : I64) (pairs : List (Pair I64 Location)) (acc : HashMap String Location) : HashMap String Location
defdecls_skip#
pub def decls_skip (input : String) (acc : List ParseDecl) : ParseResult (List ParseDecl)
defskip_docstrings#
pub def skip_docstrings (input : String) : String