lang::parser
lang/src/parser.mo
Self-hosted Monad grammar parser. Split into modules for maintainability.
pub def decls_parser (input : String) : ParseResult (List Decl)pub def decls_parser_located (input : String) : ParseResult (List Decl)pub def decls_parser_located_cfg (env : CfgEnv) (input : String) : ParseResult (List Decl)decls_parser_located under a feature set: declarations whose
#[cfg] is false under env are dropped before lowering.
pub struct LocatedDecls {
decl_list : List Decl,
spans : List SourceRange,
}A located parse, as both things a consumer needs: the lowered
declarations, and one SourceRange per declaration IN THE SAME ORDER.
SourceRange rather than a fresh type because lang/types.mo already
has exactly this shape and -- until this milestone -- had no user at
all: nothing outside its own declaration mentioned it. Reusing it keeps
one vocabulary for "a span in a file" across the parser, the checker and
the language server, and Location.offset is there for any consumer
that wants byte arithmetic back.
path is Option.none throughout: the parser is handed text, never a
filename. A consumer that knows the file fills it in.
No Parse* type appears here, per the section comment above.
pub def decls_parser_located_with_ranges (input : String) : ParseResult LocatedDeclsdecls_parser_located's sibling: the same located parse, plus one
SourceRange per declaration.
A SEPARATE entry point on purpose. The ranges cost a second
resolve_offsets_in_file pass over the declaration spans, and the
production parse must not pay for a feature only navigation reads.
Nothing here changes decls_parser_located, locate_and_lower or
build_loc_table: decl_list is built by calling exactly those, so a
caller can swap one entry point for the other and the terms do not move.
The ranges are PRE-EXPANSION, deliberately. parse_all_decls runs
expand_decls after lowering and this does not, because an outline
should show the declarations the user wrote rather than the ones a
macro generated -- and because expansion changes the declaration COUNT,
which would break the one-per-declaration alignment below.
That alignment is positional and rests on lower_parse_decls being a
1:1 map (see collect_decl_spans's own comment for the full statement
of the contract and the test that pins it).
pub def build_loc_table (whole_file : String) (ds : List ParseDecl) : HashMap String LocationResolve every span in the parse tree to a position, keyed back by
start_rem so lowering needs no arithmetic per node.
resolve_offsets_in_file wants ASCENDING absolute offsets, and this
comment used to claim pre-order collection delivers them: a span's
start_rem is a remaining length, so larger means earlier, and a
pre-order walk visits nodes left to right. That is wrong, and the
counterexample is every infix expression: a + b parses to
app (app (+) a) b, so pre-order reaches the operator node -- whose span
starts at the + -- before the operand a that precedes it in the
source. lang/types.mo inverts at span index 244 of 1469.
It cost 164x on the located parse, because resolve_offsets_in_file
silently fell back to rescanning the whole file per offset. That function
now sorts instead; see its own doc comment for the measurements. Nothing
here needs to change, but do not re-derive the false invariant.
pub def rems_to_offsets (total : I64) (rems : List I64) (acc : List I64) : List I64pub def rekey_by_rem (total : I64) (pairs : List (Pair I64 Location)) (acc : HashMap String Location) : HashMap String Locationpub def decls_skip (input : String) (acc : List ParseDecl) : ParseResult (List ParseDecl)pub def skip_docstrings (input : String) : String