hermes/TODO at develop · scottfrazer/hermes · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
2.0 TODO
========

ParseTree Optimization
----------------------

$ hermes --debug --no-color parse grammar.hgr source --tree

This will get an "annotated parse tree" with a lot of the extra bits attached that are used in the generated code.

Syntax Error Unification
------------------------

0/2/ast:Unrecognized token on line 1, column 3:
1/3/ast:Unrecognized token on line 1, column 11:
10/2/ast:No more tokens.  Expecting b
10/3/ast:Error: unexpected end of file
10/5/ast:Unexpected symbol (line 1, col 5) when parsing parse_sub.  Expected b, got <source:1:5 semi "Ow==">.
2/2/ast:Unrecognized token on line 4, column 1:
2/3/ast:Unrecognized token on line 3, column 2:
23/1/ast:Error: unexpected end of file
38/0/ast:No more tokens.  Expecting sep
4/1/ast:Unrecognized token on line 1, column 1:
6/1/ast:Unrecognized token on line 1, column 12:
6/2/ast:Unrecognized token on line 1, column 9:
json/0/ast:Error: unexpected end of file
json/5/ast:Unexpected symbol (line 3, col 5) when parsing parse__gen1.  Expected rbrace, got <source:3:5 string "ImRlZiI=">.
json/6/ast:Unrecognized token on line 2, column 5:
json/8/ast:No more tokens.  Expecting rsquare

Document public interface for all targets
-----------------------------------------

Document the interface for the public classes (just CompositeGrammar?) in hermes.grammar:

  HermesGrammar:
    __init__(name, rules)
    start - self.rules[0].nonterminal
    rules - self.rules
    terminals - list of all terminals
    nonterminals - list of all nonterminals
    standard_terminals - list of terminals that are non-abstract (i.e. not :_empty and :_end)
    ll1_nonterminals - list of nonterminals that aren't expression nonterminals
    expression_nonterminals - list of expression nonterminals
    parse_table - 2x2 matrix [nonterminal][terminal] = rule
    must_consume_token(nonterminal) - returns true if there is no production such that $nonterminal = :_empty
    first(production|nonterminal) - list of terminals in first set
    follow(nonterminal) - list of terminals in follow set
    get_rules(nonterminal) - list of rules for this nonterminal
    get_expanded_rules(nonterminal) - list of expanded rules for this nonterminal
    __str__()

  Rule
    id - auto assigned integer identifier
    nonterminal - LHS of this rule
    is_empty - true if $nonterminal = :_empty
    production - Production object
    operator (optional) - Terminal
    __eq__(), __str__()

  Production:
    morphemes - list of morphemes in this production

  ExprRule(Rule)
    nud_production - Production
    led_production - Production
    nudAst - AstSpecification or AstTranslation
    ledAst - AstSpecification or AstTranslation
    __str__()

  NonTerminal, Terminal
    id - auto assigned integer identifier
    string - string representation, without leading $ or :

  Operator
    operator - Terminal of this operator
    associativity - ['left', 'right', 'unary']
    binding_power - integer binding power.  How closely this binds relative to other operators

  AstSpecification
    name
    parameters

  AstTranslation
    idx

  ------------------------

  Grammar
    - rules (Rule)
    - terminals
    - nonterminals
    - macros
    - start
    - must_consume_tokens(nonterminal) [is_empty]
    - is_standard_terminal(terminal)
    - is_abstract_terminal(terminal)
    - conflicts
  LL1Grammar(Grammar)
    - first (dict)
    - follow (dict)
  ExpressionGrammar(Grammar)
    - nud
    - led
    - precedence
  CompositeGrammar
    - ll1_grammar
    - expression_grammars
  LL1Rule
    - id, morphemes
    - nonterminal
    - production
  ExpressionRule
    - id, morphemes
    - nud
    - led
  NonTerminal
    - id, name
  Terminal
    - id, name