View source for Earley parser

{{Short description|Algorithm for parsing context-free languages}}
{{Infobox algorithm
|name={{PAGENAMEBASE}}
|class=[[Parsing]] grammars that are [[Context-free grammar|context-free]]
|data=[[String (computer science)|String]]
|time=<math>O(n^3)</math>
|best-time={{plainlist|
* <math>\Omega(n)</math> for all [[Deterministic context-free grammar|deterministic context-free grammar]]s
* <math>\Omega(n^2)</math> for [[Ambiguous grammar|unambiguous grammars]]
}}
|average-time=<math>\Theta(n^3)</math>
|space=
}}

In [[Computer science|computer science]], the '''Earley parser''' is an [[Algorithm|algorithm]] for [[Parsing|parsing]] [[String (computer science)|strings]] that belong to a given [[Context-free language|context-free language]], though (depending on the variant) it may suffer problems with certain nullable grammars.<ref>{{cite web|last=Kegler|first=Jeffrey|title=What is the Marpa algorithm?|url=http://blogs.perl.org/users/jeffrey_kegler/2011/11/what-is-the-marpa-algorithm.html|access-date=20 August 2013}}</ref> The algorithm, named after its inventor, [[Biography:Jay Earley|Jay Earley]], is a [[Chart parser|chart parser]] that uses [[Dynamic programming|dynamic programming]]; it is mainly used for parsing in [[Computational linguistics|computational linguistics]]. It was first introduced in his dissertation<ref name=Earley1>{{cite book
 | last=Earley
 | first=Jay
 | title=An Efficient Context-Free Parsing Algorithm
 | year=1968
 | publisher=Carnegie-Mellon Dissertation
 | url=http://reports-archive.adm.cs.cmu.edu/anon/anon/usr/ftp/scan/CMU-CS-68-earley.pdf}}</ref> in 1968 (and later appeared in an abbreviated, more legible, form in a journal<ref name="Earley2">{{citation
 | doi = 10.1145/362007.362035 | url = http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/cmt-55/lti/Courses/711/Class-notes/p94-earley.pdf | archive-url = https://web.archive.org/web/20040708052627/http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/cmt-55/lti/Courses/711/Class-notes/p94-earley.pdf | url-status = dead | archive-date = 2004-07-08 | issue = 2
 | journal = [[Communications of the ACM]]
 | pages = 94–102
 | title = An efficient context-free parsing algorithm
 | volume = 13
 | year = 1970| s2cid = 47032707 }}</ref>).

Earley parsers are appealing because they can parse all context-free languages, unlike [[LR parser]]s and [[LL parser]]s, which are more typically used in [[Compiler|compiler]]s but which can only handle restricted classes of languages.  The Earley parser executes in cubic time in the general case <math>{O}(n^3)</math>, where ''n'' is the length of the parsed string, quadratic time for unambiguous grammars <math>{O}(n^2)</math>,<ref>{{cite book | isbn=978-0-201-02988-8 | author=John E. Hopcroft and Jeffrey D. Ullman | title=Introduction to Automata Theory, Languages, and Computation | location=Reading/MA | publisher=Addison-Wesley | year=1979 | url-access=registration | url=https://archive.org/details/introductiontoau00hopc }} p.145</ref> and linear time for all [[Deterministic context-free grammar|deterministic context-free grammar]]s. It performs particularly well when the rules are written [[Left recursion|left-recursively]].

== Earley recogniser ==
The following algorithm describes the Earley recogniser. The recogniser can be modified to create a parse tree as it recognises, and in that way can be turned into a parser.

== The algorithm ==
In the following descriptions, α, β, and γ represent any [[String (computer science)|string]] of [[Terminal and nonterminal symbols|terminals/nonterminals]] (including the [[Empty string|empty string]]), X and Y represent single nonterminals, and ''a'' represents a terminal symbol.

Earley's algorithm is a top-down [[Dynamic programming|dynamic programming]] algorithm. In the following, we use Earley's dot notation: given a [[Formal grammar#The syntax of grammars|production]] X → αβ, the notation X → α • β represents a condition in which α has already been parsed and β is expected.

Input position 0 is the position prior to input.  Input position ''n'' is the position after accepting the ''n''th token.  (Informally, input positions can be thought of as locations at [[Lexical analysis|token]] boundaries.)  For every input position, the parser generates a ''state set''.  Each state is a [[Tuple|tuple]] (X → α • β, ''i''), consisting of

* the production currently being matched (X → α β)
* the current position in that production (visually represented by the dot •)
* the position ''i'' in the input at which the matching of this production began: the ''origin position''

(Earley's original algorithm included a look-ahead in the state; later research showed this to have little practical effect on the parsing efficiency, and it has subsequently been dropped from most implementations.)

A state is finished when its current position is the last position of the right side of the production, that is, when there is no symbol to the right of the dot • in the visual representation of the state.

The state set at input position ''k'' is called S(''k'').  The parser is seeded with S(0) consisting of only the top-level rule.  The parser then repeatedly executes three operations:  ''prediction'', ''scanning'', and ''completion''.

* ''Prediction'':  For every state in S(''k'') of the form (X → α • Y β, ''j'') (where ''j'' is the origin position as above), add (Y → • γ, ''k'') to S(''k'') for every production in the grammar with Y on the left-hand side (Y → γ).
* ''Scanning'': If ''a'' is the next symbol in the input stream, for every state in S(''k'') of the form (X → α • ''a'' β, ''j''), add (X → α ''a'' • β, ''j'') to S(''k''+1).
* ''Completion'': For every state in S(''k'') of the form (Y → γ •, ''j''), find all states in S(''j'') of the form (X → α • Y β, ''i'') and add (X → α Y • β, ''i'') to S(''k'').

Duplicate states are not added to the state set, only new ones.  These three operations are repeated until no new states can be added to the set.  The set is generally implemented as a queue of states to process, with the operation to be performed depending on what kind of state it is.

The algorithm accepts if (X → γ •, 0) ends up in S(''n''), where (X → γ) is the top level-rule and ''n'' the input length, otherwise it rejects.

== Pseudocode ==
Adapted from Speech and Language Processing<ref name=Jurafsky>{{cite book|last=Jurafsky|first=D.|title=Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition|year=2009|publisher=Pearson Prentice Hall|isbn=9780131873216|url=https://books.google.com/books?id=fZmj5UNK8AQC}}</ref> by Daniel Jurafsky and James H. Martin, 

<source lang="pascal">
DECLARE ARRAY S;

function INIT(words)
    S ← CREATE_ARRAY(LENGTH(words) + 1)
    for k ← from 0 to LENGTH(words) do
        S[k] ← EMPTY_ORDERED_SET

function EARLEY_PARSE(words, grammar)
    INIT(words)
    ADD_TO_SET((γ → •S, 0), S[0])
    for k ← from 0 to LENGTH(words) do
        for each state in S[k] do  // S[k] can expand during this loop
            if not FINISHED(state) then
                if NEXT_ELEMENT_OF(state) is a nonterminal then
                    PREDICTOR(state, k, grammar)         // non_terminal
                else do
                    SCANNER(state, k, words)             // terminal
            else do
                COMPLETER(state, k)
        end
    end
    return chart

procedure PREDICTOR((A → α•Bβ, j), k, grammar)
    for each (B → γ) in GRAMMAR_RULES_FOR(B, grammar) do
        ADD_TO_SET((B → •γ, k), S[k])
    end

procedure SCANNER((A → α•aβ, j), k, words)
    if j < LENGTH(words) and a ⊂ PARTS_OF_SPEECH(words[k]) then
        ADD_TO_SET((A → αa•β, j), S[k+1])
    end

procedure COMPLETER((B → γ•, x), k)
    for each (A → α•Bβ, j) in S[x] do
        ADD_TO_SET((A → αB•β, j), S[k])
    end
</source>

== Example ==
Consider the following simple grammar for arithmetic expressions:
<source lang="text">
<P> ::= <S>      # the start rule
<S> ::= <S> "+" <M> | <M>
<M> ::= <M> "*" <T> | <T>
<T> ::= "1" | "2" | "3" | "4"
</source>
With the input:
 2 + 3 * 4

This is the sequence of state sets:
{| class="wikitable"
! (state no.) !! Production !! (Origin) !! Comment
|-----------------------------------------
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(0): • 2 + 3 * 4
|-
| 1 || style="font-family:monospace" |P → • S          || 0 ||  start rule 
|-
| 2 || style="font-family:monospace" |S → • S + M      || 0 ||  predict from (1) 
|-
| 3 || style="font-family:monospace" |S → • M          || 0 ||  predict from (1) 
|-
| 4 || style="font-family:monospace" |M → • M * T      || 0 ||  predict from (3) 
|-
| 5 || style="font-family:monospace" |M → • T          || 0 ||  predict from (3) 
|-
| 6 || style="font-family:monospace" |T → • number     || 0 ||  predict from (5) 
|-
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(1): 2 • + 3 * 4
|-
| 1 || style="font-family:monospace" |T → number •     || 0 ||  scan from S(0)(6) 
|-
| 2 || style="font-family:monospace" |M → T •          || 0 ||  complete from (1) and S(0)(5) 
|-
| 3 || style="font-family:monospace" |M → M • * T      || 0 ||  complete from (2) and S(0)(4) 
|-
| 4 || style="font-family:monospace" |S → M •          || 0 ||  complete from (2) and S(0)(3) 
|-
| 5 || style="font-family:monospace" |S → S • + M      || 0 ||  complete from (4) and S(0)(2) 
|-
| 6 || style="font-family:monospace" |P → S •          || 0 ||  complete from (4) and S(0)(1) 
|-
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(2): 2 + • 3 * 4 
|-
| 1 || style="font-family:monospace" |S → S + • M      || 0 ||  scan from S(1)(5) 
|-
| 2 || style="font-family:monospace" |M → • M * T      || 2 ||  predict from (1) 
|-
| 3 || style="font-family:monospace" |M → • T          || 2 ||  predict from (1) 
|-
| 4 || style="font-family:monospace" |T → • number     || 2 ||  predict from (3) 
|-
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(3): 2 + 3 • * 4
|-
| 1 || style="font-family:monospace" |T → number •     || 2 ||  scan from S(2)(4) 
|-
| 2 || style="font-family:monospace" |M → T •          || 2 ||  complete from (1) and S(2)(3) 
|-
| 3 || style="font-family:monospace" |M → M • * T      || 2 ||  complete from (2) and S(2)(2) 
|-
| 4 || style="font-family:monospace" |S → S + M •      || 0 ||  complete from (2) and S(2)(1) 
|-
| 5 || style="font-family:monospace" |S → S • + M      || 0 ||  complete from (4) and S(0)(2) 
|-
| 6 || style="font-family:monospace" |P → S •          || 0 ||  complete from (4) and S(0)(1) 
|-
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(4): 2 + 3 * • 4
|-
| 1 || style="font-family:monospace" |M → M * • T      || 2 ||  scan from S(3)(3) 
|-
| 2 || style="font-family:monospace" |T → • number     || 4 ||  predict from (1) 
|-
! scope="row" colspan="4" style="text-align:left; background:#e9e9e9;font-family:monospace" | S(5): 2 + 3 * 4 •
|-
| 1 || style="font-family:monospace" |T → number •     || 4 ||  scan from S(4)(2) 
|-
| 2 || style="font-family:monospace" |M → M * T •      || 2 ||  complete from (1) and S(4)(1) 
|-
| 3 || style="font-family:monospace" |M → M • * T      || 2 ||  complete from (2) and S(2)(2) 
|-
| 4 || style="font-family:monospace" |S → S + M •      || 0 ||  complete from (2) and S(2)(1) 
|-
| 5 || style="font-family:monospace" |S → S • + M      || 0 ||  complete from (4) and S(0)(2) 
|-
| 6 || style="font-family:monospace" |P → S •          || 0 ||  complete from (4) and S(0)(1) 
|-
|}
The state (P → S •, 0) represents a completed parse.  This state also appears in S(3) and S(1), which are complete sentences.

== Constructing the parse forest ==
Earley's dissertation<ref name=Earley3>{{cite book
 | last=Earley
 | first=Jay
 | title=An Efficient Context-Free Parsing Algorithm
 | year=1968
 | publisher=Carnegie-Mellon Dissertation
 | page=106
 | url=http://reports-archive.adm.cs.cmu.edu/anon/anon/usr/ftp/scan/CMU-CS-68-earley.pdf}}</ref> briefly describes an algorithm for constructing parse trees by adding a set of pointers from each non-terminal in an Earley item back to the items that caused it to be recognized.  But Tomita noticed<ref>{{cite book|last1=Tomita|first1=Masaru|title=Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems|date=April 17, 2013|publisher=Springer Science and Business Media|isbn=978-1475718850|page=74|url=https://books.google.com/books?id=DAjkBwAAQBAJ&q=Tomita%20Efficient%20Parsing%20for%20natural%20Language&pg=PA74|access-date=16 September 2015}}</ref> that this does not take into account the relations between symbols, so if we consider the grammar S → SS | b and the string bbb, it only notes that each S can match one or two b's, and thus produces spurious derivations for bb and bbbb as well as the two correct derivations for bbb.

Another method<ref>{{cite journal|last1=Scott|first1=Elizabeth|title=SPPF-Style Parsing From Earley Recognizers|journal=Electronic Notes in Theoretical Computer Science|date=April 1, 2008|volume=203|issue=2|pages=53–67|doi=10.1016/j.entcs.2008.03.044|doi-access=free}}</ref> is to build the parse forest as you go, augmenting each Earley item with a pointer to a shared packed parse forest (SPPF) node labelled with a triple (s, i, j) where s is a symbol or an LR(0) item (production rule with dot), and i and j give the section of the input string derived by this node. A node's contents are either a pair of child pointers giving a single derivation, or a list of "packed" nodes each containing a pair of pointers and representing one derivation.  SPPF nodes are unique (there is only one with a given label), but may contain more than one derivation for [[Philosophy:Syntactic ambiguity|ambiguous]] parses.  So even if an operation does not add an Earley item (because it already exists), it may still add a derivation to the item's parse forest.

* Predicted items have a null SPPF pointer.
* The scanner creates an SPPF node representing the non-terminal it is scanning.
* Then when the scanner or completer advance an item, they add a derivation whose children are the node from the item whose dot was advanced, and the one for the new symbol that was advanced over (the non-terminal or completed item).

SPPF nodes are never labeled with a completed LR(0) item: instead they are labelled with the symbol that is produced so that all derivations are combined under one node regardless of which alternative production they come from.

== Optimizations ==

Philippe McLean and R. Nigel Horspool in their paper [https://link.springer.com/content/pdf/10.1007%2F3-540-61053-7_68.pdf "A Faster Earley Parser"] combine Earley parsing with LR parsing and achieve an improvement in an order of magnitude.

== See also ==
* [[CYK algorithm]]
* [[Context-free grammar]]
* [[List of algorithms#Parsing|Parsing algorithms]]

== Citations ==
{{Reflist}}

== Other reference materials ==
*{{cite journal
 | last1 = Aycock | first1 = John
 | doi = 10.1093/comjnl/45.6.620
 | issue = 6
 | journal = [[The Computer Journal]]
 | pages = 620–630
 | title = Practical Earley Parsing
 | volume = 45
 | year = 2002| citeseerx = 10.1.1.12.4254
 }}
*{{citation
 | last = Leo | first = Joop M. I. M.
 | doi = 10.1016/0304-3975(91)90180-A
 | issue = 1
 | journal = Theoretical Computer Science
 | mr = 1112117
 | pages = 165–176
 | title = A general context-free parsing algorithm running in linear time on every LR(''k'') grammar without using lookahead
 | volume = 82
 | year = 1991
| doi-access = free
 }}

*{{cite conference |first= Masaru|last= Tomita|title= LR parsers for natural languages |conference= 10th International Conference on Computational Linguistics |book-title= COLING|pages= 354–357|year= 1984|url=https://aclanthology.info/pdf/P/P84/P84-1073.pdf}}

== Implementations ==

=== C, C++ ===
* [https://github.com/vnmakarov/yaep 'Yet Another Earley Parser (YAEP)'] – [[C (programming language)|C]]/[[C++]] libraries

=== Haskell ===
* [https://hackage.haskell.org/package/Earley 'Earley'] – an Earley parser [[Domain-specific language|DSL]] in [[Haskell (programming language)|Haskell]]

=== Java ===
* [http://www.cs.umanitoba.ca/~comp4190/Earley/Earley.java] – a Java implementation of the Earley algorithm
* [https://web.archive.org/web/20160303182122/http://linguateca.dei.uc.pt/index.php?sep=recursos PEN] – a Java library that implements the Earley algorithm
* [http://www.coffeeblack.org/#projects-pep Pep] – a Java library that implements the Earley algorithm and provides charts and parse trees as parsing artifacts
* [https://github.com/digitalheir/java-probabilistic-earley-parser digitalheir/java-probabilistic-earley-parser] - a Java library that implements the probabilistic Earley algorithm, which is useful to determine the most likely parse tree from an ambiguous sentence

=== C# ===

* [https://github.com/coonsta/earley coonsta/earley] - An Earley parser in C#
* [https://github.com/patrickhuber/Pliant patrickhuber/pliant] - An Earley parser that integrates the improvements adopted by Marpa and demonstrates Elizabeth Scott's tree building algorithm.
* [https://github.com/ellisonch/CFGLib ellisonch/CFGLib] - Probabilistic Context Free Grammar (PCFG) Library for C# (Earley + SPPF, CYK)

=== JavaScript ===
* [https://github.com/Hardmath123/nearley Nearley] – an Earley parser that's starting to integrate the improvements that Marpa adopted
* [https://joshuagrams.github.io/pep/ A Pint-sized Earley Parser] – a toy parser (with annotated pseudocode) to demonstrate Elizabeth Scott's technique for building the shared packed parse forest
* [https://github.com/lagodiuk/earley-parser-js lagodiuk/earley-parser-js] – a tiny JavaScript implementation of Earley parser (including generation of the parsing-forest)
* [https://github.com/digitalheir/probabilistic-earley-parser-javascript digitalheir/probabilistic-earley-parser-javascript] - JavaScript implementation of the probabilistic Earley parser

=== OCaml ===
* [https://github.com/tomjridge/tjr_simple_earley Simple Earley] - An implementation of a simple Earley-like parsing algorithm, with documentation.

=== Perl ===
* [https://metacpan.org/module/Marpa::R2 Marpa::R2] – a [[Perl]] module.  [https://jeffreykegler.github.com/Marpa-web-site/ Marpa] is an Earley's algorithm that includes the improvements made by Joop Leo, and by Aycock and Horspool.
* [https://metacpan.org/module/Parse::Earley Parse::Earley] – a Perl module implementing Jay Earley's original algorithm

=== Python ===
* [https://github.com/erezsh/lark/blob/master/lark/parsers/earley.py Lark] – an object-oriented, procedural implementation of an Earley parser, that outputs a SPPF.
* [http://nltk.org/ NLTK] – a [[Python (programming language)|Python]] toolkit with an Earley parser
* [http://pages.cpsc.ucalgary.ca/~aycock/spark/ Spark] – an object-oriented ''little language framework'' for Python implementing an Earley parser
* [https://pypi.python.org/pypi/spark_parser spark_parser] – updated and packaged version of the Spark parser above, which runs in both Python 3 and Python 2
* [https://github.com/tomerfiliba/tau/blob/master/earley3.py earley3.py] – a stand-alone implementation of the algorithm in less than 150 lines of code, including generation of the parsing-forest and samples
* [https://github.com/tomjridge/tjr_python_earley_parser tjr_python_earley_parser] - a minimal Earley parser in Python
* [https://rahul.gopinath.org/post/2021/02/06/earley-parsing/ Earley Parsing] - A well explained and complete Earley parser tutorial in Python with epsilon handling and Leo optimization for right-recursion.

=== Rust ===
* [https://github.com/kamadorueda/santiago Santiago] – A lexing and parsing toolkit for Rust implementing an Earley parser.

=== Common Lisp ===
* [http://www.cliki.net/CL-EARLEY-PARSER CL-Earley-parser] – a Common Lisp library implementing an Earley parser

=== Scheme, Racket ===
* [https://web.archive.org/web/20160401103410/http://www.cavar.me/damir/charty/scheme/ Charty-Racket] – a [[Scheme (programming language)|Scheme]]-[[Racket (programming language)|Racket]] implementation of an Earley parser

=== Wolfram ===

* [https://library.wolfram.com/infocenter/Articles/9792/ properEarleyParser] - A basic minimal implementation of an Earley parser in Wolfram programming language with some essential test cases.

=== Resources ===
* [http://accent.compilertools.net/Entire.html The Accent compiler-compiler]

{{parsers}}
[[Category:Parsing algorithms]]
[[Category:Dynamic programming]]

{{Sourceattribution|Earley parser}}