Language Parsing

Legacy Wiki Page

This page was migrated from the old MoinMoin-based wiki. Information may be outdated or no longer applicable. For current documentation, see python.org.

Small discussion and evaluation of different parsers.

💡 Please keep wiki links as wiki links, use external links only if there is no existing page for the tool.

Name

Grammar

Module

Python

Comment

shlex

C

included in the main Python distribution

Grako

PEG

Python

2.7+, 3.3+, PyPy

Tool that takes grammars in EBNF variant & and outputs memoizing (Packrat) PEG parsers in Python. Grako is different from other PEG parser generators in that the generated parsers use Python’s very efficient exception-handling system to backtrack.

reparse

Python/Regex

2.x, 3.x

Combines Regular Expressions

Plex

C

lexical analysis module for Python, foundation for Pyrex and Cython. Plex 2.0.0 is Python 2 only, the version embedded in Cython works in Python 3.x. There is also an experimental port to Python 3 (tested on Python 3.3)

Spark

Earley Parser

Python

2.3+, 3.2+, PyPy

PyPI package; github project. This parser is notably used in decompilers like uncompyle6 where using an ambigous grammar is desirable.

Yapps

LL(1)

Python

1-any, 2-1.5+

PyLR - (broken link)

LR(1) LALR(1)

C

kwParsing

PyBison

C

bison grammar with python code actions

Trap

LR

1.5.1+

PLY

SLR LALR(1)

Python

Python Lex-Yacc

ToyParserGenerator

2.2+

DParser

GLR

C

2.2+

grammar in doc strings

PyGgy - (broken link)

GLR

Python

2.2.1

pyPEG

PEG

Python

2.5+

parsimonious

PEG

Python

2.5+

SimpleParse

-

2.0+

requires mxTextTools

Martel

Python

2.0+

requires mxTextTools

mxTextTools

-

C

is not exactly a parser like we’re used to, but it is a fast text-processing engine

pyparsing

PEG, LR

Python

3.9+

parcon

Python

2.6+

Parser combinator library, similar to pyparsing

ANTLR

LL1+

Python

stand-alone tool in Java. Latest version can produce Python code

Yappy

LR(0) LR(1) SLR LALR(1)

Python

2.2+

ZestyParser

Python

Object-oriented, Pythonic parsing

Parsing

LR(1)

Python

2.5+

aperiot

LL(1)

Python

uses separate grammar files

yeanpypa

Python

inspired by pyparsing and boost::spirit

Wisent

LR(1)

Python

2.4+

has separate parser input file, parser output is a parse tree

RP

na

Python

2.6+

Simple parser using rule defined in BNF format

LEPL

Any

Python

2.6+,3+

Recursive descent with full backtracking and optional memoisation (which can handle left recursive grammars). So equivalent to GLR, but based on LL(k) core.

modgrammar

GLR

Python

3.1+

Recursive descent parser with full backtracking. Grammar elements and results are defined as Python classes, so are fully customizable. Supports ambiguous grammars.

funcparserlib

LL(*)

Python

2.4+

Recursive descent parsing library for Python based on functional combinators

pydsl

-

Python

2.7+ 3+

lrparsing

LR(1)

Python

2.6+

A fast parser, lexer combination with a concise Pythonic interface. Lots of documentation, include example parsers for SQL and Lua.

Arpeggio

PEG

Python

2.7+, 3.2+

Packrat parser. Works as interpreter. Multiple syntaxes for grammar definition. Lots of docs, examples and tutorials.

textX

Python

2.7+, 3.2+

A high-level meta-language/parser for Domain-Specific Language implementation. Built on top of Arpeggio parser. Inspired by XText. Documentation, examples and tutorials available.

pyleri

LR

Python

3.2+

A fast, stand-alone parser which can export a grammar to JavaScript (jsleri), Go (goleri), C (libcleri) or Java (jleri).

parglare

LR/GLR

Python

2.7+, 3.3+

A pure Python LR/GLR parser with integrated scanner (scannerless). Grammar in BNF format. Automata/GLR trace visualization. Full documentation and examples available.

lark

LALR(1), CFG

Python

2.7, 3.4+

LALR(1) for speed or Earley parser for any context-free grammar.

For faster performance, one may use other parser generator systems and plug them in as modules.

For example:

Example of such usage is SeeGramWrap available from Edward C. Jones Python page, which is a heavily revised and upgraded version of the ANTLR C parser that is in cgram (broken link). The lastest verson has been refactored to move some of the complexity from ANTLR to Python.

Martin von Loewis presented a paper at Python10, titled “Towards a Standard Parser Generator” that surveyed the available parser generators for Python.

Additional information on these and other parsers at Python Parsing Tools.

Books