Adapter for Plex-like sub-scanners: module venture.parser.venture_script.subscanner¶
Adapter for using Plex-like scanner libraries for writing VentureScript sublanguages.
The standard VentureScript sublanguage interface (see
venture.ripl.ripl.Ripl.register_language()
) expects the
VentureScript parser to control the input stream, and expects the
sublanguage parser to manifest as a procedure that accepts characters
and reports whether the utterance is complete and what it parsed to.
Many parser and scanner libraries, however, expect to themselves
control the input stream, i.e. provide a method like read(stream) -> result
.
Notably, this includes Plex, the library in which the VentureScript
scanner itself is written.
This module provides an adapter which uses multithreading (!) to convert between the two interfaces, so a Plex-like scanner can be used to implement a VentureScript sublanguage.
-
class
venture.parser.venture_script.subscanner.
Reader
(char_queue, token_queue)¶ Bases:
object
An input stream for consumption by a Plex-like scanner for a VentureScript sublanguage.
The consuming scanner is expected to call
read
to fetch input. The requested block size is ignored and input is given one character at a time.Relevant instances will be constructed by
Scanner
; do not make instance ofReader
directly.-
read
(_n)¶
-
-
class
venture.parser.venture_script.subscanner.
Scanner
(scanner)¶ Bases:
object
Adapter from a Plex-like scanner to a callable scanner as needed in a VentureScript sublanguage.
Usage:
class MyScanner(Plex.Scanner): def __init__(self, stream): ... ripl.register_language("some name", lambda : Scanner(MyScanner))
The
scanner
object provided to__init__
must be a constructor for a Plex-like scanner. Specifically,scanner
will be called with one argument representing the input stream (an instance ofReader
). It must return an object that has aread()
method, which, when called, consumes some quantity of the input stream represented by theReader
and returns a sequence consisting of the parsed result in the first location and an arbitrary collection of additional values. (Plex scanners return a 2-tuple by default.)The input scanner must consume only as much input as needed to determine the boundary of a complete utterance in its language, as additional characters cannot be put back into the input stream for consumption by the surrounding VentureScript parser. The sublanguage must consume the closing curly brace in the
@{<name> <utterance>}
syntax, but no additional characters.