BQL->SQL compiler.
To compile a parsed BQL query:
Compiled SQL output accumulator.
Like a write-only StringIO.StringIO(), but also does bookkeeping for parameters and subqueries.
Return a selection of bindings fit for the accumulated output.
If there were subqueries, or if this is accumulating output for a subquery, this may not use all bindings.
Return the accumulated output.
Return an output accumulator for a subquery.
Accumulate text in the output of getvalue().
Accumulate a reference to the parameter name numbered n.
Accumulate a reference to the parameter numbered n.
Perform queries winders before and unwinders after.
Each of winders and unwinders is a list of (<sql>, <bindings>) tuples.
Compile query, writing output to output.
| Parameters: |
|
|---|
BQL execution.
This module implements the main dispatcher for executing different kinds of BQL phrases. Queries, as in SELECT, ESTIMATE, and so on, are compiled into SQL; commands, as in CREATE TABLE, INSERT, and the rest of the DDL/DML (Data Definition/Modelling language) are executed directly.
Cursor for a BQL or SQL query from a BayesDB.
Execute the BQL AST phrase phrase and return a cursor of results.
Miscellaneous utilities for managing BayesDB entities.
Tables, generators, and columns are named with strs. Only US-ASCII is allowed, no Unicode.
Each table has a nonempty sequence of named columns. As in sqlite3, tables may be renamed and do not necessarily have numeric ids, so there is no way to have a handle on a table that is persistent outside a savepoint.
Each table may optionally be modelled by any number of generators, representing a parametrized generative model for the table’s data, according to a named metamodel.
Each generator models a subset of the columns in its table, which are called the modelled columns of that generator. Each column in a generator has an associated statistical type. Like tables, generators may be renamed. Unlike tables, each generator has a numeric id, which is never reused and therefore persistent across savepoints.
Each generator may have any number of different models, each representing a particular choice of parameters for the parametrized generative model. Models are numbered consecutively for the generator, and may be identified uniquely by (generator_id, modelno) or (generator_name, modelno).
Add a generator’s latent variable to a population.
NOTE: To be used ONLY by a metamodel’s create_generator method when establishing any latent variables of that generator.
Adds a variable to the population, with colno from the base table.
Return the name of the column numbered colno in generator_id.
Return a list of names of columns modelled by generator_id.
Return the number of the column column_name in generator_id.
Return a list of the numbers of columns modelled in generator_id.
Return the statistical type of the column colno in generator_id.
True if generator_id models a column named name.
True if generator_id has a model numbered modelno.
Return the metamodel of the generator with id id.
Return the name of the generator with id id.
Return the id of the population of the generator with id id.
Return the name of the table of the generator with id id.
Return the id of the generator named name in bdb.
The id is persistent across savepoints: ids are 64-bit integers that increase monotonically and are never reused.
bdb must have a generator named name. If you’re not sure, call bayesdb_has_generator() first.
Return the id of the population named name in bdb.
The id is persistent across savepoints: ids are 64-bit integers that increase monotonically and are never reused.
bdb must have a population named name. If you’re not sure, call bayesdb_has_population() first.
True if there is a generator named name in bdb.
True if the population has a latent variable by the given name.
True if there is a population named name in bdb.
True if there is a table named name in bdb.
The table need not be modelled.
True if the population has a given variable.
generator_id is None for manifest variables and the id of a generator for variables that may be latent.
Return the name of the population with id id.
Return the name of table of the population with id id.
Return the name of the column numbered colno in table.
bdb must have a table named table. If you’re not sure, call bayesdb_has_table() first.
WARNING: This may modify the database by populating the bayesdb_column table if it has not yet been populated.
Return a list of names of columns in the table named table.
The results strs and are ordered by column number.
bdb must have a table named table. If you’re not sure, call bayesdb_has_table() first.
WARNING: This may modify the database by populating the bayesdb_column table if it has not yet been populated.
Return the number of column named name in table.
bdb must have a table named table. If you’re not sure, call bayesdb_has_table() first.
WARNING: This may modify the database by populating the bayesdb_column table if it has not yet been populated.
Make sure bayesdb_column is populated with columns of table.
bdb must have a table named table. If you’re not sure, call bayesdb_has_table() first.
True if the table named table has a column named name.
bdb must have a table named table. If you’re not sure, call bayesdb_has_table() first.
WARNING: This may modify the database by populating the bayesdb_column table if it has not yet been populated.
Return the name a population variable.
Return a list of the names of columns modelled in population_id.
Return the column number of a population variable.
Return a list of the numbers of columns modelled in population_id.
Return the statistical type of a population variable.
BQL parser front end.
True if string has at least one complete BQL phrase or error.
False if empty or if the last BQL phrase is incomplete.
Yield each parsed BQL phrase AST in string.
Yield (phrase, pos) for each BQL phrase in string.
phrase is the parsed AST. pos is zero-based index of the code point at which phrase starts.
Return (phrase, pos) for the first BQL phrase in string.
May not report parse errors afterward.
SQLite3 utilities.
Return the sqlite3 column affinity corresponding to a type string.
SQLite3 connection context manager. On exit, runs close.
Execute a query returning a 1x1 table, and return its one value.
Do not call this if you cannot guarantee the result is a 1x1 table. Beware passing user-controlled input in here.
Quote name as a SQL identifier, e.g. a table or column name.
Do NOT use this for strings, e.g. inserting data into a table. Use query parameters instead.
Savepoint context manager. On return, commit; on exception, rollback.
Savepoints are like transactions, but they may be nested in transactions or in other savepoints.
Savepoint context manager that always rolls back.
Transaction context manager. On return, commit; on exception, rollback.
Transactions may not be nested. Use savepoints if you want a nestable analogue to transactions.
Miscellaneous statistics utilities.
Arithmetic mean of elements of array.
Pearson chi^2 test of independence statistic on contingency table.
https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test#Test_of_independence
Survival function for chi^2 distribution.
F-test statistic for one-way analysis of variance (ANOVA).
https://en.wikipedia.org/wiki/F-test#Multiple-comparison_ANOVA_problems
groups[i][j] is jth observation in ith group.
Approximate survival function for the F distribution.
f_sf(x, df_num, df_den) = P(F_{df_num, df_den} > x)
Summarize an array of data as (count, mean, standard deviation).
The algorithm is the “Online algorithm” presented in Knuth Volume 2, 3rd ed, p. 232, originally credited to “Note on a Method for Calculating Corrected Sums of Squares and Products” B. P. Welford Technometrics Vol. 4, No. 3 (Aug., 1962), pp. 419-420. This has the advantage over naively accumulating the sum and sum of squares that it is less subject to precision loss through massive cancellation.
This version collected 8/31/15 from https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
Pearson r, product-moment correlation coefficient, of two samples.
Covariance divided by product of standard deviations.
https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#For_a_sample
Sign of x: -1 if x<0, 0 if x=0, +1 if x>0.
Approximate CDF for Student’s t distribution.
t_cdf(x, df) = P(T_df < x)
Miscellaneous special math functions and analysis utilities.
This is not a general-purpose nor highly optimized math library: it is limited to the purposes of bayeslite for modest data sets, written for clarity and maintainability over speed.
Approximate summation of a nonnegative convergent sequence.
The sequence is assumed to converge to zero quickly without oscillating – it is truncated at the first term whose magnitude relative to the partial sum is bounded by the machine epsilon.
Continuants of a continued fraction.
contfrac must yield an infinite sequence (n0, d0), (n1, d1), (n2, d2), ..., representing the continued fraction:
n0
------------------
n1
d0 + -------------
n2
d1 + --------
d2 + ...
The kth continuant is the numerator and denominator of the continued fraction truncated at the kth term, i.e. with zero instead of the rest in the ellipsis.
If the numerator or denominator grows large in magnitude, both are multiplied by the machine epsilon, leaving their quotient unchanged.
Convergents of a continued fraction.
contfrac must yield an infinite sequence (n0, d0), (n1, d1), (n2, d2), ..., representing the continued fraction:
n0
------------------
n1
d0 + -------------
n2
d1 + --------
d2 + ...
The kth convergent is the continued fraction truncated at the kth term, i.e. with zero instead of the rest in the ellipsis.
Normalized upper incomplete gamma integral.
Equal to:
(1/\Gamma(a)) \int_x^\infty e^{-t} t^{a - 1} dt.
gamma_above is the complement of gamma_below:
gamma_above(a, x) = 1 - gamma_below(a, x).
As x goes to zero, gamma_above(a, x) converges to 1.
For x > max(1, a), this is computed by the continued fraction[1]:
1
-----------------------,
1 - s
x + -------------------
1
1 + ---------------
2 - s
x + -----------
2
1 + -------
x + ...
which is then multiplied by x^a e^{-x} / \Gamma(a).
For x <= max(1, a), this is computed by 1 - gamma_below(a, x).
[1] Abramowitz & Stegun, p. 263, 6.5.31
Normalized lower incomplete gamma integral.
Equal to:
(1/\Gamma(a)) \int_0^x e^{-t} t^{a - 1} dt.
gamma_below is the complement of gamma_above:
gamma_below(a, x) = 1 - gamma_above(a, x).
As x grows, gamma_below(a, x) converges to 1.
For x <= max(1, a), this is computed by the power series[1]:
x^a e^-x / x^2 x^3 \
----------- | 1 + x + ------- + -------------- + ... |.
a \Gamma(a) \ (a + 1) (a + 1)(a + 2) /
For x > max(1, a), this is computed by 1 - gamma_above(a, x).
[1] NIST Digital Library of Mathematical Functions, Release 1.0.9 of 2014-08-29, Eq. 8.7.1 <http://dlmf.nist.gov/8.7.E1>.
In the NIST DLMF notation, gamma_below(a, x) is P(a, x), related to \gamma^* by \gamma^*(a, x) = x^{-a} P(a, x).
To derive the power series, multiply (8.7.1) by x^a, expand \Gamma(a + k + 1) into \Gamma(a) a(a + 1)...(a + k), and factor a \Gamma(a) out of the sum.
Approximate limit of a convergent sequence.
The sequence is assumed to converge quickly without oscillating – it is truncated at the first term whose relative error from the previous term is bounded by the machine epsilon.
Sequence of partial sums of a sequence.
Relative error between expected and actual: abs((a - e)/e).
Miscellaneous utilities.
Return the sum of elements of iterable in floating-point.
This implementation uses Kahan-Babuška summation.
Return a JSON string of obj, compactly and deterministically.
Return a sorted array of the unique elements in array.
No element may be a floating-point NaN. If your data set includes NaNs, omit them before passing them here.
Return an array of the indices of the unique elements in array.
No element may be a floating-point NaN. If your data set includes NaNs, omit them before passing them here.