Open in new tab

Probly is a Python-like mini-language for probabilistic estimation. It's based on Starlark and implemented in Go.

Start with Example 1

Probly syntax

You may use any Starlark syntax. There are only the following differences to Starlark:

  • A variable may follow a probability distribution, in addition to usual types like numbers or dictionaries
  • The Starlark math module is imported by default, so you can directly use it e.g. math.sqrt(2). Probly also has a built-in sum function not available in Starlark.

This page will show you the probability distributions of all the numeric (scalar or distribution) global variables in your program (except those starting with an underscore). The values are taken at the end of the program's execution.

Probability distributions

Name to
p10 to p90
pm
plus/minus
td
times/divide
Quantiles Notes
Normal mean sd 2
LogNormal mu sigma 2 Alternatively: mean, sd
Beta alpha beta
PERT min mode max [lambd] Like the triangular, but smoother (Wikipedia)
Uniform a b 2 a need not be less than b
LogUniform a b 2 a need not be less than b
Bernoulli p
Binomial n p
Discrete x_1 p_1 x_2 p_2 ... Generic discrete distribution over any finite set of values

math

These mathematical functions and constants are available in the math module:

  • pow(x, y) - Returns x raised to the power of y
  • exp(x)
  • sqrt(x)
  • log(x, [base]) - Natural logarithm by default if base is not specified
  • e
  • pi
  • Ceil, floor, and sign manipulation:
    • ceil(x)
    • floor(x)
    • fabs(x) - Returns the absolute value of x as float
    • copysign(x, y) - Returns a value with the magnitude of x and the sign of y
  • mod(x, y) - Returns x modulo y
  • remainder(x, y)
  • round(x) - Returns the nearest integer, rounding half away from zero
  • Trigonometry (in radians unless otherwise specified):
    • acos(x)
    • asin(x)
    • atan(x)
    • atan2(y, x) - Returns atan(y / x). The result is between -pi and pi
    • cos(x)
    • sin(x)
    • tan(x)
    • degrees(x) - Converts angle x from radians to degrees
    • radians(x) - Converts angle x from degrees to radians
    • acosh(x)
    • asinh(x)
    • atanh(x)
    • cosh(x)
    • sinh(x)
    • tanh(x)
  • hypot(x, y) - Returns the Euclidean norm, sqrt(x^2 + y^2); the distance from the origin to (x, y)
  • gamma(x) - Returns the Gamma function at x

Starlark syntax

This code provides an example of the syntax of Starlark:

# Define a number
number = 18

# Define a list
numbers = [1, 2, 3, 4, 5]

# List comprehension
halves = [n / 2 for n in numbers]

# Define a function
def is_even(n):
    """Return True if n is even."""
    return n % 2 == 0

# Define a dictionary
people = {
    "Alice": 22,
    "Bob": 40,
    "Charlie": 55,
    "Dave": 14,
}

names = ", ".join(people.keys())  # Alice, Bob, Charlie, Dave

# Modify a variable in a loop
sum_even_ages = 0
for age in people.values():
    if is_even(age):
        sum_even_ages += age

# Append to a list in a loop
over_30_names = []
for name, age in people.items():
    if age > 30:
        over_30_names.append(name)

If you've ever used Python, this should look very familiar. In fact, the code above is also valid Python code. Still, this short example shows most of the language. Starlark is a very small language that implements a limited subset of Python.

For our purposes, one notable difference to Python is that the exponentiation operator ** is not supported. You have to use math.pow.

You can also look at the Starlark language specification.

Speed

Though not designed for speed, Probly is fast enough for practical purposes: around 10 milliseconds for 3,000 samples, for most examples on this page. This is due to being implemented in Go.

The time taken to return results on this page is spent overwhelmingly in web application code, not in Probly evaluation.

Interestingly, Probly is still slower than Python code that uses entirely numpy array operations, which are very well optimised. This should only begin to matter at very large scales, or if latency is critical.

Limitations

It's not currently possible to obtain and manipulate properties of a distribution within an Probly program, like so:

x = Normal(1 to 10)
y = x.std()  # Not possible

Supporting this would require some fundamental changes to the implementation of Probly, which is currently quite simple. I haven't prioritised this yet because I'm unsure how desirable the feature is.

Prior work

The to binary operator was inspired by Squiggle.

Example GiveWell iron and folic acid CEA

This example reproduces a 2018 GiveWell cost-effectiveness analysis for iron and folic acid (IFA) supplementation of school-age children.

The model considers the direct benefits of reducing anemia, as well as benefits via cognitive improvements and their effects on long-term wages. These benefits are weighed against potential risks like increased malaria mortality and side effects.

The final output is the humanitarian value per $10,000 spent, and a comparison of this value to the impact of donating the same amount to GiveDirectly.

In this model, the cost-effectiveness is largely driven by the long-term cognitive benefits of IFA supplementation.

Additional background information

GiveWell is a global health funder. They conduct in-depth research to estimate the cost-effectiveness of a given program — in terms of humanitarian benefit (e.g. lives saved) per dollar.

GiveWell developed this cost-effectiveness analysis in the context of this grant investigation. GiveWell also has a report about iron supplementation generally.

Distribution details

multiples_of_cash

Mean 8.15
Std. dev. 7.29
Variance 53.2
Quantile
0.05 2.24
0.25 3.82
0.50 6.00
0.75 9.78
0.95 21.3

value_per_10_000_usd

Mean 126
Std. dev. 91.3
Variance 8 340
Quantile
0.05 44.5
0.25 70.2
0.50 99.1
0.75 149
0.95 298

p_anemia

Mean 0.465
Std. dev. 0.074 1
Variance 0.005 49
Quantile
0.05 0.341
0.25 0.417
0.50 0.466
0.75 0.517
0.95 0.584

Simulation data

CSV

Download CSV

Preview

_cohort p_anemia value_per_10_000_usd multiples_of_cash
0 10 000 0.493 83.9 6.02
1 10 000 0.447 106 6.84
2 10 000 0.434 76.8 5.87
... ... ... ... ...
2997 10 000 0.519 161 30.1
2998 10 000 0.468 176 3.65
2999 10 000 0.462 122 3.41

API

Get the simulation data (and more) in a machine-readable format: /api/sim/BPvSymMGW37a6jqbTrcfRQ/