Raw File
Tip revision: d31b47629b8174d136eca79c37de7578fc36df22 authored by Robert Moss on 06 February 2024, 17:19:48 UTC
Update MCTS [compat]
Tip revision: d31b476
# POMDPStressTesting.jl

[![Build Status](](

Adaptive stress testing of black-box systems, implemented within the [POMDPs.jl]( ecosystem.

See the [documentation]( for more details.

# Citation

If you use this package for research purposes, please cite the following:


  title = {{POMDPStressTesting.jl}: Adaptive Stress Testing for Black-Box Systems},
  author = {Robert J. Moss},
  journal = {Journal of Open Source Software},
  year = {2021},
  volume = {6},
  number = {60},
  pages = {2749},
  doi = {10.21105/joss.02749}

# Interface
To stress test a new system, the user has to define the `GrayBox` and `BlackBox` interface outlined in [`src/GrayBox.jl`]( and [`src/BlackBox.jl`](

### GrayBox Interface
The `GrayBox` simulator and environment interface includes:
* `GrayBox.Simulation` type to hold simulation variables
* `GrayBox.environment(sim::Simulation)` to return the collection of environment distributions
* `GrayBox.transition!(sim::Simulation)` to transition the simulator, returning the log-likelihood

### BlackBox  Interface
The `BlackBox` system interface includes:
* `BlackBox.initialize!(sim::Simulation)` to initialize/reset the system under test
* `BlackBox.evaluate!(sim::Simulation)` to evaluate/execute the system under test
* `BlackBox.distance(sim::Simulation)` to return how close we are to an event
* `BlackBox.isevent(sim::Simulation)` to indicate if a failure event occurred
* `BlackBox.isterminal(sim::Simulation)` to indicate the simulation is in a terminal state

Functions ending with `!` may modify the `Simulation` object in place.

# Solvers
Several solvers are implemented.

#### Reinforcement learning solvers
* [`MCTSPWSolver`](

#### Deep reinforcement learning solvers<sup>1</sup>
* [`TRPOSolver`](
* [`PPOSolver`](

#### Stochastic optimization solvers
* [`CEMSolver`](

#### Baseline solvers
* [`RandomSearchSolver`](

# Example
[![Example Notebook](](

An example implementation of the AST interface is provided for the Walk1D problem:
* **Julia source**: [`test/Walk1D.jl`](
* **Jupyter notebook**: [`Walk1D.ipynb`](
* **Descriptive tutorial-style write-up**: [`walk1d.pdf`](./test/pdf/walk1d.pdf) (created using [TeX.jl](

<!-- ( -->

<p align="center">
  <a href="./test/pdf/walk1d.pdf">
    <img src="./test/svg/walk1d.svg">

<!-- With an accompanying notebook: [`Walk1D.ipynb`]( -->

# Installation

Install the `POMDPStressTesting.jl` package via:
] add POMDPStressTesting

### Testing
To run the test suite, you can use the Julia package manager.
] test POMDPStressTesting

# Contributing
We welcome contributions! Please fork the repository and submit a new Pull Request.

Package maintained by Robert Moss:

<sup>1</sup> TRPO and PPO thanks to [Shreyas Kowshik's]( initial implementation.
back to top