- Erratum
- Open Access
- Published:

# Erratum to: Generalized Thompson sampling for sequential decision-making and causal inference

*Complex Adaptive Systems Modeling*
**volume 2**, Article number: 4 (2014)

- The original article was published in Complex Adaptive Systems Modeling 2014 2:2

## Abstract

No abstract.

## Decisions in the presence of latent variables

We correct errors in equations (14), (15) and (19) of the main text.

### Equations (14) and (15)

Nature’s probability of flipping either coin does not actually depend on the agent’s prediction, so we can replace the conditional probabilities *p*_{0}(*θ*|*x*) by *p*_{0}(*θ*). We have then an inner variational problem:

with the solution

and the normalization constant ${Z}_{\beta}\left(x\right)={\Sigma}_{\theta}{p}_{0}\left(\theta \right)exp\left(\beta U(x,\theta )\right)$ and an outer variational problem as described by equation (16) in the main text. Note that deliberation renders the two variables *x* and *θ* dependent.

### Equation (19)

In the case of *α* = *β* and uniform prior ${p}_{0}\left(x\right)=\mathcal{U}\left(x\right)$, equation (17) reduces to

where ${Z}_{\alpha}=\underset{x}{\Sigma}\underset{\theta}{\Sigma}{p}_{0}\left(\theta \right){e}^{\alpha U(x,\theta )}$. Note that *e*^{αU(x,θ)}/*Z*_{
α
} is in general not a conditional distribution. However, equation (19) can be equivalently rewritten as

where we have expanded the fraction by $\underset{{x}^{\text{'}}}{\Sigma}{e}^{\alpha U({x}^{\text{'}},\theta )}$.

This last equality can also be obtained by stating the same variational problem in reverse causal order of *x* and *θ*, which is the natural statement of the Thompson sampling problem. The nested variational problem then becomes

with the solutions

and

with normalization constant ${Z}_{\beta \alpha}=\underset{\theta}{\Sigma}{p}_{0}\left(\theta \right)exp\left(\beta /\alpha \phantom{\rule{0.3em}{0ex}}log\underset{x}{\Sigma}{p}_{0}\left(x\right){e}^{\alpha U(x,\theta )}\right)$. In the limit *α* → *∞* and *β* → 0, the Thompson sampling agent is determined by the solutions $p\left(x\right|\theta )=\delta (x-arg\underset{{x}^{\text{'}}}{max}U({x}^{\text{'}},\theta ))$ and *p*(*θ*)=*p*_{0}(*θ*). Sampling an action from $p\left(x\right)=\underset{\theta}{\Sigma}p\left(\theta \right)p\left(x\right|\theta )$ is much cheaper than sampling an action from equation (18) because of the reversed causal order in *θ* and *x*, which implies that *β*/*α*→ 0 in equation (ii) instead of *β*/*α*→*∞* as in equation (17).

In the case of *α*=*β* the solutions for the two different causal orders of *x* and *θ* are equivalent. Assuming again a uniform prior ${p}_{0}\left(x\right)=\mathcal{U}\left(x\right)$, we can compute the Thompson sampling agent from equation (i) and equation (ii) for *α*=*β* to be

which is exactly equivalent to *p*(*x*) in equation (19). To sample from equation (19), we draw *θ*~*p*_{0}(*θ*) and accept $x~{p}_{0}\left(x\right)=\mathcal{U}\left(x\right)$ if *u*≤*e*^{αU(x,θ)}/*e*^{αT}, where $u~\mathcal{U}[\phantom{\rule{0.3em}{0ex}}0;1]$.

## References

- 1.
Ortega, PA, Braun DA: Generalized Thompson sampling for sequential decision-making and causal inference.

*Complex Adaptive Systems Modeling*2014, 2: 2.

## Author information

### Affiliations

### Corresponding authors

## Additional information

The online version of the original article can be found at 10.1186/2194-3206-2-2

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

### Cite this article

Ortega, P.A., Braun, D.A. Erratum to: Generalized Thompson sampling for sequential decision-making and causal inference.
*Complex Adapt Syst Model* **2, **4 (2014). https://doi.org/10.1186/s40294-014-0004-x

Received:

Accepted:

Published: