Suppose I want to learn the value of a parameter $$\theta\in\mathbb{R}$$. My prior is that $$\theta$$ is normally distributed with variance $$\sigma_0^2$$. I observe $$n\ge1$$ signals $$\DeclareMathOperator{\Cor}{Cor} \DeclareMathOperator{\E}{E} \DeclareMathOperator{\Var}{Var} \newcommand{\R}{\mathbb{R}} \renewcommand{\epsilon}{\varepsilon} s_i=\theta+\epsilon_i$$ of $$\theta$$. The errors $$\epsilon_i$$ in these signals are independent of $$\theta$$. They are jointly normally distributed with equal variances $$\Var(\epsilon_i)=\sigma^2$$ and pairwise correlations $$\Cor(\epsilon_i,\epsilon_j)=\begin{cases} 1 & \text{if}\ i=j \\ \rho & \text{otherwise}. \end{cases}$$ I assume $$-1/(n-1)\le\rho\le1$$ so that this distribution is feasible.1

Observing $$s_1,s_2,\ldots,s_n$$ is the same to observing the sample mean $$\bar{s}_n\equiv\frac{1}{n}\sum_{i=1}^ns_i,$$ which is normally distributed and has conditional variance $$\Var(\bar{s}_n\mid\theta)=\frac{(1+(n-1))\rho\sigma^2}{n}$$ under my prior. The posterior distribution of $$\theta$$ given $$\bar{s}_n$$ is also normal and has variance $$\Var(\theta\mid\bar{s}_n)=\left(\frac{1}{\sigma_0^2}+\frac{n}{(1+(n-1)\rho)\sigma^2}\right)^{-1}.$$ Both variances are (i) decreasing in $$n$$ when $$\rho<1$$ and (ii) increasing in $$\rho$$ when $$n>1$$. Intuitively, if the signals are not perfectly correlated then observing more gives me more information about $$\theta$$. If they are negatively correlated then their errors “cancel out” and the sample mean $$\bar{s}_n$$ gives me a precise estimate of $$\theta$$.

The chart below shows how $$\Var(\bar{s}_n\mid\theta)$$ and $$\Var(\theta\mid\bar{s}_n)$$ vary with $$\rho$$ and $$n$$ when $$\sigma_0=\sigma=1$$. If $$\rho=-1/(n-1)$$ then $$\epsilon_1+\epsilon_2+\cdots+\epsilon_n=0$$, and so $$\Var(\bar{s}_n\mid\theta)=0$$ and $$\Var(\theta\mid\bar{s}_n)=0$$ because $$\bar{s}_n=\theta$$. Whereas if $$\rho=1$$ then signals $$s_2$$ through $$s_n$$ provide the same information as $$s_1$$, and so $$\Var(\bar{s}_n\mid\theta)=\Var(s_1\mid\theta)$$ and $$\Var(\theta\mid\bar{s}_n)=\Var(\theta\mid s_1)$$.

1. For example, it is impossible for three normal variables to have equal variances and pairwise correlations of $$-1$$. See here for an explanation. ↩︎