Lecture 16

In-class notes: CS 505 Spring 2025 Lecture 16

Error Reduction for BPP

Recall that a language $L$ is in BPP if there exists a strict polynomial time probabilistic Turing machine $M$ such that for any $x$, $\mathrm{Pr}[M(x)=L(x)]\ge 2/3$. Equivalently stated, for a deterministic Turing machine $M$ running in polynomial time, it holds that ${\mathrm{Pr}}_r[M(x,r)=L(x)]\ge 2/3$ for all $x$, where $r\in \{0,1{\}}^{\mathrm{poly}(|x|)}$.

Here, the error probability $2/3$ is convenient, but arbitrary. We’ll see that the class BPP is equivalently defined for any error probability at least $1/2$.

Definition. For any $\epsilon \in (0,1/2)$, the class ${\mathbf{BPP}}_{1/2+\epsilon }$ is the set of all languages $L$ such that there exists a probabilistic Turing machine $N$ running in strict polynomial time such that for all $x$, $\mathrm{Pr}[M(x)=L(x)]\ge 1/2+\epsilon (|x|)$.

Under the above definition, we can set $\epsilon =n^{-c}$ for constant $c>0$ for $n=|x|$, or even $\epsilon =1/2-2^{-n^c}$. We’ll show that this BPP class with reduced error is equivalent to the standard BPP definition.

Theorem. For all constants $c>0$, it holds that $\mathbf{BPP}={\mathbf{BPP}}_{1/2+n^{-c}}$.

Proof. Clearly, $\mathbf{BPP}\subseteq {\mathbf{BPP}}_{1/2+n^{-c}}$ since $2/3\ge 1/2+n^{-c}$ for all $c>0$ and large enough $n\ge 2$. Next, we show the other direction, namely ${\mathbf{BPP}}_{1/2+n^{-c}}\subseteq \mathbf{BPP}$. We show this via the following claim.

Claim. Suppose $L$ is decidable by a PTM (in polynomial time) $M$ such that for all $x$, $\mathrm{Pr}[M(x)=L(x)]\ge 1/2+|x{|}^{-c}$ for some constant $c>0$. Then there exists a PTM $M^{\prime }$ running in polynomial time and a constant $d>0$ such that for all $x$, $\mathrm{Pr}[M^{\prime }(x)=L(x)]\ge 1-2^{-|x{|}^d}$.

We won’t show the full proof, but just the main ideas. The idea is that the machine $M^{\prime }$ will simply run $M$ for some polynomial number of times, then output the majority of the outputs. Let $k=q(|x|)=8|x{|}^{2c+d}+1$. Then, $M^{\prime }(x)$ will independently run $M(x)$ $k$ times. Let $b_1,\dots ,b_k\in \{0,1\}$ be the output bits of the $k$ independent runs of $M(x)$. Then, $M^{\prime }(x)$ outputs $b=\mathrm{majority}(b_1,\dots ,b_k)$, where $b=1$ if $\lceil k/2\rceil$ of the bits $b_1,\dots ,b_k$ are $1$; otherwise, $b=0$. It can be shown using the Chernoff bound that if $\mathrm{Pr}[M(x)=L(x)]\ge 1/2+|x{|}^{-c}$, then $\mathrm{Pr}[M^{\prime }(x)=L(x)]\ge 1-2^{-|x{|}^d}$ under the parameters we’ve set. Notice also that $M^{\prime }(x)$ runs in polynomial time since it runs $M(x)$ a polynomial number of times.

With the claim, for large enough $n=|x|$, it holds that $1-2^{-|x{|}^d}\ge 2/3$, which shows that $L\in \mathbf{BPP}$ whenever $L\in {\mathbf{BPP}}_{1/2+n^{-c}}$. $\square$

BPP vs. Other Classes

Now, we examine the relationship between BPP and other complexity classes.

BPP vs P

First, we naturally ask: what is the relationship between P and BPP? Clearly, $\mathbf{P}\subseteq \mathbf{BPP}$ since any deterministic Turing machine is also probabilistic (you can either set ${\delta }_0={\delta }_1$, or have $M(x,r)$ ignore the random input $r$). Many complexity theorists actually believe that $\mathbf{P}=\mathbf{BPP}$, which concerns the rich field of derandomization and hardness amplification. But we will not examine these fields in this course.

BPP vs PH

At first glance, it is not clear what the relationship between the polynomial hierarchy and BPP is. It turns out that BPP sits low in the polynomial hierarchy.

Theorem. $\mathbf{BPP}\subseteq {\Sigma }_2^p\cap {\Pi }_2^p$.

Proof. Note that since $\mathbf{BPP}=\mathbf{coBPP}$, it suffices to show either $\mathbf{BPP}\subseteq {\Sigma }_2^p$ or ${\Pi }_2^p$. We show that $\mathbf{BPP}\subseteq {\Sigma }_2^p$.

Let $L\in {\mathbf{BPP}}_{1-2^{-n}}$ (which is equivalent to BPP by the previous theorem and claim). Using the DTM definition of BPP, there exists a DTM $M$ running in polynomial time such that for all $x$, it holds that ${\mathrm{Pr}}_r[M(x,r)=L(x)]\ge 1-2^{-|x|}$, where $r\in \{0,1{\}}^{p(|x|)}$ for some polynomial $p$.

Let $m=p(n)$ for $n=|x|$. We define a set $S_x\subseteq \{0,1{\}}^m$ as the set of all good strings $r$ for $x$. That is, $S_x=\{r\in \{0,1{\}}^m|M(x,r)=1\}$; i.e., the set of all $r$ such that $M(x,r)=1$. Otherwise, if $r\notin S_x$, we say that $r$ is bad for $x$.

1. Notice that if $x\in L$, it holds that $|S_x|\ge 2^m\cdot (1-2^{-n})$. This is because ${\mathrm{Pr}}_r[M(x,r)=1]\ge 1-2^{-n}$ when $x\in L$, and thus there must be at least $1-2^{-n}$ strings $r$ such that $M(x,r)=1$.
2. If $x\notin L$, it holds that $|S_x|\le 2^m\cdot 2^{-n}=2^{m-n}$. This is because if $x\notin L$, then ${\mathrm{Pr}}_r[M(x,r)=1]\le 2^{-n}$, so there is at most $2^{-n}$ fraction of strings $r$ such that $M(x,r)=1$.

Now, the goal is to encode the set $S_x$ as a ${\Sigma }_2^p$ statement. We’ll need the following tool. For any $S\subseteq \{0,1{\}}^m$ and any vector $u\in \{0,1{\}}^m$, define $S+u:=\{s+u|s\in S\}$. Now set $k=\lceil m/n\rceil +1\ge 2$.

Claim 1. If $|S_x|\le 2^{m-n}$, then for all $u_1,\dots ,u_k\in \{0,1{\}}^m$, it holds that $$\bigcup _{i=1}^k(S+u_i)\ne \{0,1{\}}^m.$$

Proof of Claim 1. Notice that for any $u\in \{0,1{\}}^m$, we have $|S+u|=|S|=2^{m-n}$. Then by a simple Union bound, we have $$\bigcup _{i=1}^k(S+u_i)\le \sum _{i=1}^k|S+u_i|=k\cdot 2^{m-n}<2^m.$$

Claim 2. If $|S|\ge (1-2^{-n})\cdot 2^m$ then there exists $u_1,\dots ,u_k$ such that $$\bigcup _{i=1}^k(S+u_i)=\{0,1{\}}^m.$$

Proof of Claim 2. We use the probabilistic method. If we can show that for uniformly and independently sampled $u_1,\dots ,u_k\in \{0,1{\}}^m$, $$\underset{u_1,\dots ,u_k}{\mathrm{Pr}}\left[\bigcup _{i=1}^k(S+u_i)=\{0,1{\}}^m\right]>0,$$ then there must exist vectors $u_1^{\ast },\dots ,u_k^{\ast }$ such that $$\bigcup _{i=1}^k(S+u_i^{\ast })=\{0,1{\}}^m.$$ For $r\in \{0,1{\}}^m$, let $B_r$ denote the “bad event” that $r\notin {\cup }_{i=1}^k(S+u_i)$. We show that $\mathrm{Pr}[\exists r\in \{0,1{\}}^m{B}_r]<1$.

Consider $B_r$ for any $r\in \{0,1{\}}^m$. We show that $\mathrm{Pr}[B_r]<2^{-m}$. Let $B_r^i$ denote the event that $r\notin S+u_i$. Equivalently stated, $r+u_i\notin S$. Notice that $B_r={\cap }_{i=1}^k{B}_r^i$.

Now, since $u_i$ is uniformly sampled, we know that $r+u_i$ is uniformly distributed in $\{0,1{\}}^m$. So we know that $\mathrm{Pr}[r+u_i\in S]\ge 1-2^{-n}$, which implies that $\mathrm{Pr}[r+u_i\notin S]\le 2^{-n}$. So $\mathrm{Pr}[B_r^i]\le 2^{-n}$. Finally, all $B_r^i$ are independent, so we have $\mathrm{Pr}[B_r]=\mathrm{Pr}[B_r^i{]}^k\le 2^{-nk}<2^{-m}$, where the last inequality follows since $nk=n(\lceil m/n\rceil +1)\ge m+n>m$. This implies again by the Union bound that $\mathrm{Pr}[\exists r{B}_r]<k\cdot 2^{-m}<1$, which implies that $$\underset{u_1,\dots ,u_k}{\mathrm{Pr}}\left[\bigcup _{i=1}^k(S+u_i)=\{0,1{\}}^m\right]>0,$$ so there exists vectors $u_1,\dots ,u_k$ such that $$\bigcup _{i=1}^k(S+u_i^{\ast })=\{0,1{\}}^m.$$

Now, given the two claims above, we can now decide $L\in {\mathbf{BPP}}_{1/2+2^{-n}}$ using a ${\Sigma }_2^p$ machine as follows. For any $x$, define the machine $N$ which operates as follows. $N(x,u_1,\dots ,u_k,r)$ outputs $1$ if and only if $r\in {\cup }_{i=1}^k(S_x+u_i)$, where $k,m=\mathrm{poly}(|x|)$. Therefore, $x\in L$ if and only if $\exists u_1,\dots ,u_k\in \{0,1{\}}^m\forall r\in \{0,1{\}}^{m}N(x,u_1,\dots ,u_k,r)=1$. Thus, $L\in {\Sigma }_2^p$. $\square$

Randomized Reductions

We can define a slightly weaker notion of reduction than the polynomial time reductions we’ve seen before. We’ll see randomized reductions now.

Definition. For languages $B,C$, we say that $B$ is randomized polynomial-time reducible to $C$, denoted by $B{\le }_{r}C$, if there exists a polynomial time probabilistic Turing machine $M$ such that for all $x$, we have $\mathrm{Pr}[C(M(x))=B(x)]\ge 2/3$.

Note that randomized reductions are not transitive! That is, if $A{\le }_{r}B$ and $B{\le }_{r}C$, it is not necessarily the case that $A{\le }_{r}C$. However, randomized reductions are still useful. One can show that if $B{\le }_{r}C$ and $C\in \mathbf{BPP}$, then $B\in \mathbf{BPP}$.

NP under Randomized Reductions?

We can define an NP-like class for NP under randomized reductions. This is the class $\mathbf{BP}\cdot \mathbf{NP}$. $$\mathbf{BP}\cdot \mathbf{NP}=\{L|L{\le }_{r}3SAT\}$$ Note that we can equivalently define $\mathbf{NP}$ as $\mathbf{NP}=\{L|L{\le }_{p}3SAT\}$.

Generally speaking, complexity theorists believe that $\mathbf{NP}\ne \mathbf{BP}\cdot \mathbf{NP}$. They also do not believe that $\overline{3SAT}\in \mathbf{BP}\cdot \mathbf{NP}$ because of the following lemma.

Lemma. If $\overline{3SAT}\in \mathbf{BP}\cdot \mathbf{NP}$, then $\mathbf{PH}={\Sigma }_3^p$.

Randomized Space-bounded Computations

We can also examine space-bounded computations through the lens of probabilistic Turing machines. The most interesting space-bounded randomized computations are those which only use logarithmic space.

Definition. The class $\mathbf{BPL}$ is the set of all languages $L$ such that there exists a strict polynomial-time PTM $M$ using $O(\log (n))$ additional space for inputs of length at most $n$ such that $\mathrm{Pr}[M(x)=L(x)]\ge 2/3$.

BPL is the log-space equivalent of BPP, and we can similarly log-space equivalents of RP, coRP, and ZPP, denoted as RL, coRL, and ZPL.

Theorem.

1. $\mathbf{RL}\subseteq \mathbf{NL}\subseteq \mathbf{P}$.
2. $\mathbf{BPL}\subseteq \mathbf{P}$.

Boolean Circuits

We will now turn our attention to Boolean circuits, or just circuits. Circuits are inherently a non-uniform model of computation. That is, circuits have a fixed input length, rather than being able to operate over infinitely many input lengths. For example, Turing machines are a uniform computation model, where a single Turing machine $M$ takes infinitely many inputs $x\in \{0,1{\}}^{\ast }$.

Circuits, on the other hand, can only operate over a fixed input length. For example, a circuit $C_n$ computes some function over inputs $x\in \{0,1{\}}^n$ for a fixed $n$, and every $n$ has some different circuit.

Definition. A Boolean circuit $C$ of size $m$ with $n$-bit inputs is a directed acyclic graph on $m$ vertices with the following syntax.

• The $n$ input vertices have in-degree $0$ and unlimited out-degree.
• The remaining $m-n$ non-input nodes, which we call gates, are all labeled AND, OR, and NOT (corresponding to the Boolean functions AND, OR, NOT) and operate as follows.
  • AND and OR gates both have in-degree 2 (or fan-in 2) out-degree 1 (fan-out 1).
  • NOT gates have in-degree and out-degree 1.
  • There is a single output gate with out-degree 0 (note this gate can be an input node/gate, or any internal node).

Circuit Families

Since circuits are non-uniform, one circuit cannot decide an entire language $L\subseteq \{0,1{\}}^{\ast }$ (unless $L$ only contains strings of one fixed length). Thus, we need to define circuit families to handle variable length inputs.

Definition. Let $T:\mathbb{N}\to \mathbb{N}$ be a function. A $T$-sized circuit family is a sequence of circuits $\{C_n{\}}_{n\in \mathbb{N}}$ such that $|C_n|\le T(n)$ and $C_n$ has $n$ input gates for all $n\in \mathbb{N}$. We say that a language $L$ is in the class $\mathbf{SIZE}(T)$ if there exists an $O(T)$-sized circuit family which decides $L$; that is, for all $n\in \mathbb{N}$ and $x\in \{0,1{\}}^n$, $x\in L$ if and only if $C_n(x)=1$.

Examples.

1. The unary language $L=\{1^n:n\in \mathbb{N}\}$ lies within $\mathbf{SIZE}(n)$; that is, $L\in \mathbf{SIZE}(n)$. Moreover, any unary language $L\subseteq \{1{\}}^{\ast }$ is in $\mathbf{SIZE}(n)$.

2. For $L=\{(x,y,z):x,y,z\in \mathbb{N},z=x+y\}$, we have $L\in \mathbf{SIZE}(n^2)$.