Lecture 18

In-class notes: CS 505 Spring 2025 Lecture 18

Proofs from Last Lecture

Today, we’ll begin by proving two theorems from last lecture.

NP vs. P/poly

Theorem. If $\mathbf{NP}\subseteq {\mathbf{P}}_{/poly}$, then $\mathbf{PH}={\Sigma }_2^p$.

Proof. We show that if $\mathbf{NP}\subseteq {\mathbf{P}}_{/poly}$, then ${\Pi }_2^p\subseteq {\Sigma }_2^p$. It suffices to show that the ${\Pi }_2^p$-complete problem ${\Pi }_2\mathrm{SAT}$ is in the class ${\Sigma }_2^p$. Recall the definition of ${\Pi }_2\mathrm{SAT}$. $${\Pi }_2\mathrm{SAT}=\{\phi :\forall u\in \{0,1{\}}^n\exists v\in \{0,1{\}}^n\phi (u,v)=1\}$$ For convenience, we will assume Boolean formulas have $2n$ variables.

Under the assumption that $\mathbf{NP}\subseteq {\mathbf{P}}_{/poly}$, we know that $\mathrm{SAT}\in {\mathbf{P}}_{/poly}$. Now, this tells us that there exists a circuit family $\{C_n{\}}_{n\in \mathbb{N}}$ such that for any Boolean formula $\varphi$, if $|\varphi |=m$ then $\varphi \in \mathrm{SAT}$ if and only if $C_m(\varphi )=1$; moreover, $|C_m|=\mathrm{poly}(m)$.

Let $\phi$ be a formula on $2n$ variables. Then, for any $u\in \{0,1{\}}^n$, we have that $\varphi =\phi (u,\cdot )$ is a Boolean formula on $n$ variables. Here, we can see that $\varphi \in \mathrm{SAT}$ if and only if there exists $v\in \{0,1{\}}^n$ such that $\varphi (v)=1$, if and only if $\phi (u,v)=1$.

By our assumption, $\varphi \in \mathrm{SAT}$ if and only if $C_m(\varphi )=1$ for $|\varphi |=m$, if and only if $C_m(\phi ,u)=1$ for any $u\in \{0,1{\}}^n$. We will use this fact to construct a new circuit family $\{C_n^{\prime }{\}}_{n\in \mathbb{N}}$ to take as input $\phi$ and $u$ and output $v$ such that $\phi (u,v)=1$. In particular, for $m=|\phi |$ and $u\in \{0,1{\}}^n$, $C_m^{\prime }(\phi ,u)=v$ if and only if there exists $v$ such that $\phi (u,v)=1$. The circuit $C_m^{\prime }$ will use multiple copies of the circuit $C_m$ as a sub-circuit. Intuitively, $C_m^{\prime }(\phi ,u)$ will construct the formula $\varphi =\phi (u,\cdot )$. Let $\varphi$ have variables $x_1,\dots ,x_n$. Now, $C_m^{\prime }$ will iteratively set $x_i=0$ and $x_i=1$, use $C_m$ to test if $\varphi (x_i=0)$ or $\varphi (x_i=1)$ and iteratively build the satisfying assignment $v$ for $\varphi$. Clearly, since $|C_m|$ is polynomial, then $|C_m^{\prime }|$ is also $\mathrm{poly}(m)$.

Thus, for any $\phi$, we have that $\phi \in {\Pi }_2\mathrm{SAT}$ if and only if $\forall u\in \{0,1{\}}^n\exists v\in \{0,1{\}}^n$ such that $\phi (u,v)=1$, which we have shown happens if and only if $\forall u\in \{0,1{\}}^n\exists v\in \{0,1{\}}^n$ such that $C_m^{\prime }(\phi ,u)=v$ for $m=|\phi |$. Now, the definition of ${\mathbf{P}}_{/poly}$ only tells us that the family $\{C_n^{\prime }{\}}_{n\in \mathbb{N}}$, but not how to construct it! Here, we’ll rely on the fact that since $|C_n^{\prime }|=q(n)$ for some polynomial $n$, then we only need $O(q^2(n))$ bits to write the description of $C_n^{\prime }$.¹

Therefore, we can now flip the quantifiers in the above formula as follows. We have that $\phi \in {\Pi }_2\mathrm{SAT}$ if and only if $$\exists w\in \{0,1{\}}^{q^2(m)}\forall u\in \{0,1{\}}^n(w=C_m^{\prime }\wedge \phi (u,C_m^{\prime }(\phi ,u))=1).$$ Here, $w$ is the bit-string which encodes the circuit $C_m^{\prime }$, and $m=|\phi |$. Clearly, the above formula encodes ${\Sigma }_2\mathrm{SAT}$. Therefore, we have shown that $\phi \in {\Pi }_2\mathrm{SAT}$ if and only if $\phi \in {\Sigma }_2\mathrm{SAT}$, so ${\Pi }_2^p\subseteq {\Sigma }_2^p$, and thus the polynomial hierarchy collapses. $\square$

BPP vs. P/poly

Theorem. $\mathbf{BPP}\subseteq {\mathbf{P}}_{/poly}$.

Proof. Let $L\in \mathbf{BPP}$. Then, there exists a polynomial-time deterministic Turing machine $M$ such that for all $n\in \mathbb{N}$ and $x\in \{0,1{\}}^n$, we have ${\mathrm{Pr}}_r[M(x,r)=L(x)\ge 1-1/2^{n+1}$. Here, $r\in \{0,1{\}}^m$ for some $m=\mathrm{poly}(n)$. Equivalently, ${\mathrm{Pr}}_r[M(x,r)\ne L(x)]\le 1/2^{n+1}$.

For any $x\in \{0,1{\}}^n$, we say that the string $r\in \{0,1{\}}^m$ is bad for $x$ if $M(x,r)\ne L(x)$. Otherwise, we say that $r$ is good for $x$. For any $x$, define $S_x=\{r:r\text{ is bad for }x\}\subseteq \{0,1{\}}^m$.

By definition, for any $x$, we have $|S_x|\le 2^m/2^{n+1}$ since ${\mathrm{Pr}}_r[M(x,r)\ne L(x)]\le 1/2^{n+1}$. Now, if we take the union of all $S_x$, we can upper bound its size via the Union Bound: $$\begin{array}{r}\left|\bigcup _{x\in \{0,1{\}}^n}{S}_x\right|\le \sum _{x\in \{0,1{\}}^n}|S_x|\le 2^n\cdot \frac{2^m}{2^{n+1}}=\frac{2^m}{2}.\end{array}$$ Note that the set $T={\cup }_x{S}_x$ is the set of all $r$ such that $r$ is bad for every $x$. So we have shown that $|T|\le 2^m/2<2^m$. In particular, this implies that there must exist at least one string $r^{\ast }$ that is good for every $x$. That is, $\exists r^{\ast }\in \{0,1{\}}^m\forall x\in \{0,1{\}}^{n}M(x,r^{\ast })=L(x)$.

Now, since $M$ runs in polynomial-time, say $p(n)$ for inputs of length $n$, we can implement $M$ via a circuit of size at most $O(p(n{)}^2+m)$ size for each input of length $n$. Let $C_n$ be this circuit. In particular, $C_n$ will have the string $r^{\ast }$ hard-coded, and will compute $M(x,r^{\ast })$. We complete this process for every $n$, hard-coding each appropriate string $r^{\ast }$. This implies that $L\in {\mathbf{P}}_{/poly}$. $\square$

Circuit Lower Bounds

In principle, circuit lower bounds allow us to circumvent the diagonalization barriers to showing P vs NP. In particular, if $\mathbf{NP}⊈{\mathbf{P}}_{/poly}$, then $\mathbf{P}\ne \mathbf{NP}$ since $\mathbf{P}\subseteq {\mathbf{P}}_{/poly}$. Thus, if we could show that a single language $L\in \mathbf{NP}$ must have a super-polynomial sized circuit family deciding it, we have shown the result. However, the best lower bound known for any $L\in \mathbf{NP}$ states that $L$ is decidable by a circuit family of size at least $(5-o(1))n$. This lower bound is quite far from the unconditional lower bound due to Shannon.

Theorem. For all $n\in {\mathbb{Z}}^{+}$, there exists $f:\{0,1{\}}^n\to \{0,1\}$ such that $f$ is not computable by any circuit $C$ of size $|C|\le 2^n/(10n)$.

Proof. First, the number of function $f:\{0,1{\}}^n\to \{0,1\}$ is $2^{2^n}$. Next, for any circuit $C$, if $|C|=S$, then we only need at most $9\cdot S\log (S)$ bits to represent $C$ as a binary string. This implies the number of circuits of size $S$ is at most $2^{9S\log (S)}<2^{S^2}$. Setting $S=2^n/(10n)$, this implies that the number of circuits of size $S$ is at most $2^{(2^n/(10n){)}^2}<2^{2^n}$ since $(2^n/(10n){)}^2<2^n$ for large enough $n.$ $\square$

Note that the above bound on the circuit size was improved in at least two subsequent works.

• $(1-\epsilon )2^n/n$ for all $\epsilon >0$.
• $2^n(1+\log (n)/n-O(1/n))$.

Non-Uniform Time Hierarchy

Just like with Turing machines, circuits have their own “time hierarchy” theorem.

Theorem. Let $T,T^{\prime }:\mathbb{N}\to \mathbb{N}$ be two functions such that $n<10\cdot T(n)<T^{\prime }(n)<2^n/n$. Then, $\mathbf{SIZE}(T)⊊\mathbf{SIZE}(T^{\prime }).$

Proof. Let $f:\{0,1{\}}^{\ell }\to \{0,1\}$ be a function that is not computable by any circuit $C$ of size at most $|C|\le 2^{\ell }/(10\ell )$. Now, recall that any function $g:\{0,1{\}}^n\to \{0,1\}$ can be implemented by a circuit $C^{\prime }$ of size $n\cdot 2^n$.

Set $\ell =(1.1)\log (n)$ and define the function $g:\{0,1{\}}^n\to \{0,1\}$ as follows: $g(x)=f(x_1,\dots ,x_{\ell })$. That is, $g$ simply throws away the last $n-\ell$ bits of its input $x$ and just computes $f(x_1,\dots ,x_{\ell })$.

We know that $g\in \mathbf{SIZE}(2^{\ell }\cdot 10\ell )\subseteq \mathbf{SIZE}(n^2)$, where the last subset inclusion is due to the fact that $\ell =O(\log (n))$. However, we know by definition that $g\notin \mathbf{SIZE}(2^{\ell }/(10\ell )\subseteq \mathbf{SIZE}(n)$, where the last subset inclusion is again due to $\ell =O(\log (n))$.

Parallel Complexity

Complexity theory also attempts to define what it means for a computation to have an efficient parallel program implementation. They do this via circuits and the class $\mathbf{NC}$.

Definition. For all $i\in \mathbb{N}$, the class ${\mathbf{NC}}^i$ is the set of all languages $L$ decidable by a circuit family $\{C_n{\}}_{n\in \mathbb{N}}$ such that

1. $|C_n|\le \mathrm{poly}(n)$, and
2. $\mathrm{depth}(C_n)\le O({\log }^i(n))$, where $\mathrm{depth}(C_n)$ is the length of the longest path from an input node to (any) output node. Moreover, Nick’s Class is defined as $\mathbf{NC}={\cup }_{i\in \mathbb{N}}{\mathbf{NC}}^i$.

Note that the class ${\mathbf{NC}}^0$ is a special class since it is all constant-depth circuits. By our definition of circuits, we only have circuits with bounded fan-in $2$. Therefore, all circuits in ${\mathbf{NC}}^0$ have outputs which can only depend on a constant number of the input bits. In particular, this implies that ${\mathbf{NC}}^0$ is a (logspace) uniform circuit class! Otherwise, we can define uniform NC as NC restricted to logspace uniform circuit families.

We can also remove the restriction of bounded fan-in and obtain the class AC.

Definition. For all $i\in \mathbb{N}$, the class ${\mathbf{AC}}^i$ is identical to the class ${\mathbf{NC}}^i$, except nodes in the circuits have unbounded (polynomial) fan-in. Then, $\mathbf{AC}={\cup }_i{\mathbf{AC}}^i$.

Theorem. For all $i$,

• ${\mathbf{NC}}^i\subseteq {\mathbf{AC}}^i\subseteq {\mathbf{NC}}^{i+1}$.
• ${\mathbf{NC}}^0⊊{\mathbf{AC}}^0$.

Note that it is unknown if the first statement in the above theorem is strict.

Theorem. A language $L$ has an efficient parallel algorithm² if and only if $L\in \mathbf{NC}$.

Parallel Complexity Major Open Questions

The major question with parallel complexity is $\mathbf{P}=\mathbf{NC}$. Complexity theorists generally believe that $\mathbf{P}\ne \mathbf{NC}$, but are currently unable to even separate ${\mathbf{NC}}^1$ form $\mathbf{PH}$. The study of this question motivates P-completeness. A language $L$ is P-complete if $L\in \mathbf{P}$ and $L^{\prime }{\le }_{L}L$ for all $L^{\prime }\in \mathbf{P}$ (i.e., closed under logspace reductions).

Theorem. If $L$ is a P-complete language, then

1. $L\in \mathbf{NC}$ if and only if $P=\mathbf{NC}$.
2. $L\in \mathbf{L}$ if and only if $\mathbf{P}=\mathbf{L}$ (recall that $\mathbf{L}=\mathbf{DSPACE}(\log (n))$).

The following is a natural P-complete language which could possibly resolve these open questions. $$\mathrm{CIR}\text{-}\mathrm{EVAL}=\{(C,x):C(x)=1\text{ and }C\text{ is a Boolean circuit.}\}$$

Polynomial Hierarchy via Exponential-size Circuits

We complete our study of Boolean circuits by giving yet another characterization of the polynomial hierarchy.

Definition. A circuit family $\{C_n{\}}_n$ is DC-uniform if there exists a polynomial-time Turing machine $M$ such that $M(n,i)$ outputs the $i$-th bit of the binary description of $C_n$.

Note that a DC-uniform circuit family can have exponential-size circuits $C_n$.

Theorem. A language $L\in \mathbf{PH}$ if and only if $L$ is decidable by a DC-uniform circuit family $\{C_n{\}}_n$ with the following properties for all $n$.

1. $C_n$ only has AND, OR, and NOT gates.
2. $|C_n|\le 2^{n^{O(1)}}$ and $\mathrm{depth}(C_n)=O(1)$.
3. $C_n$ has unbounded (exponential) fan-in.
4. Every NOT gate in $C_n$ is only at the input level.

Note that if we allow the circuits in the above theorem to have larger than constant depth, then we have characterized $\mathbf{EXP}$.

1. To see this, consider the adjacency matrix view of the circuit $C_n^{\prime }$. ↩

2. Aurora and Barak do not define what this term means. ↩