Lecture 17

In-class notes: CS 505 Spring 2025 Lecture 17

Continuing our discussion on circuits, recall that $\mathbf{SIZE}(T)$ is the set of all languages $L$ that are decidable by an $O(T)$-sized circuit family $\{C_n{\}}_{n\in \mathbb{N}}$, where for all $x$, we have $x\in L$ if and only if $C_{|x|}(x)=1$.

We highlight two facts related to circuits.

1. Any (3)CNF Boolean formula is a special-case of circuits.
2. Recall at the beginning of class this semester, we stated that every Boolean function $f:\{0,1{\}}^n\to \{0,1\}$ has an $O(n\cdot 2^n)$ sized CNF formula computing it. This readily implies that any such function $f$ is computable by an $O(n\cdot 2^n)$-sized circuit.
   • Shannon improved this bound to $O(2^n/n)$ for any such $f$.
   • Others improved over Shannon, giving an upper bound of $(2^n/n)(1+o(n))$.

Circuit Complexity: P/poly

We’ll not discuss what complexity theorists feel is one of the most important circuit complexity classes: ${\mathbf{P}}_{/poly}$. Intuitively, this is the circuit equivalent of $\mathbf{P}$.

Definition. The class ${\mathbf{P}}_{/poly}$ is the set of all polynomial-sized circuits. That is, ${\mathbf{P}}_{/poly}={\cup }_{c\in \mathbb{N}}\mathbf{SIZE}(n^c)$.

Theorem. $\mathbf{P}\subseteq {\mathbf{P}}_{/poly}$.

Proof. First, recall that any time-$T$ Turing machine $M$ has an equivalent oblivious Turing machine $N$ running in time $O(T\log (T))$ (in class, we saw a simpler proof of $O(T^2)$). The properties of this machine $N$ are as follows.

1. For all $x$, $M(x)=N(x)$.
2. At any timestep $i$, the position of the tape heads of $N$ are a function of $|x|$ and the current timestep $i$ (that is, they only depend on these two quantities).

Crucially, (2) above tells us for any fixed input length $n$, the machine $N(x)$ for any $x\in \{0,1{\}}^n$ moves its heads the exact same way. Now, to show the theorem, we will argue that every time-$T$ oblivious TM has an $O(T)$-sized circuit family deciding it. The remainder of the proof will be a high-level overview.

Let $t=T(|x|)$ denote the runtime of $N(x)$. For timestep $i\in [t]$, we define $z_i$ to be a snapshot of the execution of $N$ on input $x$. This snapshot $z_i$ contains

• The current state of $N$ at timestep $i$
• All $k$ symbols under the $k$ tape heads of $N$.

Since for every fixed Turing machine, the number of states and tapes is constant, we have that $|z_i|=O(1)$ for every $i$. Now, we define a transcript of $N$ as $z_1,z_2,\dots ,z_t$, where $z_1$ is the initial snapshot of $N$ in its initial configuration, and snapshot $z_i$ follows from $z_{i-1}$ via the transition function. That is, we can write $\delta (z_{i-1})=z_i$ (to abuse notation).

The key observation here is that since each $z_i$ is constant-size, and moving from $z_i$ to $z_{i+1}$ for every $i$ only depends on $i$ and $|x|$, we can compute each $z_i$ using a constant-sized circuit. This constant-sized circuit, which we denote by $c_i$, takes as input $z_{i-1}$ and $i$ (and assumes that $n=|x|$) and outputs $z_i$. Then, to construct the final circuit $C_n$, we simply compose all of these sub-circuits together to compute $z_1,z_2,\dots ,z_t$, and a final sub-circuit which reads $z_t$ and outputs $1$ if and only if $z_t$ contains the accept state.
This circuit has size $O(t)=O(T(|x|))$, and we can define such a circuit $C_n$ for every input length $n$. Finally, the circuit has worst-case size $O(T^2)$ if we started with a non-oblivious Turing machine. $\square$

Note. In the above proof, the transformation from the non-oblivious Turing machine to a circuit can be performed in polynomial-time and logarithmic space. We will use this fact later.

P is a strict subset of P/poly

The non-uniform power of ${\mathbf{P}}_{/poly}$ can be showcased in a number of ways. Here, we’ll first show that the inclusion of the above theorem is strict. That is, $\mathbf{P}⊊{\mathbf{P}}_{/poly}$. This follows from last lecture when we stated that any unary language is in ${\mathbf{P}}_{/poly}$.

Lemma. For any $L\subseteq \{1^n:n\in \mathbb{N}\}$, we have $L\in {\mathbf{P}}_{/poly}$.

Proof. For every $n$, we have two cases. First, if $1^n\notin L$, we define the circuit $C_n$ to be the constant-sized circuit encoding the Boolean formula $(x_1\wedge {\overline{x}}_1)$, ignoring all other input bits.¹ This formula always outputs $0$, so it will always reject since $1^n\notin L$ (and no other strings of length $n$ are in $L$ too).

Now, if $1^n\in L$, we define $C_n$ to be the $O(n)$-size circuit encoding $(x_1\wedge x_2\wedge \cdots \wedge x_n)$. This formula outputs $1$ if and only if $x=1^n$. $\square$

Why does this show $\mathbf{P}⊊{\mathbf{P}}_{/poly}$? It is because we can encode Turing undecidable problems into unary languages. Consider the following unary halting problem language.

$$UHALT=\{1^n:⟨n{⟩}_2=(M,x)\text{ and }M(x)\text{ halts.}\}$$

Clearly, $UHALT\subset \{1{\}}^{\ast }$, so $UHALT\in {\mathbf{P}}_{/poly}$, but $UHALT$ is not Turing decidable!

Alternate Proof of Cook-Levin Theorem

We can use circuits and obtain an alternate proof of the Cook-Levin theorem. To do this, we’ll need the language of circuit satisfiablility. We let $\mathrm{CKT}\text{-}\mathrm{SAT}$ denote the set of all strings $C\in \{0,1{\}}^{\ast }$ such that $C$ encodes a circuit with a $1$-bit output and there exists $w\in \{0,1{\}}^{\ast }$ such that $C(w)=1$.

Theorem. $\mathrm{CKT}\text{-}\mathrm{SAT}$ is NP-complete.

Proof. It is clear that $\mathrm{CKT}\text{-}\mathrm{SAT}\in \mathbf{NP}$ since given the encoding $C$ and a string $w$, one can check in $O(|C|)$ time if $C(w)=1$. Now, we show that $\mathrm{CKT}\text{-}\mathrm{SAT}$ is NP-hard. To see this, we must show that for all $L\in \mathbf{NP}$, we have $L{\le }_p\mathrm{CKT}\text{-}\mathrm{SAT}$. Fortunately, in our proof that $P\subseteq {\mathbf{P}}_{/poly}$, the transformation from a Turing machine $M$ running in time $T$ to a $O(T^2)$-sized circuit family is (1) a polynomial-time transformation; and (2) works for non-deterministic Turing machines as well. Or, if we strictly use deterministic Turing machines, this proof shows how to transform any polynomial-time verifier $M_L$ (i.e., $x\in L$ iff $\exists u$ such that $M_L(x,u)=1$) into a polynomial-sized circuit family $\{C_n{\}}_{n\in \mathbb{N}}$ such that $x\in L$ if and only if $\exists w$ such that $C_{|x|}(x,w)=1$. $\square$

With the above theorem, to show an alternate proof of the Cook-Levin theorem, we must now show that $\mathrm{CKT}\text{-}\mathrm{SAT}{\le }_{p}3SAT$. This is done by constructing a 3CNF formula $\varphi$ as follows. Consider any circuit $C$ and any node $v_i$ in $C$ with parent nodes $v_j$ and $v_k$.

1. If $v_i$ is the AND of $v_j$ and $v_k$, we encode the Boolean formula “$(z_j\wedge z_k)=z_i$” in the formula $\varphi$, using the fact that the Boolean operator “$=$” can be written as $x=y\mapsto (x\vee \overline{y})\wedge (\overline{x}\vee y)$. To turn this into a 3CNF, we can encode “$(z_j\wedge z_k)=z_i$” as 4 clauses in 3CNF form. The final expression will be $$({\overline{z}}_i\vee {\overline{z}}_j\vee z_k)\wedge ({\overline{z}}_i\vee z_j\vee {\overline{z}}_k)\wedge ({\overline{z}}_i\vee z_j\vee z_k)\wedge (z_i\vee {\overline{z}}_j\vee {\overline{z}}_k).$$
2. If $v_i$ is the OR of $v_j$ and $v_k$, we again do the same thing as above. This will result in 3 clauses in 3CNF form; namely, $$(z_i\vee {\overline{z}}_j)\wedge (z_i\vee {\overline{z}}_k)\wedge ({\overline{z}}_i\vee z_j\vee z_k),$$ where we can turn the two 2CNF clauses into 3CNF by simply repeating a variable already in the clause.
3. If $v_i$ is the NOT of $v_j$, then we simply encode ${\overline{z}}_j=z_i$ into a 3CNF, resulting in 2 clauses in 3CNF form.

Crucially, for every triple of nodes $v_i,v_j,v_k$ (or just $v_i,v_j$ in the case of NOT), there is a constant-sized 3CNF formula on 3 (or 2) variables to compute it. This means that $|phi|=O(|C|)$, so the transformation is polynomial-time in $|C|$. Finally, clearly we have that $C$ is satisfiable if and only if $\varphi$ is satisfiable. Thus, we have given an alternate proof of the Cook-Levin Theorem.

Corollary. If $A\in \mathbf{DTIME}(T)$, then $A\in \mathbf{SIZE}(T^2)$.²

Uniformly Generated Circuits

So far we have seen that circuits are quite powerful and can decide Turing undecidable languages. This is because of the non-uniform computation model of circuits. In particular, to show that $L\in {\mathbf{P}}_{/poly}$, it is enough that there exists a circuit family $\{C_n{\}}_n$ which decides $L$. This does not ever consider if this family is constructible at all! So this begs the natural question: what if we restrict our attention to circuit families which are only efficiently constructible?

Definition. A circuit family $\{C_n{\}}_{n\in \mathbb{N}}$ is said to be P-uniform if there exists a polynomial-time Turing machine $M$ such that for all $n\in \mathbb{N}$, $M(1^n)=C_n$. That is, $M(1^n)$ outputs the description of the circuit $C_n$ in $O(n^c)$ time for some constant $c$.

Unfortunately, restricting our attention to such circuits only gives us P.

Theorem. A language $L$ is decidable by a P-uniform circuit family if and only if $L\in \mathbf{P}$.

Proof. For all $x$, let $n=|x|$. Suppose that $L$ is decidable by a P-uniform circuit family $\{C_n\}$. This means that $x\in L$ if and only if $C_n(x)=1$, where $M(1^n)=C_n$ in $\mathrm{poly}(n)$ time. Define a new Turing machine $N$ that on input $x$ simply runs $M(1^{|x|})$ to obtain $C_{|x|}$, then evaluates $C_{|x|}(|x|)$ and outputs the circuit outputs. Clearly, this machine $N$ runs in polynomial time since $|C_n|=\mathrm{poly}(n)$ and $M$ runs in $\mathrm{poly}(n)$ time.

The other direction follows directly from the above corollary: $L\in \mathbf{DTIME}(T)$ implies $L\in \mathbf{SIZE}(T^2)$. In particular, for any $L\in \mathbf{P}$ with Turing machine $M$, we can construct a new DTM $N$ which on input $x$ outputs the circuit for the oblivious Turing machine which decides $L$.

Logspace Uniform Circuits

We can further restrict P-uniform circuits to require that they be implicitly logspace computable. That is, $M(1^n)=C_n$ using $O(\log (n))$ space on the non-output tape. More formally, $(⟨M(1^n){⟩}_2{)}_i=(C_n{)}_i$ in $O(\log (n))$ space (i.e., we can compute the $i$-th bit of the representation of $C_n$ in logarithmic space).

Theorem. $L$ is decidable by a logspace uniform circuit family if and only if $L\in \mathbf{P}$.

This again follow from our proof that $\mathbf{P}\subseteq {\mathbf{P}}_{/poly}$ since the transformation in that proof is implicitly logspace computable, and the fact that logspace computable circuits are already P-uniform circuits.

P/poly vs Other Classes

Circuits are, in principle, one way to overcome the barriers of diagonalization and relativization which do not allow us to separate P and NP. We’ll now examine how ${\mathbf{P}}_{/poly}$ relates to other complexity classes. For now, we’ll only state the results and prove (some of) them in the next lecture.

P/poly vs. NP

Theorem. If $\mathbf{NP}\subseteq {\mathbf{P}}_{/poly}$, then $\mathbf{PH}={\Sigma }_2^p$.

P/poly vs. EXP

Theorem. If $\mathbf{EXP}\subseteq {\mathbf{P}}_{/poly}$, then $\mathbf{EXP}={\Sigma }_2^p$.

P/poly vs. BPP

Theorem. $\mathbf{BPP}\subseteq {\mathbf{P}}_{/poly}$.

1. Technically speaking, we should be defining the circuit as $(x_1\wedge {\overline{x}}_1\wedge x_2\cdots \wedge x_n)$ since $C_n$ takes $n$ inputs. Clearly, this circuit has size $O(n)$. ↩

2. More precisely, $\mathbf{SIZE}(T\log (T))$. ↩