Lecture 12

In-class notes: CS 505 Spring 2025 Lecture 12

The Polynomial Hierarchy

Last time, we defined the class ${\Sigma }_2^p$. Today, we’ll generalize this class and define the Polynomial Hierarchy.

Definition. For any $i\in {\mathbb{Z}}^{+}$, the class ${\Sigma }_i^p$ is the set of all languages $L$ such that there exists a deterministic Turing machine $M_L$ deciding $L$ and a polynomial $q$ such that for all $x\in \{0,1{\}}^{\ast }$ $$\begin{array}{rl}x\in L⟺& \exists w_1\in \{0,1{\}}^{q(|x|)}\\ & \forall w_2\in \{0,1{\}}^{q(|x|)}\\ & \exists w_3\in \{0,1{\}}^{q(|x|)}\\ & \ddots \\ & Q_i{w}_i\in \{0,1{\}}^{q(|x|)}\\ & M_L(x,w_1,w_2,w_3,\dots ,w_i)=1.\end{array}$$ Here, $Q_i=\exists$ if $i$ is odd and $Q_i=\forall$ if $i$ is even.

Given this generalization of ${\Sigma }_2^p$, we can define the Polynomial Hierarchy, which we denote as $\mathbf{PH}$.

Definition. $\mathbf{PH}={\bigcup }_{i\in {\mathbb{Z}}^{+}}{\Sigma }_i^p$.

Note that we can also define the co-classes of any ${\Sigma }_i^p$. These classes are denoted by ${\Pi }_i^p$.

Definition. For any $i\in {\mathbb{Z}}^{+}$, ${\Pi }_i^p=\mathbf{co}{\Sigma }_i^p=\{L:\overline{L}\in {\Sigma }_i^p\}$. Equivalently, $L\in {\Pi }_i^p$ if there exists a deterministic Turing machine $M_L$ deciding $L$ and a polynomial $q$ such that for all $x\in \{0,1{\}}^{\ast }$ $$\begin{array}{rl}x\in L⟺& \forall w_1\in \{0,1{\}}^{q(|x|)}\\ & \exists w_2\in \{0,1{\}}^{q(|x|)}\\ & \forall w_3\in \{0,1{\}}^{q(|x|)}\\ & \ddots \\ & Q_i{w}_i\in \{0,1{\}}^{q(|x|)}\\ & M_L(x,w_1,w_2,w_3,\dots ,w_i)=1.\end{array}$$ Here, $Q_i=\exists$ if $i$ is even and $Q_i=\forall$ if $i$ is odd.

Actually, we also have that $\mathbf{PH}={\bigcup }_{i\in {\mathbb{Z}}^{+}}{\Pi }_i^p$. This is due to the following lemma.

Lemma. For all $i\ge 1$, we have ${\Sigma }_i^p\subseteq {\Pi }_{i+1}^p\subseteq {\Sigma }_{i+2}^p$.

Proof. Let $L\in {\Sigma }_i^p$. By definition, there exists DTM $M_L$ and polynomial $q$ such that for all $x$, we have $x\in L$ if and only if $$\exists w_1\forall w_2\dots Q_i{w}_i{M}_L(x,w_1,\dots ,w_i)=1,$$ where each $w_j\in \{0,1{\}}^{q(|x|)}$. To see that $L\in {\Pi }_{i+1}^p$, we can add a “dummy” quantifier $\forall w_0$ in front of $\exists w_1$. We can define a machine $M_L^{\prime }$ which takes as input $(x,w_0,w_1,\dots ,w_i)$, ignores $w_0$, and outputs $M_L(x,w_1,\dots ,w_i)$. This is clearly in ${\Pi }_{i+1}^p$. Note the same strategy works for a language $L\in {\Pi }_{i+1}^p$ and lifting it to ${\Sigma }_{i+2}^p$. $\square$

Properties of $\mathbf{PH}$

It is widely believed that ${\Sigma }_i^p⊊{\Sigma }_{i+1}^p$, ${\Pi }_i^p⊊{\Pi }_{i+1}^p$, and ${\Pi }_i^p⊊{\Sigma }_{i+1}^p$ for all $i\ge 1$. This is because otherwise, the Polynomial Hierarchy collapses. That is, there exists $i$ such that ${\Sigma }_i^p={\Sigma }_{i+1}^p$ or ${\Pi }_i^p={\Pi }_{i+1}^p$ (which implies that ${\Sigma }_i^p={\Pi }_i^p$). We say this is a collapse of $\mathbf{PH}$ because if this happens, the $\mathbf{PH}={\Sigma }_i^p={\Pi }_i^p$.

Theorem.

1. If there exists $i$ such that ${\Sigma }_i^p={\Pi }_i^p$, then $\mathbf{PH}={\Sigma }_i^p={\Pi }_i^p$.
2. If $\mathbf{P}=\mathbf{NP}$, then $\mathbf{PH}=\mathbf{P}$.

Proof. Notice that (2) is actually just a corollary of (1) when $i=1$. However, we’ll directly prove (2), as the ideas in this proof readily extend to any $i$. We’ll do a proof by induction. Suppose that $\mathbf{P}=\mathbf{NP}$. Recall this also implies that $\mathbf{NP}=\mathbf{coNP}$. Our proof by induction will establish that ${\Sigma }_i^p,{\Pi }_i^p\subseteq \mathbf{P}$ for all $i$.

For the base case, we have that ${\Sigma }_1^p={\Pi }_1^p=\mathbf{P}$ since $\mathbf{P}=\mathbf{NP}=\mathbf{coNP}$. For the inductive step, assume that ${\Sigma }_i^p,{\Pi }_i^p\subseteq \mathbf{P}$. We show that ${\Sigma }_{i+1}^p,{\Pi }_{i+1}^p\subseteq \mathbf{P}$.

First, we show the case for $L\in {\Sigma }_{i+1}^p$. Given $L\in {\Sigma }_{i+1}^p$, by definition we have a DTM $M_L$ which decides $L$. That is, for polynomial $q$ and any $x$, $x\in L$ if and only if $$\exists w_1\forall w_2\dots Q_{i+1}{w}_{i+1}{M}_L(x,w_1,\dots ,w_{i+1})=1,$$ where all $w_j\in \{0,1{\}}^{q(|x|)}$.

Define new language $L^{\prime }$ to be all pairs $(x,w_1)$ such that $$\forall w_2\dots Q_{i+1}{w}_{i+1}{M}_L(x,w_1,\dots ,w_{i+1})=1.$$ Clearly, we have that $L^{\prime }\in {\Pi }_i^p\subseteq \mathbf{P}$ by our inductive hyptophesis. This implies there exists a DTM $M^{\prime }$ such that $(x,w_1)\in L^{\prime }$ if and only if $M^{\prime }(x,w_1)=1$, where $M^{\prime }$ runs in polynomial time. Now, we can replace the decider $M_L$ for $L$ with $M^{\prime }$ as follows. Under this $M^{\prime }$, we have that $x\in L$ if and only if $\exists w_1{M}^{\prime }(x,w_1)=1$. Now this implies that $L\in \mathbf{NP}=\mathbf{P}$. Therefore, ${\Sigma }_{i+1}^p\subseteq \mathbf{P}$.

For the other case, where $L\in {\Pi }_{i+1}^p$, we simply do the above strategy for the complement language $\overline{L}\in {\Sigma }_{i+1}^p$. $\square$

Complete Problems for $\mathbf{PH}$

We can readily define complete problems for ${\Sigma }_i^p$, ${\Pi }_i^p$, and $\mathbf{PH}$, using the same definition of polynomial-time reducibility we have for $(\mathbf{co})\mathbf{NP}$. And, in fact, for every $i$, ${\Sigma }_i^p$ and ${\Pi }_i^p$ have complete problems.

However, we do not believe that $\mathbf{PH}$ has a complete problem/language $L$. This is because if one were to exist, then $\mathbf{PH}$ again collapses to some level $i$.

Theorem. If there exists $L\in \mathbf{PH}$ that is $\mathbf{PH}$-complete, then there exists $i$ such that $\mathbf{PH}={\Sigma }_i^p={\Pi }_i^p$.

Proof. Recall that $\mathbf{PH}={\cup }_i{\Sigma }_i^p={\cup }_i{\Pi }_i^p$. So for any $L\in \mathbf{PH}$, there exists $i$ such that $L\in {\Sigma }_i^p$ or ${\Pi }_i^p$. Assume that $L$ is $\mathbf{PH}$-complete. Then, for all $L^{\prime }\in \mathbf{PH}$, we have $L^{\prime }{\le }_{p}L$. Suppose that $L\in {\Sigma }_i^p$. This implies that if $L^{\prime }\in {\Sigma }_j^p$ or $L^{\prime }\in {\Pi }_j^p$ for $j>i$, we can in polynomial time decide $L^{\prime }$ using a decider for $L$. Since $L\in {\Sigma }_i^p$, this implies that $L^{\prime }\in {\Sigma }_i^p$ as well. The same holds if $L\in {\Pi }_i^p$. Thus, $\mathbf{PH}$ collapses to level $i$. $\square$

Recall that $\mathrm{TQBF}$ is the complete problem for $\mathbf{PSPACE}$. Recalling the definition of $\mathrm{TQBF}$, it is a more general quantified Boolean formula and caputres computations in $\mathbf{PH}$. So we have $\mathbf{PH}\subseteq \mathbf{PSPACE}$. However, as a corollary of the above theorem, unless $\mathbf{PH}$ collapses, then $\mathbf{PH}$ is a strict subset of $\mathbf{PSPACE}$.

Corollary. Unless $\mathbf{PH}$ collapses, $\mathbf{PH}\ne \mathbf{PSPACE}$.

This is simply because $\mathrm{TQBF}$ is a complete problem for $\mathbf{PSPACE}$, so if $\mathbf{PH}=\mathbf{PSPACE}$, then $\mathbf{PH}$ has the complete problem $\mathrm{TQBF}$, so it must collapse to some level $i$.

Complete Problems for an ${\Sigma }_i^p$ and ${\Pi }_i^p$

In contrast to $\mathbf{PH}$, for any fixed $i$, the classes ${\Sigma }_i^p$ and ${\Pi }_i^p$ each have complete problems.

1. For ${\Sigma }_i^p$, we define the language ${\Sigma }_i\mathrm{SAT}$ as follows. $${\Sigma }_i\mathrm{SAT}=\{\varphi :\exists u_1\forall u_2\cdots Q_i{u}_i\varphi (u_1,\dots ,u_i)=1\},$$ where $Q_i=\exists$ if $i$ is odd and $Q_i=\forall$ if $i$ is even. Then, ${\Sigma }_i\mathrm{SAT}$ is ${\Sigma }_i^p$-complete.

2. For ${\Pi }_i^p$, we define the language ${\Pi }_i\mathrm{SAT}$ as follows. $${\Pi }_i\mathrm{SAT}=\{\varphi :\forall u_1\exists u_2\cdots Q_i{u}_i\varphi (u_1,\dots ,u_i)=1\},$$ where $Q_i=\exists$ if $i$ is even and $Q_i=\forall$ if $i$ is odd. Then, ${\Pi }_i\mathrm{SAT}$ is ${\Pi }_i^p$-complete.

3. A different complete problem for ${\Sigma }_2^p$, due to Umans in 1998. This is called $\mathrm{Succinct}\text{-}\mathrm{Set}\text{-}\mathrm{Cover}$. The problem is defined as follows. The input to the problem is a set $\{{\varphi }_1,\dots ,{\varphi }_m\}$ of $m$ $3$DNF formulas each with $n$ variables, and an integer $k$. This input is said to be in $\mathrm{Succinct}\text{-}\mathrm{Set}\text{-}\mathrm{Cover}$ if and only if there exists a subset $S\subset [m]$ of size at most $k$ such that the formula ${\phi }_S={\bigvee }_{i\in S}{\varphi }_i$ is a tautology. More formally, $$\exists S\subset [m]\forall x\in \{0,1{\}}^n\left(|S|\le k\wedge \left(\underset{i\in S}{\bigvee }{\varphi }_i\right)(x)=1\right).$$ Clearly, this language is in ${\Sigma }_2^p$. It turns out it is also ${\Sigma }_2^p$-hard.

Alternating Turing Machines: Generalized Non-Determinism

Let’s think back to the non-deterministic Turing machine definition of $\mathbf{NP}$. We say that $L\in \mathbf{NP}$ if there exists a NTM $M$ which decides $L$. This was defined as follows: for all $x$, $x\in L$ if and only if $M(x)=1$, where we say $M(x)$ outputs 1 if and only if there is at least one set of non-deterministic choices $M$ makes on input $x$ that causes $M(x)$ to output $1$. Viewing this as a computation tree, or from the point of view of our configuration graph $G_{M,x}$, it says that there is at least one path from $c_{start}$ to any accepting configuration $c_{accept}$ in $G_{M,x}$. Moreover, all paths to halting configurations have length $\mathrm{poly}(|x|)$.

Now, thinking back to $\mathbf{coNP}$, it is reversed: an NTM $M$ decides $L\in \mathbf{coNP}$ if for any $x$, $x\in L$ if and only if in the graph $G_{M,x}$, all halting configurations are accepting. If this happens, we say that $M(x)=1$; otherwise, if there is at least one rejecting configuration, $M(x)=0$.

How can we model this for, say, ${\Sigma }_2^p$? That is, is there a way to describe a non-deterministic machine which decides $L\in {\Sigma }_2^p$? Looking at the deterministic verifier definition, $L\in {\Sigma }_2^p$ if there exists deterministic verifier $D$ such that for all $x$, $$x\in L⟺\exists w_1\forall w_{2}D(x,w_1,w_2)=1.$$

It is not immediately clear how we would define a non-deterministic Turing machine to decide $L\in {\Sigma }_2^p$. For languages in $\mathbf{NP}$, it was enough that there existed at least one accepting configuration in $G_{M,x}$, and for $\mathbf{coNP}$, we needed all halting configurations to be accepting in $G_{M,x}$. Thinking to the verification definitions of $(\mathbf{co})\mathbf{NP}$, this makes sense. But how can we generalize it to ${\Sigma }_2^p$, and more generally ${\Sigma }_i^p$ and ${\Pi }_i^p$?

Alternating Turing Machines

The answer is alternating Turing machines. These machines, in some sense, generalize non-determinism as follows.

Definition. A non-determinstic Turing machine $M$ is said to be an alternating Turing machine (ATM) if it additionally labels non-halting states with $\exists$ or $\forall$ such that for any $x$, $M(x)=1$ if and only if the starting configuration is labeled $1$ after the following process.

1. Let $G_{M,x}$ be the directed configuration graph for $M(x)$.
2. Label all configurations $c_{accept}$ with $1$ and $c_{reject}$ with $0$.
3. For all non-halting nodes $v$ in the graph $G_{M,x}$, corresponding to some non-halting configuration:
   • If the state of $v$ is labeled $\exists$, label $v$ with a $1$ if and only if at least one child of $v$ is labeled $1$.
   • If the state of $v$ is labeled $\forall$, label $v$ with a $1$ if and only if all children of $v$ are labeled $1$. We say that an ATM $M$ runs in time $T$ if all root-to-leaf paths in $G_{M,x}$ have length at most $O(T(|x|))$ for any input $x$. Finally, we say that an ATM $M$ is $i$-alternating if on any root-to-leaf path in $G_{M,x}$, it alternates between $\exists$ and $\forall$ state label quantifiers at most $i-1$ times.

Under the above definition, it is clear that $L\in {\Sigma }_2^p$ is decidable in polynomial time on an ATM which is $2$-alternating. That is, the ATM begins only with $\exists$ states, then at some point switches to $\forall$ states, until the computation halts. We can now define alternating time and space.

Definition. For function $T$, we say that $L\in \mathbf{ATIME}(T)$ if there exists an ATM which decides $L$ in at most $O(T)$ time. Similarly, for function $S$, we say that $L\in \mathbf{ASPACE}(S)$ if there exists an ATM which decides $L$ in at most $O(S)$ space.