Lecture 11

In-class notes: CS 505 Spring 2025 Lecture 11

Certificate Definition of $\mathbf{NL}$

Just like with $\mathbf{NP}$, we can define the class $\mathbf{NL}$ using a deterministic Turing machine that takes a certificate as additional input to help verify whether or not a string $x$ is in a language $L$. Recall that for both $\mathbf{L}$ and $\mathbf{NL}$, the space used by the input tape is not counted towards the overall space complexity, and the input tape is read-only. So it makes sense that a certificate definition of $\mathbf{NL}$ does not count the size of the certificate when restricting the space complexity.

However, we actually need a stronger property to define $\mathbf{NL}$ with respect to certificates. We need one additional special tape, which we call the certificate/witness tape, with the following properties:

1. the space used on the certificate tape is not counted against the overall space usage (as the witness could be polynomial in size); and
2. The tape is read-once.

Property (2) above turns out to be crucial, as if you allow the machine to read the certificate multiple times (i.e., it is a read-only tape like the input tape), then you actually end up back in the class $\mathbf{NP}$.

Definition. We say a language $L$ is in $\mathbf{NL}$ if there exists a deterministic Turing machine $M$ with a special read-once certificate tape and a polynomial $p$ such that for all $x\in \{0,1{\}}^{\ast }$, $x\in L$ if and only if there exists $w\in \{0,1{\}}^{p(|x|)}$ such that $M(x,w)=1$, where $x$ is the input and is on the input (i.e., read-only) tape and $w$ is the certificate/witness and is on the special certificate (i.e., read-once) tape, and $M$ uses $O(\log (|x|))$ additional space on its read/write work tapes.

$\mathbf{NL}=\mathbf{coNL}$

Recall that we do not believe that $\mathbf{NP}=\mathbf{coNP}$. This is because under the certificate definition of $\mathbf{NP}$, there needs to exist at least one certificiate such that the deterministic verifier outputs $1$ (i.e., $M(x,w)=1$ for at least one witness $w$). In contrast, for $\mathbf{coNP}$, the machine $M$ has to output $1$ for all certificates of polynomial length (i.e., $M(x,w)=1$ for all $w$). So, intuitively, $\mathbf{NP}=\mathbf{coNP}$ says that we can verify exponentially many certificates using a single certificate of polynomial length. Thus, we do not believe this to be the case.

Turning towards $\mathbf{NL}$, consider the class $\mathbf{coNL}=\{L:\overline{L}\in \mathbf{NL}\}$. Under the certificate definition, again a machine for $\mathbf{coNL}$ must output $1$ for all polynomial-sized witnesses $w$. So it was not believed to hold that $\mathbf{NL}=\mathbf{coNL}$.

However, this was shown to be true! In particular, it was shown that the $\mathbf{coNL}$-complete problem $\overline{\mathrm{PATH}}$ lies in $\mathbf{NL}$. Recall that $\mathrm{PATH}=\{(G,s,t):\exists \text{ a path from }s\text{ to }t\text{ in directed graph }G\}$ is $\mathbf{NL}$-complete. Under this definition, we have $$\overline{\mathrm{PATH}}=\{(G,s,t):\text{there does not exist a path from }s\text{ to }t\text{ in directed graph }G\}.$$

Theorem. $\overline{\mathrm{PATH}}\in \mathbf{NL}$.

Proof. The key insight into the proof is that if we can enumerate all the vertices which are reachable from $s$ in the graph $G$, and $t$ is not in this set, then we have shown there does not exist a path from $s$ to $t$. We’ll build a certificate which verifies the sizes and the contents of the set of vertices which are reachable from $s$. Our certificate will consist of many sub-certificates, which we will use all together to build the final certificate. The main challenges here are (1) the certificate must be read-once, and (2) we must verify the certificate using only logarithmic space.

Let $n=|V|$ denote the number of vertices in the graph $G$. Moreover, we will uniquely label each vertex using the set $[n]$. Note that $|E|=O(n^2)$, so our goal is still to construct a certificate of size $\mathrm{poly}(n)$ while using $O(\log (n))$ space. First, let $v\in V$. We say that $v$ is reachable from $s$ in $G$ if there exists a path from $s$ to $v$. Now, for every $i\in \{0,1,\dots ,n\}$, we can define the set $C_i\subseteq V$ to be the set of vertices $v\in V$ such that $v$ is reachable from $s$ in at most $i$ steps. Notice by definition we have $C_{i-1}\subseteq C_i$.

Now, if we can generate a certificate for the set $C_n$ and show that $t\notin C_n$, we are done. This is because the set $C_n$ is the set of all vertices in the graph that are reachable from $s$ in at most $n$ steps. Since the graph has $n$ vertices, if there is a path from $s$ to $v$ with more than $n$ steps, there is another path from $s$ to $v$ with at most $n$ steps.

Building Sub-certificates. We will build a number of sub-certificates that will help us build our final read-once certificate. First, for any $i\in [n]$ and $u\in C_i$, we give a read-once certificate showing the statement “$u\in C_i$”. We’ll call this certificate a $P1$ certificate.

$P1$ certificate:

• The certificate will consist of $k+1$ vertices $(v_0,v_1,\dots ,v_k)$.
• Verification of the certificate proceeds as follows.
  1. Check if $v_0=s$. This can be done in logarithmic space since this is just comparing $\log (n)$ bits.
  2. For all $j\in [k]$, check if $(v_{j-1},v_j)\in E$. Note here that $E$ is part of the input and can be read multiple times. This again can be done in logarithmic space since we are just storing a counter for $k$, and comparing two bit strings of length $\log (n)$.
  3. For all $j\in [k]$, check if $u=v_j$. Again, this is easily done in logarithmic space.
  4. Count up to $k$ and check that $k\le i$. This again only needs $O(\log (n))$ bits.

Note here that checks (2) and (3) are done at the same time since we are reading the certificate once. The certificate $P1$ will be used in our final certificate. One main issue is that we need to be able to verify for any $i$ that $C_i$ is actually the set of all vertices reachable from $s$ in at most $i$ steps in the graph $G$. If we do not verify this, then it is trivial to come up with a certificate that $t\notin C_n$ when it actually is.

We’ll now build on the ideas of the certificate $P1$. In particular, we’ll need two certificates that are more complicated.

• Certificate $D1$: this certificate will certify the statement $u\notin C_i$ given that $|C_i|=s_i$. Here, given means that we have already verified this statement to be true.
• Certificate $D2$: this certificate will certify that $s_i=|C_i|$ given that $s_{i-1}=|C_{i-1}|$. We’ll use the certificate $P1$ to help us build these two certificates. Note that by definition, we know that $C_0=\{s\}$, and it does not need to be included. This will serve as our base case for the final certificate, which will be an “inductive” certificate that continuously builds up to showing that $|C_n|=s_n$ and $t\notin C_n$.

$D1$ certificate: certifying that $v\notin C_i$ given $s_i=|C_i|$.

• Certificate $P1$ gives us a certificate for verifying $u\in C_i$ for any $u$.
  • Let $w_{u,i}$ be this certificate.
• We construct a new certificate for the statement $v\notin C_i$. We denote this certificate by ${\overline{w}}_{v,i}$. It is defined as $${\overline{w}}_{v,i}=(w_{u_1,i},w_{u_2,i},\dots ,w_{u_{s_i},i}).$$ Here, we have $C_i=\{u_1,u_2,\dots ,u_{s_i}\}$ and $u_i<u_{i+1}$ for all $i\in [s_i-1]$.
• Given we know that $s_i=|C_i|$, we can use the certificate ${\overline{w}}_{v,i}$ to verify that $v\notin C_i$ as follows.
  1. Run the verification procedure on all $w_{u_j,i}$ to certify they are valid (i.e., the verification procedure for $P1$).
  2. Check that $u_j<u_{j+1}$ for all $j\in [s_i-1]$.
  3. Check that $v\ne u_j$ for all $j\in [s_i]$.
  4. Check that ${\overline{w}}_{v,i}$ has exactly $s_i$ certificates.

Again, all of these checks can be arranged so they are executed in a read-once manner, and they all require only $O(\log (n))$ bits of space to check.

Before building the certificate $D2$, we actually need another helper certificate, which we denote as $D1.5$. This certificate will certify $v\notin C_i$ given $s_{i-1}=|C_{i-1}|$.

$D1.5$ certificate: certifying that $v\notin C_i$ given $s_{i-1}=|C_{i-1}|$.

• The certificate here will be identical to the certificate for $D1.5$, except for $s_{i-1}$.
• Let $\{u_1,\dots ,u_{s_{i-1}}\}=C_{i-1}$ such that $u_j<u_{j+1}$ for all $j\in [s_{i-1}-1]$. Also let $w_{u_j,i-1}$ denote the $P1$ certificate for $u_j\in C_{i-1}$ for all $j\in [s_{i-1}]$.
• Let ${\tilde{w}}_{v,i}=(w_{u_1,i-1},\dots ,w_{u_{s_{i-1}},i-1})$. This is our certificate for $D1.5$.
• To verify ${\tilde{w}}_{v,i}$, we perform nearly identical checks as with $D1$.
  1. Verify all $w_{u_j,i-1}$ are valid.
  2. Check $u_j<u_{j+1}$ for all $j$.
  3. Check that $v\ne u_j$ and $v$ is not a neighbor of $u_j$ for all $j$.
  4. Exactly $s_{i-1}$ $P1$ certificates are given.

As always, we can rearrange the above checks to execute them in a read-once manner. Now we have the tools we need to construct a certificate for $D2$.

$D2$ certificate: certifying that $|C_i|=s_i$ given $|C_{i-1}|=s_{i-1}$.

• Certificate $P1$ certifies that $u\in C_i$.
• Certificate $D1.5$ certifes that $v\notin C_i$ given $s_{i-1}=|C_{i-1}|$. We are also given this by our inductive assumption for this certificate.
• We use this assumption plus the above certificates to construct a certificate for $s_i=|C_i|$.
• We let ${\hat{w}}_i$ denote this certificate.
  • It will be the concatenation of $n$ certificates, one for every vertex: $${\hat{w}}_i=({\alpha }_{1,i},{\alpha }_{2,i},\dots ,{\alpha }_{n,i}).$$ Here, $${\alpha }_{j,i}=\{\begin{array}{ll}{w}_{j,i}& \text{ a }P1\text{ certificate if }j\in C_i\\ {\tilde{w}}_{j,i}& \text{ a }D1.5\text{ certificate if }j\notin C_i\end{array}$$
• We certify ${\hat{w}}_i$ as follows.
  1. Check that the number of $P1$ certificates is $s_i$ and that these $w_{j,i}$ certificates are valid.
  2. Certify all ${\tilde{w}}_{j,i}$ certificates using the procedure of $D1.5$.

Final Certificate. All together, we have the tools for the final certificate to verify that there is no path from $s$ to $t$ in the graph $G$. The final certificate $w$ will consist of $n+1$ certificates. $$w=({\hat{w}}_1,{\hat{w}}_2,\dots ,{\hat{w}}_n,{\overline{w}}_{t,n}).$$ Here, ${\hat{w}}_j$ is a $D2$ certificate verifying that $s_j=|C_j|$ given $s_{j-1}=|C_{j-1}|$. Note that since we are given $s_0=1$ by definition, the certificate iteratively builds the sets $C_1$ then $C_2$, up to $C_n$. Once the final size of the set $C_n$ is certified, the final certificate is a $D1$ certificate which certifies that $t\notin C_n$ given $s_n=|C_n|$. $\square$

The following is a corollary of the proof.

Corollary. For all space constructible functions $S\ge \log (n)$, we have $\mathbf{NSPACE}(S)=\mathbf{coNSPACE}(S)$.

Introduction to the Polynomial Hierarchy

The motivation for studying the Polynomial Hierarchy is quite simple: sometimes, non-determinism is not strong enough to capture a decision problem itself.

First, recall the independent set problem. $$\mathrm{INDSET}=\{(G,k):G\text{ has an independent set of size at least }k\}$$ Consider the following natural modification of the independent set problem, which we call exact independent set. $$\mathrm{Exact}\text{-}\mathrm{INDSET}=\{(G,k):\text{ the largest independent set in }G\text{ has size }k\}$$

Another way to rephrase the exact independent set problem is as follows. $(G,k)\in \mathrm{Exact}\text{-}\mathrm{INDSET}$ if and only if $\exists$ an independent set $S$ of size $k$ in $G$ such that $\forall$ other independent sets $S^{\prime }$, $|S^{\prime }|\le k$.

If we stated this with a Turing machine, it would look like $$(G,k)\in \mathrm{Exact}\text{-}\mathrm{INDSET}⟺\exists w_1\forall w_{i}M((G,k),w_1,w_2),$$ where intuitively $w_1$ is the certificate for the independent set $S$ of size $k$ and $w_2$ are all other independent sets in $G$ (which the machine $M$ checks that they have size at most $k$). Clearly, this is not captured by the certificate definition of $\mathbf{NP}$, which is equivalent to the NTM definition. So here, non-determinism alone is not enough.

The Class ${\Sigma }_2^p$

Before examining the full Polynomial Hierarchy, let’s capture the set of languages that follow the same structure as the exact independent set problem we outlined above. This is the class ${\Sigma }_2^p$.

Definition. The class ${\Sigma }_2^p$ is the set of all languages $L$ such that there exists a deterministic Turing machine $M$ (the verifier) and a polynomial $q$ such that for all $x\in \{0,1{\}}^{\ast }$, $x\in L$ if and only if $\exists w_1\in \{0,1{\}}^{q(|x|)}\forall w_2\in \{0,1{\}}^{q(|x|)}M(x,w_1,w_2)=1$.

It is helpful to see another example of a language in the class ${\Sigma }_2^p$. This is what is known as $\mathrm{MIN}\text{-}\mathrm{EQ}\text{-}\mathrm{DNF}$, and is defined as follows. $$\mathrm{MIN}\text{-}\mathrm{EQ}\text{-}\mathrm{DNF}=\{(\phi ,k):\exists \text{ DNF formula }\psi \text{ of size at most }k\text{ that is equivalent to }\phi \}$$ Here, a DNF formula is a “reverse” CNF: it is a big OR of many ANDs. Stating it again, $(\phi ,k)\in \mathrm{MIN}\text{-}\mathrm{EQ}\text{-}\mathrm{DNF}$ if $\exists$ DNF $\psi$ of size at most $k$ such that $\forall u$, we have $\phi (u)=\psi (u)$ (they agree on all possible assignments).

Notice, in fact, that both $\mathbf{NP}$ and $\mathbf{coNP}$ are contained in ${\Sigma }_2^p$.