Lecture 5

In-class notes: CS 505 Spring 2025 Lecture 5

Cook-Levin Theorem Wrap-Up

Recall from last time, we are trying to prove that $SAT$ is $\mathbf{NP}$-complete. To do so, we considered the single-tape non-deterministic Turing machine definition of $\mathbf{NP}$. Our goal is to show that for any $\mathbf{NP}$ language $L$, we have $L{\le }_{p}SAT$. That is, $L$ is poly-time reducible to $SAT$.

From last time, we were able to construct an $n^k\times n^k$ table which encoded the execution of an NTM deciding $L$ on input $x$. From this table, we constructed the Boolean formula

$$\varphi ={\varphi }_{start}\wedge {\varphi }_{accept}\wedge {\varphi }_{cell}\wedge {\varphi }_{move}.$$

The last thing to show is that our definition of ${\varphi }_{move}$ correctly captures the correctness of the NTM $M$ deciding $L$. Recall that ${\varphi }_{move}$ was defined with respect to $2\times 3$ windows in the table, and it tried to capture the notion of a legal window. That is, $${\varphi }_{move}=\underset{\begin{array}{c}0\le i<n^k-1\\ 0\le j<n^k-2\end{array}}{\bigwedge }[\text{Window }{W}_{i,j}\text{ is legal}].$$

Claim. If the table has a correct starting configuration, and all $2\times 3$ windows are legal, then $row[i+1]$ is a correct transition from $row[i]$ for all $0\le i<n^k$.

Proof. To prove the claim, first consider any such $i$. Let $row[i]$ and $row[i+1]$ be the $i$ and $i+1$ rows of the table. Call $row[i]$ the upper configuration and $row[i+1]$ the lower configuration.

Consider all windows $W_{i,j}$ for $0\le j<n^k-2$. That is, we look at all windows in the upper and lower configuration. We now define when window $W_{i,j}$ is legal. Legal windows fall into two categories: windows which contain a state and those which do not.

• No state in the window. Suppose window $W_{i,j}$ contains no state. Then we say that $W_{i,j}$ is legal if and only if the two elements in the center column are equal. The window below is an example. $$W_{i,j}=\left[\begin{array}{ccc}a& b& c\\ c& b& c\end{array}\right]$$ Note that even though in the above example, the first column has $a$ then $c$, this would be a legal window because it is possible the tape head is just to the left of $a$ in the upper configuration, writes $c$ over $a$, then moves left.

• State in the window. Suppose that window $W_{i,j}$ contains a state. Then window $W_{i,j}$ is legal if and only if the upper and lower configuration in this window is consistent with the transition function of the Turing machine. In particular, by our construction of the table and since the NTM is a single-tape NTM, a state in the window represents the current position of the tape head. First, we know that when transitioning from the upper configuration to the lower configuration, the state can move at most one position (left, right, or stay). This is easy to check for. Then we know that in the table, the tape head only touches the cell immediately to its right. That is, if the state is in $cell[i,j]$, then the tape head is reading from/writing to $cell[i,j+1]$. In a nutshell, the computation of a Turing machine is highly local: it can’t jump large distances in a single time-step. Examples of legal windows are given below. $$\begin{array}{rlrlr}\left[\begin{array}{ccc}a& q_1& b\\ a& q_2& b\end{array}\right]& & \left[\begin{array}{ccc}{q}_1& a& b\\ c& q_2& b\end{array}\right]& & \left[\begin{array}{ccc}{q}_1& b& c\\ d& c& c\end{array}\right]\end{array}$$

• Special windows. There are two special windows in any pair of upper and lower configurations: $W_{i,0}$ and $W_{i,n^k-3}$. These represent the edges of the table. These windows are legal if and only if: (1) they satisfy both of the above constraints; and (2) they have the fixed $\#$ symbol on the edges. See the examples below.

$$\begin{array}{rlr}{W}_{i,0}=\left[\begin{array}{ccc}\#& q_1& a\\ \#& a& q_2\end{array}\right]& & W_{i,n^k-3}=\left[\begin{array}{ccc}c& d& \#\\ q_7& d& \#\end{array}\right]\end{array}$$

By the above notion of legal windows, if all windows in the upper and lower configuration are legal, then it represents a correct transition to $row[i+1]$ from $row[i]$. Inductively, this means that if we start with a correct starting configuration, and every window in the table is legal, then each pair of upper and lower configurations represents a valid transition from $row[i]$ to $row[i+1]$, and hence the table correctly captures the computation of the decider $M$ for language $L$.

We conclude by giving the Boolean formula for ${\varphi }_{move}$. To do so, we simply need to give a Boolean formula for the statement “$W_{i,j}\text{ is a legal window}$.” Define the set $S$ as follows. $$S=\left\{(a_1,a_2,a_3,a_4,a_5,a_6):a_i\in C\forall i\in [6]\wedge \left[\begin{array}{ccc}{a}_1& a_2& a_3\\ a_4& a_5& a_6\end{array}\right]\text{ is a legal window}\right\}.$$ Here, recall that $C$ is the cell alphabet of our table.

Given this set $S$, the Boolean formula for the statement “$W_{i,j}\text{ is a legal window}$” is expressed as $$\underset{(a_1,\dots ,a_6)\in S}{\bigvee }\left(x_{i,j,a_1}\wedge x_{i,j+1,a_2}\wedge x_{i,j+2,a_3}\wedge x_{i+1,j,a_4}\wedge x_{i+1,j+1,a_5}\wedge x_{i+1,j+2,a_6}\right)$$ What is this formula saying? It says that given a tuple $(a_1,\dots ,a_6)$, which I know by the definition of $S$ represents some legal window, is the current window $W_{i,j}$ this legal window? That is, it is asking if $$W_{i,j}\stackrel{?}{=}\left[\begin{array}{ccc}{a}_1& a_2& a_3\\ a_4& a_5& a_6\end{array}\right]$$ is true. We take a big OR over all legal windows to make sure that window $W_{i,j}$ is some legal window.

If this big OR is true, then we know $W_{i,j}$ is some legal window. This gives us the final expression for ${\varphi }_{move}$ as $${\varphi }_{move}=\underset{\begin{array}{c}0\le i<n^k-1\\ 0\le j<n^k-3\end{array}}{\bigwedge }\underset{(a_1,\dots ,a_6)\in S}{\bigvee }\left(x_{i,j,a_1}\wedge x_{i,j+1,a_2}\wedge x_{i,j+2,a_3}\wedge x_{i+1,j,a_4}\wedge x_{i+1,j+1,a_5}\wedge x_{i+1,j+2,a_6}\right).$$ All together, we have that $\varphi ={\varphi }_{start}\wedge {\varphi }_{accept}\wedge {\varphi }_{cell}\wedge {\varphi }_{move}$ is satisfiable if and only if the NTM we are encoding in the table is accepting.

The final piece of the puzzle is arguing that we can construct $\varphi$ in polynomial time. Note that the cell alphabet $C=\Gamma \cup Q\cup \{\#\}$ is of constant size with respect to the input length by definition of Turing machines.

• For ${\varphi }_{start}$, given an input $x$ to the NTM $M$ for deciding the language $L$, the starting configuration of the machine is fixed. Thus, the starting row of the table is fixed as well. For $n=|x|$, the starting row of the table contains $n^k$ cells, which corresponds to $n^k$ literals in ${\varphi }_{start}$. This can clearly be constructed in $O(n^k)$ time.
• For ${\varphi }_{accept}$, recall that we are simply scanning the entire $n^k\times n^k$ table for an accepting state. The table has total size $n^{2k}$, so this formula clearly has size $n^{2k}$ and can be constructed in $O(n^{2k})$ time.
• For ${\varphi }_{cell}$, it is a big AND of $n^{2k}$ pairs $i,j$. Within this big AND, we have two constant sized subformulas. First, the formula checking that cell $i,j$ contains a valid symbol $s\in C$. Since $|C|$ is constant, the size of this formula is constant. Then this subformula is AND’d with a big AND of an OR which checks that cell $i,j$ doesn’t contain both symbol $s\in C$ and $t\in C$. Again, since $|C|$ is constant, this subformula is constant. So the total size of ${\varphi }_{cell}$ is $O(n^{2k})$ and can be constructed in this much time as well.
• Similarly, for ${\varphi }_{move}$, the size of the set $S$ is at most $|C|!/(|C|-6)!$, which is constant size (something like $O(|C{|}^6)$) since $|C|$ and $6$ are constants. So the inner formula is a constant size, whereas the whole formula ${\varphi }_{move}$ is a big AND of pairs $i,j$ from $0$ to at most $n^k-1$. So ${\varphi }_{move}$ has size $O(n^{2k})$ and can be constructed in this much time.

This completes the proof of the Cook-Levin theorem. $\square$

Other NP-Complete Problems

SAT is a step-up from the (useless) $\mathbf{NP}$-Complete problem TMSAT. However, a general Boolean formula (like those given in SAT) may be difficult to handle when trying to understand specific $\mathbf{NP}$ problems. Thus, we turn our attention to the wide variety of other $\mathbf{NP}$-complete problems.

First, we show that given any $\mathbf{NP}$-complete problem/language, if we want to show some other language is $\mathbf{NP}$-complete, we only need to reduce our known $\mathbf{NP}$-complete language to our new language.

Theorem 5.1. If $A$ is an $\mathbf{NP}$-complete language, and $B\in \mathbf{NP}$ such that $A{\le }_{p}B$, then $B$ is $\mathbf{NP}$-complete.

Proof. Recall the transitive property of polynomial-time reducible languages. Let $L_1,L_2,L_3$ be languages such that $L_1{\le }_p{L}_2$ and $L_2{\le }_p{L}_3$. Then we know that $L_1{\le }_p{L}_3$.

By our assumption, $A$ is $\mathbf{NP}$-complete. This means that $A\in \mathbf{NP}$ and $L{\le }_{p}A$ for all $L\in \mathbf{NP}$. By our other assumption, we know that $A{\le }_{p}B$. By the transitive property above, we now know that $L{\le }_{p}B$ for any $L\in \mathbf{NP}$. Thus, $B$ is $\mathbf{NP}$-complete. $\square$

Now, rather than having to do a complete Cook-Levin Theorem style proof for new languages we want to show are $\mathbf{NP}$-complete, it suffices to just reduce from a language we know is $\mathbf{NP}$-complete!

3SAT

We turn to our next (and possibly favorite) $\mathbf{NP}$-complete language: 3SAT. First, we need to set up some terminology.

Let $\varphi$ be a Boolean formula. We say that $\varphi$ is in conjunctive normal form (or $\varphi$ is a CNF formula) if $$\varphi ={\varphi }_1\wedge {\varphi }_2\wedge \cdots \wedge {\varphi }_n$$ such that ${\varphi }_i$ only contains ORs of literals/variables (and their negations). We call each ${\varphi }_i$ a clause of $\varphi$. One example of a CNF formula $\varphi$ with $3$ clauses is given below.

$$\varphi =(x_1\vee {\overline{x}}_2)\wedge x_3\wedge ({\overline{x}}_1\vee x_4\vee x_5\vee x_6)$$

We say that $\varphi$ is a $k$-CNF formula if each clause ${\varphi }_i$ contains exactly $k$ literals. An example of a $2$-CNF formula with $3$ clauses is given below.

$$\varphi =(x_1\vee x_2)\wedge ({\overline{x}}_3\vee x_4)\wedge ({\overline{x}}_2\vee x_3)$$

Definition (3SAT). The language $3SAT$ is the set of all satisfiable $3$-CNF formulas. That is, $$3SAT=\{⟨\varphi ⟩:\varphi \text{ is a 3-CNF and is satisfiable}\}.$$

Complexity theorists prefer $3SAT$ over other NP-complete languages since it is simple, has very little combinatorial structure, and occurs in many differnt contexts such as constraint satisfaction problems.

The other part of the Cook-Levin Theorem (that I hid from you earlier) is that $3SAT$ is NP-complete.

Theorem (Cook-Levin, Part 2). $3SAT$ is $\mathbf{NP}$-complete.

Proof. $3SAT\in \mathbf{NP}$ is immediate. What remains to be shown is that $3SAT$ is NP-hard. We could show that $SAT{\le }_{p}3SAT$ using our above theorem, but it is actually simpler just to modify the proof of the Cook-Levin theorem directly to give us a 3-CNF formula.

Recall $\varphi ={\varphi }_{start}\wedge {\varphi }_{accept}\wedge {\varphi }_{cell}\wedge {\varphi }_{move}$ from the proof of the Cook-Levin theorem. First, we will change $\varphi$ slightly so that it is a CNF formula (we are almost there already). Once we have put $\varphi$ in CNF form, we will then transform it into a $3$-CNF formula.

Since $\varphi$ is the AND of $4$ clauses, we just need to make sure that each of these clauses is an OR of 1 or more literals. First consider ${\varphi }_{start}$. Recall that it was simply ANDing $n^k$ literals together: $${\varphi }_{start}=x_{0,0,\#}\wedge x_{0,1,q_0}\wedge x_{0,1,u_1}\wedge \cdots \wedge x_{0,n+1,u_n}\wedge x_{0,n+2,\square }\wedge \cdots \wedge x_{0,n^k-2,\square }\wedge x_{0,n^k-1,\#}.$$ So ${\varphi }_{start}$ is already a CNF formula with $n^k$ clauses of single literals (there are no ORs).

Now consider ${\varphi }_{accept}$. Remember that this is simply a big OR over the entire table, checking if there is at least one accepting state. So ${\varphi }_{accept}$ will be a single clause of our CNF formula.

Now consider ${\varphi }_{cell}$. From before, this is given by $${\varphi }_{cell}=\underset{0\le i,j\le n^k-1}{\bigwedge }\left[\left(\underset{s\in C}{\bigvee }{x}_{i,j,s}\right)\wedge \left(\underset{\begin{array}{c}s,t\in C\\ s\ne t\end{array}}{\bigwedge }({\overline{x}}_{i,j,s}\vee {\overline{x}}_{i,j,t})\right)\right].$$ Notice that ${\varphi }_{cell}$ is already in CNF form. The big OR over $s\in C$ is a single clause, which gets AND’d with the formula ${\bigwedge }_{\begin{array}{c}s,t\in C\\ s\ne t\end{array}}({\overline{x}}_{i,j,s}\vee {\overline{x}}_{i,j,t})$, which is itself a CNF formula. Then all of these formulas are AND’d together, meaning the final formula ${\varphi }_{cell}$ is in CNF form.

Finally, for ${\varphi }_{move}$, it is a big AND of a big OR of a constant number of ANDs (6 ANDs). Using Boolean equivalences, we can convert the inner formula ${\bigvee }_{(a_1,\dots ,a_6)\in S}(a_1\wedge a_2\wedge \cdots \wedge a_6)$¹ into a new formula where it is a big AND of some (again constant) number of ORs. This conversion increases the formula size by at most a polynomial-time factor, so ${\varphi }_{move}$ is still of polynomial size. All together, this transforms ${\varphi }_{move}$ into CNF form in polynomial time.

This together establishes that we can convert our original formula $\varphi ={\varphi }_{start}\wedge {\varphi }_{accept}\wedge {\varphi }_{cell}\wedge {\varphi }_{move}$ into an equivalent CNF formula, say ${\varphi }^{\prime }={\varphi }_1\wedge \cdots \wedge {\varphi }_m$ for some $m$ such that each ${\varphi }_i$ is the OR of one or more literals. We now convert ${\varphi }^{\prime }$ into a $3$-CNF. This can be done as follows. For any $i\in [m]$, consider the clause ${\varphi }_i$.

• If ${\varphi }_i$ has 3 literals, we are done and can move on to the next clause.
• If ${\varphi }_i$ has less than 3 literals, we transform it into an equivalent formula ${\varphi }_i^{\prime }$ with exactly 3 literals. For example, if ${\varphi }_i$ has one literal, say $x_j$, we simply write ${\varphi }_i^{\prime }=(x_j\vee x_j\vee x_j)$. If ${\varphi }_i$ has two literals, say $x_{j_1}$ and $x_{j_2}$, we pick one of the literals arbitrarily (e.g., always pick the first one) and repeat it, giving ${\varphi }^{\prime }=(x_{j_1}\vee x_{j_1}\vee x_{j_2})$. Clearly ${\varphi }_i$ and ${\varphi }_i^{\prime }$ are equivalent.
• If ${\varphi }_i$ has more than 3 literals, we will split ${\varphi }_i$ into a $3$-CNF formula using extra variables. For example, if ${\varphi }_i=x_1\vee x_2\vee x_3\vee x_4$, we introduce the variable $z$ and convert ${\varphi }_i$ to formula $${\varphi }_i^{\prime }=(x_1\vee x_2\vee z)\wedge (\overline{z}\vee x_2\vee x_3).$$ This conversion has the property that if ${\varphi }_i$ has a satisfying assignment, then there exists an assignment to the variable $z$ such that ${\varphi }_i^{\prime }$ is also satisfied by $z$ plus the assignment for ${\varphi }_i$. In our above example, the vector $(x_1,x_2,x_3,x_4)=(1,0,0,0)$ is a satisfying assignment for ${\varphi }_i$, and so a satisfying assignment for ${\varphi }_i^{\prime }$ would be $(x_1,x_2,x_3,x_4,z)=(1,0,0,0,0)$ (set $\overline{z}=1$).
  
  In general, if ${\varphi }_i$ has $\ell \ge 4$ literals, we introduce $\ell -3$ new variable $z_1,\dots ,z_{\ell -3}$ and transform ${\varphi }_i$ into a 3CNF formula with $\ell -2$ clauses. If ${\varphi }_i$ has literals $a_1,\dots ,a_{\ell }$, then we construct ${\varphi }_i^{\prime }$ as $${\varphi }_i^{\prime }=(a_1\vee a_2\vee z_1)\wedge ({\overline{z}}_1\vee a_3\vee z_2)\wedge ({\overline{z}}_2\vee a_4\vee z_3)\wedge \cdots \wedge ({\overline{z}}_{\ell -3}\vee a_{\ell -1}\vee a_{\ell }).$$ Clearly, ${\varphi }_i^{\prime }$ can be constructed in polynomial time from $\varphi$.

All together, this gives us our new 3CNF formula ${\varphi }^{\prime }$ that is equivalent to the formula $\varphi$ we constructed in the Cook-Levin Theorem. $\square$

Independent Set

The independent set problem on an undirected graph $G=(V,E)$ asks if there exists a set of node/vertices $S\subseteq V$ of size at least $k$ such that they are pairwise disconnected. That is, for every $u,v\in S$, $(u,v)\notin E$. As a set, this is written as $$INDSE{T}_k=\{⟨G=(V,E),k⟩:\exists S\subset V\text{ s.t. }|S|\ge k\wedge (u,v)\notin E\forall u,v\in S\}.$$

Theorem. $INDSE{T}_k$ is NP-complete.

Proof. Clearly $INDSE{T}_k\in \mathbf{NP}$. To see this, one can simply specify a set $S$ of size at least $k$. Then, verification that $S$ is an independent set takes at most $O(n^2)$ time per pair of $u,v\in S$ since in the worst case you must scan the entire set $E$ per check. So the total time is at most $O(n^3)$ in the worst case, where $n=|V|$.

Now we show that $INDSE{T}_k$ is NP-complete. We do this by giving a reduction from $3SAT$. That is, $3SAT\le INDSE{T}_k$.

Suppose that $\varphi$ is a 3CNF formula with $k$ clauses: $\varphi ={\varphi }_1\wedge {\varphi }_2\wedge \cdots \wedge {\varphi }_k$. Assume that $\varphi$ has $\ell$ literals $x_1,\dots ,x_{\ell }$ (note their negation are also literals). For each clause ${\varphi }_i$, write ${\varphi }_i=(a_i\vee b_i\vee c_i)$, where $a_i,b_i,c_i$ are the literals of clause ${\varphi }_i$. (For example, if ${\varphi }_i=(x_2\vee {\overline{x}}_7\vee x_3)$, then $a_i=x_2$, $b_i={\overline{x}}_7$, and $c_i=x_3$.)

For each clause ${\varphi }_i$, we create a cluster of $3$ nodes/vertices in a graph $G$. Label each vertex in this cluster with $a_i$, $b_i$, $c_i$. This gives us $k$ clusters of $3$ nodes each, where each cluster of $3$ nodes is associated with ${\varphi }_i$ and labeled $a_i,b_i,c_i$.

Now we connect nodes in this graph with $3k$ vertices. First, create a triangle in each cluster. That is, for each $i$, connect $(a_i,b_i)$, $(b_i,c_i)$, and $(c_i,a_i)$ (note the graph is undirected so $(b_i,a_i)$, $(c_i,b_i)$ and $(a_i,c_i)$ are also edges). Next, we connect each node with its negation. For example, if $a_1=x_3$ and $b_7={\overline{x}}_3$, then we add the edge $(a_1,b_7)$ to the graph. We claim that the given 3CNF $\varphi$ is satisfiable if and only if our constructed graph $G$ above has an independent set of size $k$.

First, suppose that $\varphi$ has a satisfying assignment. Let $u\in \{0,1{\}}^{\ell }$ be the satisfying assignment. That is, $x_i=u_i$ is our satisfying assignment for $i\in [\ell ]$. From $u$, we build a $k$-independent set in the graph $G$. Now since $\varphi$ is satisfied, the assignment $u$ satisfies every clause ${\varphi }_i$ of $\varphi$. Since every clause ${\varphi }_i$ is satisfied, as least one of its literals $a_i$, $b_i$, or $c_i$ is equal to $1$. For each literal ${\varphi }_i$, choose only one of its satisfied literals and add it to the set $S$. For example, if $a_i=1$, $b_i=1$, and $c_i=0$ under assignment $u$, then add $a_i$ or $b_i$ to $S$ (but not both; simply choose one of them).

We claim this set $S$ constructed in this manner is an independent set of size $k$. First, suppose that a literal $a_j\in S$ for some $j$ (e.g., $a_1$). By construction of the graph $G$, we know that $a_j$ is connected to both $b_j$ and $c_j$. But by our selection of the set $S$, we only choose a single node from each cluster to add to the set. So $b_j,c_j\notin S$. Now what if literals $c_i,a_j\in S$ for $i\ne j$? By construction of $G$, we know that $a_j$ and $c_i$ are connected if and only if $a_j$ is the negation of the literal represented by $c_i$. For example, $a_j={\overline{x}}_3$ and $c_i=x_3$ would be connected in the graph. However, by assumption, the assignment $u$ is a satisfying assignment. This means that if $a_j$ were satisfied (i.e, $a_j=1$), then $c_i$ would not be satisfied (i.e., $c_i=0$). So every element $s,s^{\prime }\in S$ satisfies the property that $(s,s^{\prime })\notin E$. Thus, $S$ is an independent set of size $k$.

Now suppose that our constructed graph $G$ has an independent set of size $k$. We reconstruct a satisfying assignment for the formula $\varphi$. Let $S$ be the $k$-independent set. Then for every $s,s^{\prime }\in S$, we know that $(s,s^{\prime })\notin E$. We construct our satisfying assignment directly from the set $S$. Suppose $s\in S$. Then set literal $s$ in $\varphi$ to be $1$. For example if $s=a_j={\overline{x}}_3$, then we set $a_j={\overline{x}}_3=1$ in the satisfying assignment (that is, $x_3=0$). We claim this is a satisfying assignment. This is by construction of the graph $G$.

• Suppose $s\in S$ and $s$ is in cluster $i$. Then we know $s$ is connected to every other node in its cluster ($a_i,b_i,c_i$ are all connected in cluster $i$). So we know that the other nodes in this cluster are not in $S$ (otherwise it would not be an independent set).
• Let $s,s^{\prime }\in S$ such that $s$ is in cluster $i$ and $s^{\prime }$ is in cluster $j$. We know that $s,s^{\prime }$ are not connected. This implies that $s$ and $s^{\prime }$ are not a literal and their negation (e.g., $s=x_2$ and $s^{\prime }={\overline{x}}_2$ is not possible). This means that we don’t obtain an assignment which sets both a literal and its negation to $1$.

Thus, our constructed assignment is satisfying. This completes the proof. $\square$

1. Please see the actual definition of ${\varphi }_{move}$ for the correct formula here. I am just using a shorthand for demonstration. ↩