Lecture 7

In-class notes: CS 505 Spring 2025 Lecture 7

Diagonalization

Suppose we are given complexity classes ${\mathbf{C}}_1$ and ${\mathbf{C}}_2$. How can we show they are different? That is, show ${\mathbf{C}}_1\ne {\mathbf{C}}_2$.

We’ve seen the technique of diagonalization before when we showed undecidability of certain languages. Diagonalization is a general technique that gives us one way of showing the above result: differentiating between complexity classes. Intuitively, if we are given $L_1\in {\mathbf{C}}_1$ and $L_2\in {\mathbf{C}}_2$, diagonalization allows us to differentiate between ${\mathbf{C}}_1$ and ${\mathbf{C}}_2$ as follows.

• If $M_1$ decides $L_1$, then we want to say $M_1$ is different from any decider $M_2$ for $L_2$.
• We do this by arguing for any $x$, if $M_2(x)=1$ then $M_1(x)=0$, and vice versa.

Origins of Diagonalization

Diagonalization was originally introduced by Georg Cantor. He used diagonalization to prove that $|\mathbb{N}|<|\mathbb{R}|$. That is, the set of all natural numbers is strictly smaller than the set of all real numbers. This result at the time was not well received: these are both infinite sets, how could you possibly reason about them being different sizes?

This proof first relies on defining when two infinite-sized sets are the same size. Briefly, two sets of infinite size $S_1,S_2$ are said to have the same size $|S_1|=|S_2|$ if there exists a bijection $f$ from $S_1$ to $S_2$. That is, for every $s_1\in S_1$, there is a unique $s_2\in S_2$ such that $f(s_1)=s_2$.

For example, we know that $|\mathbb{N}|=|\mathbb{Z}|$ via the following bijection. $$\begin{array}{c}0\mapsto 01\mapsto 12\mapsto -13\mapsto 24\mapsto -2\cdots \\ f(i)=\{\begin{array}{ll}-\lceil i/2\rceil & i\text{ is even}\\ \lceil i/2\rceil & i\text{ is odd}\end{array}.\end{array}$$

Under this definition of set equality, Cantor showed that $|\mathbb{N}|<|\mathbb{R}|$. We’ll do an easier proof by showing $\mathbb{N}$ is smaller than the entire interval of real numbers $[0,1]$.

Theorem. $|\mathbb{N}|<|[0,1]|$.

Proof. We do a proof by contradiction. Suppose that $|\mathbb{N}|=|[0,1]|$. This means there is a bijection from $\mathbb{N}$ to $[0,1]$. We can write the bijection as an infinite table.

From this table, we’ll construct a new real number $r\in [0,1]$ that is not in the above bijection. We construct the real number $r=0.r_0{r}_1{r}_2\dots$ digit by digit. We’ll index digits in the right column of the table starting with $0$ (i.e., the first digit after the decimal point is the 0-th digit). Let $f$ denote the bijection described by the table.

Then, for each $i\in \mathbb{N}$, we define $r_i=d$ for any $d\in \{0,1,\dots ,9\}$ such that $d\ne f(i{)}_i$. That is, the $i$-th digit of $r$ will be explicit different from the $i$-th digit of the real number $f(i)$. As a picture, we look at the digits in the table on the diagonal.

Taking the positions on the diagonal, we construct the new real number $r=0.27619\dots$. Now, there does not exist any $i\in \mathbb{N}$ such that $f(i)=r$. This is because for every $i$, we have $r_i\ne f(i{)}_i$, which implies that $r\ne f(i)$. Thus, the mapping $f$ cannot exist. $\square$

Time Hierarchies

With diagonalization fleshed out more, we can now discuss time hierarchies.

Deterministic Time Hierarchy

First, we’ll show a time hierarchy theorem for deterministic computations.

Theorem. Let $f$ and $g$ be time constructible functions such that $f=o(g/\log (g))$. Then $\mathbf{DTIME}(f)⊊\mathbf{DTIME}(g)$.

As a corollary of the above theorem, we have:

Corollary. There exists a language $L$ decidable in time $f$ but not decidable in time $o(f/\log (f))$ for any time constructible function $f$.

We now prove the time hierarchy theorem.

Proof. As you might expect from the preceding discussion, we’ll have a diagonalization proof. First, we build a deterministic Turing machine $D$ as follows.

For any $x\in \{0,1{\}}^n$ and for any $n\in \mathbb{N}$, $D(x)$ does the following.

1. Compute $k=\lceil g(n)/\log (g(n))\rceil$.
2. Simulate $M_x(x)$.
3. If $M_x(x)$ halts within $k$ steps, output $1-M_x(x)$.
4. Else output $0$.

Note that step (1) can be done in $O(g(n))$ time since $g$ is time constructible. If $M_x$ runs in time $T$, then step (2) runs in time $O(T(n)\log (T(n)))$ since we can do universal simulation with only logarithmic overhead.

Now, let $A=\mathcal{L}(D)$ denote the language of $D$; i.e., $A=\{x|D(x)=1\}$. By definition, $D$ decides $A.$ Moreover, $A\in \mathbf{DTIME}(g)$ since $D(x)$ only simulates $M_x$ for at most $O(g(|x|)/\log (g(|x|)))$ steps.

Claim. $A\notin \mathbf{DTIME}(f)$.

This is where our diagonalization comes into play. Suppose this claim is not true. This implies $A\in \mathbf{DTIME}(f)$, and there is a decider $M_A$ deciding $A$ in time $O(f)$.

Now, consider running $D(⟨M_A⟩)$. Suppose that $n=|⟨M_A⟩|$. Then, $D$ simulates $M_A(⟨M_A⟩)$ for at most $k=O(g(n)/\log (g(n))$ steps. Notice that $M_A$ runs in time $f$ on any input. In particular, $M_A(⟨M_A⟩)$ runs in time $O(f(n))$. By universal simulation, we know that $D$ simulates $M_A(⟨M_A⟩)$ in time $O(f(n)\log (f(n)))$. Since $f=o(g/\log (g))$ we have that $M_A(⟨M_A⟩)$ halts in at most $f(n)=o(g(n)/\log (g(n)))$ steps. So, for large enough $n$,¹ $M_A$ runs for less than $k=\lceil g(n)/\log (g(n))\rceil$ steps. Moreover, by universal simulation, $D$ still runs in at most $O(g(n))$ steps on this input.

This implies that $D(⟨M_A⟩)$ completes the simulation of $M_A(⟨M_A⟩)$ and outputs $1-M_A(⟨M_A⟩)$. However, this implies that $D(⟨M_A⟩)\ne M_A(⟨M_A⟩)$. We assumed that $M_A$ decides the language $A$, which $D$ also decides by definition, but these two machines differ on this input. This is a contradiction, so $M_A$ does not exist. $\square$

We have two important corollaries from the time hierarchy theorem.

Corollary. For all $1\le {\epsilon }_1<{\epsilon }_2$, we have $\mathbf{DTIME}(n^{{\epsilon }_1})⊊\mathbf{DTIME}(n^{{\epsilon }_2})$.

Corollary. $\mathbf{P}⊊\mathbf{EXP}$.

Non-deterministic Time Hierarchy

Now, we move on to show the non-deterministic time hierarchy theorem.

Theorem. Let $f,g$ be time constructible functions such that $f(n+1)=o(g(n))$. Then, $\mathbf{NTIME}(f)⊊\mathbf{NTIME}(g)$.

Proof. Unfortunately, we cannot do a standard diagonalization here. With the deterministic time hierarchy, we simulated the machine (which shouldn’t have existed), and were able to flip its output. The simulation was deterministic and always output the opposite of the machine $M_A$. However, with non-deterministic simulation, there could be exponentially many outputs on a single input. Recall that a non-deterministic decider needs to only output accept on at least one computation path, and reject on all (when rejecting a string).

The idea behind the non-deterministic simulation will be to do a lazy simulation. In particular, our diagonalization will only differ on a single output; for all other outputs, we will output the correct bit (of the machine we are simulating). This will be enough to derive our contradiction.

We proceed with the proof. Let $i\in \mathbf{N}$ and let $M_i$ denote the machine described by $⟨i{⟩}_2$. Now let $f$ be a function such that $f(0)=1$, $f(1)=2$, and $f(i+1)=2^{f(i{)}^2}$. Let $g$ be any function such that $f(n+1)=o(g(n))$.

Build a non-deterministic Turing machine $D$ which does the following. $D$ takes as inputs strings of the form $1^n$ for any $n\in \mathbf{N}$, where $1^n$ denotes the string of $n$ 1’s.

$D(1^n):$

1. Compute $i$ such that $f(i)<n\le f(i+1)$.
2. If $f(i)<n<f(i+1)$:
   1. Non-deterministically simulate $M_i(1^{n+1})$ for at most $g(n)$ steps.
   2. If $M_i$ halts within $g(n)$ steps, output $M_i(1^{n+1})$.
   3. Else output $1$.
3. If $n=f(i+1)$:
   1. Deterministically simulate $M_i(1^{f(i)+1})$ by trying all computation paths.
   2. Output $1-M_i(1^{f(i)+1})$.

Now, we argue that $D(1^n)$ runs in time $O(g(n))$. First, step (1) takes at most $O(f(n))=o(g(n))$ time. Second, all of step (2) only takes $O(g(n))$ time. Third, step (3.1) takes at most $O(2^{f(i)+1})=O(f(i+1))$ time, which overall takes $O(g(n))$ time. Therefore, $D(1^n)$ runs in time $O(g(n))$.

Let $A=\mathcal{L}(D)$. By the above discussion, we know that $A\in \mathbf{NTIME}(g)$.

Claim. $A\notin \mathbf{NTIME}(f)$.

Again, suppose this is not the case. Then there is an NTM $M_A$ which decides $A$ in at most $O(f)$ time. Let $n=|⟨M_A⟩|$ be large enough such that for $i$ satisfying $f(i)<n\le f(i+1)$, we have $M_i=M_A$.

Now, run $D(1^n)$. If $n<f(i+1)$, then $D$ simulates $M_i(1^{n+1})$ for at most $g(n)$ steps. By construction, $M_i=M_A$, and $M_A(1^{n+1})$ runs in non-deterministic time $f(n+1)=o(g(n))$. So the simulation halts before $g(n)$ steps and $D(1^n)=M_A(1^{n+1})$.

This implies the following equalities. $$\begin{array}{c}D(1^{f(i)+1})=M_A(1^{f(i)+2})\\ D(1^{f(i)+2})=M_A(1^{f(i)+3})\\ D(1^{f(i)+3})=M_A(1^{f(i)+4})\\ ⋮\\ D(1^{f(i+1)-1})=M_A(1^{f(i+1)})\end{array}$$

Moreover, by assumption, $M_A$ and $D$ both decide the same language $A$. This implies for all $j\in \{f(i)+1,f(i)+2,\dots ,f(i+1)\}$, we have $$D(1^j)=M_A(1^j).$$ This actually shows that $D(1^j)=D(1^{j+1})$ for all $j\in \{f(i)+1,f(i)+2,\dots ,f(i+1)-1\}$; similarly, the same is true for $M_A$: $M_A(1^j)=M_A(1^{j+1})$ for all $j$.

Now suppose that $n=f(i+1)$. By construction, $D(1^n)$ now simulates $M_A(1^{f(i)+1})$ deterministically and outputs $1-M_A(1^{f(i)+1})$. Here, $M_A$ outputs $1$ if there exists an accepting path, and outputs $0$ otherwise. This implies that $D(1^{f(i+1)})\ne M_A(1^{f(i)+1})$. But this is a contradiction since above we established that $D(1^j)=M_A(1^j)$ for all $j\in \{f(i)+1,f(i)+2,\dots ,f(i+1)\}$. Thus, $M_A$ cannot exist. $\square$

1. Recall that every Turing machine has an infinite number of equivalent strings which describe said machine. ↩