Lecture 9

In-class notes: CS 505 Spring 2025 Lecture 9

Space Complexity

So far, we have only focused on the time complexity of computations. However, an equally important metric to consider is space complexity. Intuitively, though we would like computations to be fast, time is an abundant resource. For example, we do not need to get new hardware to let computations run longer, we simply just let our computers run longer. However, for space, we cannot simply “download more RAM” or disk space. So it is an important factor to consider in computational complexity.

Definition. Let $S:\mathbb{N}\to \mathbb{N}$. A language $L$ is in the class $\mathbf{DSPACE}(S)$ if there exists a deterministic Turing machine $M_L$ which decides the language $L$ using at most $O(S(|x|))$ additional space. That is, for any $x\in \{0,1{\}}^{\ast }$, $x\in L$ if and only if $M_L(x)=1$ and uses $O(S(|x|))$ space on its non-output work tapes.

Intuitively, when considering the space constraints of an algorithm, we do not count the input length against the space usage (since it must read all the input to do a deterministic computation). We also do not count the output tape (and, in fact, sometimes we remove the output tape and simply encode accept/reject in the final halting state).

Similarly, we define the class $\mathbf{NSPACE}(S)$ as the set of all languages decidable by a non-deterministic Turing machine using at most $O(S)$ space on its work tapes for any non-deterministic computational path made by the NTM.

Power of Space-bounded Computations

Intuitively, we believe space-bounded computations to be more powerful than time-bounded ones. This is because we cannot reuse time, but we can continually reuse space. As an example, recall that we do not believe $\mathrm{SAT}\in \mathbf{P}$; that is, no polynomial-time algorithm solves/decides $\mathrm{SAT}$. However, $\mathrm{SAT}\in \mathbf{DSPACE}(n)$: it is solvable only using linear space by a deterministic Turing machine.

To see this, let $\hat{M}$ be a DTM which takes as input a SAT formula $\varphi$. To decide if $\varphi \in \mathrm{SAT}$ (i.e., $\varphi$ is satisfiable), we can use linear space and simply test all possible assignments. Suppose that $\varphi$ has $m$ variables $x_1,\dots ,x_m$.

$\hat{M}(\varphi )$:

• For all $u_1,\dots ,u_m\in \{0,1\}$
  • Test if $\varphi (u_1,\dots ,u_m)=1$.
  • If yes, output $1$.
• Output $0$.

Note we are only keeping track of $m$ bits (i.e., we are counting from $0$ to $2^m-1$ in binary) and testing if this produces a satisfying assignment. Testing if $\varphi$ is satisfiable under assignment $x_i=u_i$ requires $O(n)$ space, where $|\varphi |=n$, and clearly $m\le n$. Therefore, we only need linear space to decide if $\varphi$ is satisfiable. Notably, this is an exponential time algorithm, but only uses linear space.

Space Constructible Functions

Like with time complexity, for space complexity we care about the notion of space constructible functions. As you may guess, these are functions that are constructible in limited space.

Definition. A function $S:\mathbb{N}\to \mathbb{N}$ such that $S(n)\ge \log (n)$ is space constructible if there exists a deterministic Turing machine $M$ such that on input $x\in \{0,1{\}}^{\ast }$, $M(x)$ outputs $S(|x|)$ using at most $O(S(|x|))$ space on any of its work tapes.

Theorem. For every space constructible function $S$, we have $$\mathbf{DTIME}(S)\subseteq \mathbf{DSPACE}(S)\subseteq \mathbf{NSPACE}(S)\subseteq \mathbf{DTIME}(2^{O(S)}).$$

Proof. Clearly $\mathbf{DTIME}(S)\subseteq \mathbf{DSPACE}(S)$ since any deterministic Turing machine running in time $O(S)$ can use at most $O(S)$ space on its work tapes. Similarly, $\mathbf{DSPACE}(S)\subseteq \mathbf{NSPACE}(S)$ since every deterministic computation is also a non-deterministic computation.

What remains to show is $\mathbf{NSPACE}(S)\subseteq \mathbf{DTIME}(2^{O(S)})$. To see this, we use the notion of a configuration graph. We let $G_{M,x}$ denote the configuration graph of $M(x)$ for any NTM $M$. The graph is a directed acyclic graph which consists of all possible configurations of the tape(s) of $M(x)$. There is a unique starting node, which corresponds to the unique starting configuration of every Turing machine. There is an edge from node $u$ to $v$ in the graph if the configuration $v$ can be reached by a valid transition from the configuration $u$ in a single step of the computation.

The graph $G_{M,x}$ is said to be accepting if there exists a path from the starting configuration to some halting configuration that is accepting. Moreover, if such a path exists, then $M(x)$ accepts. One final thing we need is the following fact.

Fact. Any NTM $M$ running in time $O(T)$ can be converted to an equivalent NTM $M^{\prime }$ where the transition function of $M^{\prime }$ outputs at most two states per computation step and $M^{\prime }$ runs in time $O(T)$.

By definition, we know that the non-deterministic Turing machine transition function is defined as $$\delta :Q\times {\Gamma }^k\to \mathcal{P}(Q\times {\Gamma }^{k-1}\times \{L,R,S{\}}^k).$$ By definition, for a fixed Turing machine $M$, all of $Q$, $\Gamma$, and $k$ are fixed. So $|\mathcal{P}(Q\times {\Gamma }^{k-1}\times \{L,R,S{\}}^k)|=\Theta (1)$. Given this, there is a way to convert the machine with transition function $\delta$ to an equivalent machine with transition function ${\delta }^{\prime }$ where ${\delta }^{\prime }$ outputs at most $2$ configurations per input.

All together, this allows us to notice the following properties about the configuration graph $G_{M,x}$. If $M$ uses $O(S)$ space on any input, then

1. $G_{M,x}$ has at most $2^{O(S)}$ vertices. In particular, every vertex/configuration can be encoded with $O(S)$ space (or $O(k\cdot S)$ for a $k$-tape machine, but $k$ is considered constant).
2. There exists a CNF $\psi$ of size $O(S)$ such that $\psi (C,C^{\prime })=1$ if and only if $C$ and $C^{\prime }$ are valid configuration sand $C^{\prime }\in \delta (C)$; that is, $C^{\prime }$ is reachable from $C$ in a single step of the computation.

(1) above follows from the above Fact and subsequent discussion. (2) follows directly from the Cook-Levin theorem. In particular, the formula $\psi$ is simply the formula which is checking that all computation windows are valid, and all symbols are valid.

With all of this setup, let $M$ be an NTM which decides some language $L\in \mathbf{NSPACE}(S)$. We construct a machine $M^{\prime }$ which decides $L$ in $2^{O(S)}$ time. On input $x$, the machine $M^{\prime }$ construct the graph $G_{M,x}$, which takes $2^{O(S)}$ time. Construction of this graph requires checking edges between nodes, which is done in $O(S)$ time per pair of vertices since $\psi$ has size $O(S)$. This gives a total time of $(2^{O(S)}{)}^2=2^{O(S)}$. Then $M^{\prime }$ simply searches for a path from the starting configuration to an accepting configuration, again taking at most $2^{O(S)}$ time. $\square$

Space Hierarchies

Unsurprisingly, just like with time constructible functions, we have similar space hierarchy theorems for space constructible functions.

Theorem. Let $f,g$ be space constructible functions such that $f=o(g)$. Then $\mathbf{DSPACE}(f)⊊\mathbf{DSPACE}(g)$.

The proof is identical to the deterministic time hierarchy theorem, except we do not have the logarithmic loss. This is because of universal simulation: when simulating a time $T$ Turing machine, we need $O(T\log (T))$ time, but to simulate a space $S$ Turing machine, we only need $O(S)$ space.

Space Complexity Classes

Just like with time complexity, we have complexity classes related to space bounded computations. Four of them will be of interest to us.

1. $\mathbf{PSPACE}={\bigcup }_{c\in \mathbb{N}}\mathbf{DSPACE}(n^c)$.
   • This is the space analogue of $\mathbf{P}$.
2. $\mathbf{NPSPACE}={\bigcup }_{c\in \mathbb{N}}\mathbf{NSPACE}(n^c)$.
   • This is the space analogue of $\mathbf{NP}$.
3. $\mathbf{L}=\mathbf{DSPACE}(\log (n))$.
4. $\mathbf{NL}=\mathbf{NSPACE}(\log (n))$.

Some facts about the above classes.

• It is an open question of $\mathbf{L}=\mathbf{NL}$. We believe it to not be true; that is, $\mathbf{L}\ne \mathbf{NL}$.
• In a somewhat surprising result, $\mathbf{PSPACE}=\mathbf{NPSPACE}$.
• In an even more surprising result, $\mathbf{NL}=\mathbf{coNL}$.

We’ll get to the bottom two points in our later discussions on space complexity. Let’s see some examples now.

Space Complexity Class Examples

$\mathrm{SAT}\in \mathbf{PSPACE}$

This implies that $\mathbf{NP}\subseteq \mathbf{PSPACE}$. We can see this since for any language $L\in \mathbf{NP}$, under the certificate definition there exists a DTM $M$ and a polynomial $p$ such that for all $x$, we have $x\in L$ if and only if there exists a string $w\in \{0,1{\}}^{p(|x|)}$ such that $M(x,w)=1$. We can easily construct a polynomial space DTM $M^{\prime }$ which on input $x$ simply iterates over all possible strings $w$ and checks if $M(x,w)=1$. This only uses polynomial space since we can reuse the space for the string $w$.

Languages in $\mathbf{L}$

Checking if a string has an even number of $1$’s is in $\mathbf{L}$, and so is checking if the product of two integers is equal to a third integer. Formally, let $$\begin{array}{c}\mathrm{EVEN}=\{x\in \{0,1{\}}^{\ast }:x\text{ has an even number of 1’s}\};\\ \mathrm{MULT}=\{(⟨x{⟩}_2,⟨y{⟩}_2,⟨z{⟩}_2):x,y,z\in \mathbb{N}\wedge x\cdot y=z\}.\end{array}$$

The language $\mathrm{EVEN}$ is decidable in only logarithmic space because if $n=|x|$ is the size of the input, counting the number of $1$’s in $x$ only takes $O(\log (n))$ bits, then checking if this is even simply requires checking if the least significant bit is $0$. This only takes logarithmic space.

For $\mathrm{MULT}$, we are given the binary representation of three integers $x$, $y$, and $z$. Assume each integer is $n$ bits. So the input is of length $3n$. Then, to multiply $x$ and $y$, one can simply do the grade school multiplication algorithm over the binary representations of $x$ and $y$. If we are not careful and try to write down the result of $x\cdot y$, we would require $n$-bits since $z$ is also $n$ bits. So we can simply compute $x\cdot y$ bit by bit and compare to the bits of $z$. We will have to track carry over bits, which will be at most $O(\log (n))$ bits to track.

Languages in $\mathbf{NL}$

Deciding if a directed graph has a path between two vertices $s$ and $t$ is in $\mathbf{NL}$. Formally, let $\mathrm{PATH}$ be the following language. $$\mathrm{PATH}=\{(G,s,t):G\text{ is a directed graph with a path from s to t}\}$$ The number of vertices in $G$ is $n$ and the number of edges is at most $n^2$. To see that $\mathrm{PATH}\in \mathbf{NL}$, we give a non-deterministic Turing machine which decides $\mathrm{PATH}$ using $O(\log (n))$ space. Notice that if there exists a path from $s$ to $t$ in $G$, then it is of length at most $n$. Thus, a non-deterministic Turing machine simply performs a depth first search of depth at most $n$ starting at $s$. If it finds $t$, then it outputs accept; otherwise it outputs reject. If there is a path from $s$ to $t$, then there exists a series of non-deterministic choices to make for the depth first search that will end up at $t$.

$\mathbf{PSPACE}$-complete Languages

We continue our study of space complexity by examining $\mathbf{PSPACE}$-complete languages. Just like with $\mathbf{NP}$-complete languages, we define $\mathbf{PSPACE}$-completeness with respect to polynomial time reductions.

Definition. A language $L$ is $\mathbf{PSPACE}$-complete if

1. $L\in \mathbf{PSPACE}$; and
2. $\forall L^{\prime }\in \mathbf{PSPACE}$, we have $L^{\prime }{\le }_{p}L$.

True Quantified Boolean Formulas

Up to now, we have only seen “fixed” Boolean formulas. For example, $\varphi =(x_1\vee x_2)\wedge ({\overline{x}}_3\vee x_2)$. Then, we say that $\varphi \in \mathrm{SAT}$ if and only if $\exists x_1,x_2,x_3\in \{0,1\}$ such that $\varphi (x_1,x_2,x_3)=1$.

We can generalize the above to include different quantifiers. For example:

• $\exists x_1\forall x_2,x_3\varphi (x_1,x_2,x_3)=1$;
• $\forall x_1,x_2,x_3\varphi (x_1,x_2,x_3)=1$ (notice this is the language $\mathrm{TAUTOLOGY}$, the $\mathbf{coNP}$-complete language);
• $\forall x_1\exists x_2\forall x_3\varphi (x_1,x_2,x_3)=1$.

With this, we can define quantified Boolean formulas.

Definition. Let $\varphi$ be a Boolean formula with $m$ variables. Then, we say that $\phi =Q_1{x}_1{Q}_2{x}_2\cdots Q_m{x}_m\varphi (x_1,\dots ,x_m)$ is a quantified Boolean formula, where $Q_i\in \{\exists ,\forall \}$ and $x_i\in \{0,1\}$ for all $i\in [m]$. We say that $\phi$ is a true quantified Boolean formula if $Q_1{x}_1{Q}_2{x}_2\cdots Q_m{x}_m\varphi (x_1,\dots ,x_m)=1$.

Building on what we have, we let $\mathrm{TQBF}$ be the set of all true quantified Boolean formulas $\psi$.

Theorem. $\mathrm{TQBF}$ is $\mathbf{PSPACE}$-complete.

Aside: the Essence of $\mathbf{PSPACE}$

One can actually think of $\mathrm{TQBF}$ as the set of two-player games with perfect information such that player 1 has a winning strategy. In particular, the essence of $\mathbf{PSPACE}$ turns out to be finding optimal strategies for player 1 in a 2 player game with perfect information (that is, no randomness and no hidden information such as a hand in card games). Three concrete examples: Chess, Go, and Tic-Tac-Toe (though for Tic-Tac-Toe you have to phrase it differently since drawing the game is the optimal strategy). Some other subtleties (which we will not get into in this class) that arise here are, for example, finite boards being easy to deal with in $\mathrm{PSPACE}$, which need to be fixed by making the board a 2D infinite grid.

Proving $\mathrm{TQBF}$ is $\mathbf{PSPACE}$-complete

We’ll begin the proof of this theorem, and finish it in next lecture. For now, we’ll show that $\mathrm{TQBF}\in \mathbf{PSPACE}$. Let $\psi$ be a QBF on $m$ variables $x_1,\dots ,x_m$. We let $|\psi |=n$. We construct a decider $M$ to check if $\psi =1$, with the goal being this decider uses at most $\mathrm{poly}(n)$ space for some polynomial in $n$.

Let $M$ be a simple recursive Turing machine which does the following.

$M(\psi )$:

1. If $\psi$ has no quantifiers, output $1$ if $\psi =1$ and output $0$ otherwise.
2. Else if $\psi =\exists x\in \{0,1\}\varphi$¹
   • Run $M(\varphi )$ with $x=0$ and $x=1$.
   • If $M(\varphi )$ returns $1$ on either of these inputs, return $1$. Else return $0$.
3. Else if $\psi =\forall x\in \{0,1\}\varphi$
   • Run $M(\varphi )$ with $x=0$ and $x=1$.
   • Return $1$ if and only if $M(\varphi )$ returns $1$ on both these inputs; otherwise return $0$.

First, clearly $M(\psi )$ decides if $\psi =1$. In the base case, if $\psi$ has all variable set to a value, then $M$ simply checks if $\psi$ is satisfied by the generated assignment. Then, if $\psi =\exists x\varphi$, $M$ returns $1$ if and only if assigning $x=0$ or $x=1$ returns true, which is enough since we only need at least one of these to return true. Finally, if $\psi =\forall x\varphi$, then $M$ returns $1$ if and only if assigning $x=0$ and $x=1$ both result in a true QBF.

To finish the proof, note we already established we only need $O(m)$ space in the base case of the recursion. The recursion depth is $m$, and at every level we are only storing a single variable worth of information, so the final space complexity is $O(m)$. This finishes the first part of the proof; namely, we have shown $\mathbf{TQBF}\in \mathbf{PSPACE}$.

1. This is another QBF with 0 or more quantifiers. ↩