The Limits of Computation

This article provides a survey of key papers that characterise computable functions, but also provides some novel insights as follows. It is argued that the power of algorithms is at least as strong as functions that can be proved to be totally computable in type-theoretic translations of subsystems of second-order Zermelo Fraenkel set theory. Moreover, it is claimed that typed systems of the lambda calculus give rise naturally to a functional interpretation of rich systems of types and to a hierarchy of ordinal recursive functionals of arbitrary type that can be reduced by substitution to natural number functions.


Secure voting
Suppose you want to make a voting machine for use in an imaginary election between two parties. 1 Let Σ = {r, d}, for no particular reason.
A string consists of a series of votes for the candidates.E.g., rrdddrd means "two people voted for r, then three people voted for d, then one more person voted for r, then one more person voted for d".
A voting machine is a program that takes as input a string of rs and ds and then reports whether person r won the election.
For simplicity, this model assumes centralized voting, e.g., done online.
Question: Given a Turing machine that someone claims is a secure voting machine, could we automatically check whether it's really a secure voting machine?

Secure voting
The secure voting problem is the following: Given a TM M, is the language of M {w ∈ Σ * | w has more rs than ds}?
Claim: This problem is not decidable; there is no algorithm that can check an arbitrary TM to verify that it's a secure voting machine!
A program that decides whether arbitrary input programs are secure voting machines is self-defeating.It therefore doesn't exist.

A decider for secure voting
Suppose that, somehow, we managed to build a decider for the secure voting problem: We could represent this in software as a procedure

Interpreting this result
This tells us that there is no general algorithm that we can follow to determine whether a program is a secure voting machine.
In other words, any general algorithm to check voting machines will always be wrong on at least one input.
The previous example might seem contrived, but it's not.This is a problem we really would like to be able to solve -but it's provably impossible!

What can we do?
Design algorithms that work in many but not all cases.This is often done in practice!Fall back on human verification of voting machines.
We do this too!Carry a healthy degree of skepticism about electronic voting machines.
We were born skeptical.

Wrapping up undecidability
We've seen a general pattern in proving undecidability (i.e., non-membership in R): Assume the language in question -usually a language about TMs -is decidable.
Build a machine that decides whether it has the property, then chooses to do something contrary to that property.
Conclude that something is terribly wrong, meaning that the original language wasn't decidable.

Two intuitions
The avoid your fate intuition: Construct a machine so that it learns its fate (i.e., decides what to do next), then actively chooses to do the opposite.
The impossible bind intuition: Background: Cantor's diagonalization method Consider the sets: ,

Infinite cardinalities
Two sets have the same size if there is a way to pair their elements off without leaving any elements uncovered.{0, 1, 2, 3, 4, 5, 6, 7, 8 What would that look like?

The diagonalization proof
No matter how we pair up elements of S and subsets of S, the complemented diagonal won't appear in the table.
In row n, the nth element must be wrong.
No matter how we pair up elements of S and subsets of S, there is always at least one subset left over.
The result is Cantor's Theorem: Every set is strictly smaller than its power set.

What does this have to do with computation?
Consider: "The set of all Turing machines (computer programs)" "The set of all languages (problems to solve)" We can use Cantor's diagonalization proof to show that there must be languages that are nonrecognizable because there are more distinct languages than there are Turing machines to recognize them.
The full proof of this is in the book.
We'll move on to using diagonalization to show that there is a specific language L D that is not recognizable.

Languages, TMs, and TM encodings
Recall: The language of a TM M is the set Some of the strings in this set might be descriptions of TMs.
What happens if we list off all Turing machines, looking at how those TMs behave given other TMs as input?All descriptions of Turing machines, listed in the same order This row is the "characteristic sequence" (or "characteristic vector") for M 0 .It specifies all the strings that are members of L(M 0 ).
We can also represent "Accept" as 1 and "Reject" as 0.
Flip all "accept"s to "no"s and vice versa.
What TM has this behavior?
"The language of all TMs that do not accept their descriptions" The diagonalization language, which we denote L D , is defined as We constructed this language to be different from the language of every Turing machine.
Therefore L D ∉ RE! Let's prove this.

Feeling incomplete
The fact that non-RE languages exist supports the claim: There are statements that are true but not provable.
Intuitively, given any non-RE language, there will be some string in the language that cannot be proven to be in the language.
Problem: Decide whether a mathematical statement is true or false.
Gödel's incompleteness theorem tells us there is no algorithm that can solve this problem.
Within any self-consistent mathematical system powerful enough to express arithmetic, there exist statements that can neither be proved true nor false.
Kurt Gödel gave a mathematical equivalent of "This sentence is not true".
In 1931 -not long before Turing introduced the Halting Problem -Gödel showed that a system of logic that is capable of generating all true statements ("complete") can be used to generate a statement P which in essence refers to itself, saying "P is false".
Mathematicians saw their job as proving or disproving statements, and Gödel's theorem proved that their "job" was -in certain instancesimpossible.

What this means
On a more philosophical note, you could interpret the previous result in the following way:

There are inherent limits about what mathematics can teach us.
There's no automatic way to do math.There are true statements we can't prove.
This doesn't mean that mathematics is worthless, but it means we need to temper our expectations of it.
Some mathematicians and philosophers have ascribed almost mystical properties to the incompleteness theorem.
A few believe it proves human intuition somehow surpasses the power of a computer -that humans may "intuit" truths that are impossible for computers to prove or disprove.This is fallacious; whether people can successfully make intuitive leaps that cannot be made by computers, the incompleteness theorem provides no reason to believe that there are mathematical statements that can be proved by a mathematician but not by a computer.
Humans cannot compute non-computable problems any more than computers can.
the natural numbers including 0 S = {n | n ∈ ℕ 0 and n is even} Is ℕ 0 bigger than S, i.e., is |ℕ 0 | > |S|?By definition, two sets have the same size if there is a way to pair their elements off without leaving any elements uncovered.By definition, two sets have the same size if there is a way to pair their elements off without leaving any elements uncovered.
Do all infinite sets have the same cardinality?