The syllabus
for this course is available electronically.
Some supplementary material
available on the web.
Module One: Review (Text: Chapters 1 and 2)
This course is about mathematical models of computers and algorithms.
The purpose of this module is to review concepts and techniques for
the study of theoretical computer science, in particular, of
formal languages and machines. We also offer a bit of historical background,
emphasizing the long history of humans thinking about languages and
machines. For another perspective, try this introduction .
The mathematical study of computation begins by understanding the mathematics
of strings of symbols. An alphabet is a finite string of symbols.
A string over an alphabet is a finite sequence of symbols from the
alphabet. To write a string, we simply juxtapose the symbols;
thus, 0001001 is a string over the alphabet {0,1}. A string with no
symbols is called the empty string and is denoted by the greek symbol lambda.
The set of all finte strings over an alphabet E is denoted by E*. The length
of a string is the number of symbols it has. We can define other operations
on strings like concatenation and reversal.
A central issue in the theory of computation is the representation of languages
by finite specifications. The first example we use is regular expressions .
Regular languages are sets of strings that can be expressed as regular expressions.
By the end of this module, the student
should be refreshed in the knowledge of the following:
These are the problems the student should be able to solve at the end of
this module:
Answer to homework 2 in postscript format. .
Module 2: Context-Free Grammars, Languages, and Parsing
Think of yourself as a language processor.
There are two parts of processing a language:
recognition , telling whether something is a legal sentence of the language, and
generation , producing legal structures of the language. A language generater
must "start" somewhere, then follow some "rules" that in a finte amount of time produces
a "string". The sequence of applications of some rule to produce a string is called a
derivation.
Regular expressions can be viewed as language generators; they describe the
rules for producing strings using the operations "select", "repeat" and "juxtapose".
In this module we shall study a more complex type of language generator, called context-free
grammars . Computer programs written in any programming language must satisfy rigid
criteria to be syntactically correct and therefore amenable to interpretation by a machine.
Context-free grammars are used extensively in modeling the syntax of programming
languages. A compiler for a programming language must contain a parser, an
algorithm to determine whether a given string is in the language of the grammar, and,
if so, to construct the parse tree of the string. A parse tree is a tree, rooted
at a node labeled with the start symbol, and whose leaves spell out a string of a language
defined by the grammar that produced the tree.
A parser is top-down if the parse tree for the string is constructed top-down,
i.e., from the root down. Alternatively, if the parse tree is constructed from the leaves
to the root, it is said to be bottom-up . There is no one best way to parse context-free
languages, and different methods are sometimes preferable for different grammars.
Context-free grammars can be parsed efficiently. However, to be efficiently parsed, they should
be in an efficient format. In Chapter 5, we study ways of transforming an arbitrary context-free
grammar into an equivalent one (i.e., one that generates the same language) that can be effectively
parsed.
By the end of this module, the student should know the following:
These are some problems the student should be able to solve at the end of
this module:
Answer to homework 3 in postscript format. .
Answer to homework 4 in postscript format. .
Answer guide to exam 1 (graduate students). .
Answer guide to exam 1 (undergraduate students). .
Module 3: Finite Automata and Languages
In this module and the next, we define increasingly powerful models of
computation, more and more sophisticated devices for accepting and
generating languages. First, we study a very restricted model of
computation called a finite automaton . What makes it such a
restricted model is that it has no memory outside its fixed central
processor. Strings are fed into the device by means of an input tape ,
divided into squares, with one symbol inscribed in each square. The finite control
can be, at one moment, in one of a finite number of states . The control can sense
what symbols are on the tape by means of a movable reading head . Initially, the
reading head is placed at the left-most square of the tape, and the control is set in a designated
initial state . At regular intervals the automaton reads a symbol from the tape and
enters a new state. A deterministic finite automaton (dfa) is one in which the state
entered is determined only by the current state and symbol just read. After reading one symbol
the head moves to the right one square. This process is repeated until the whole string is read.
The automaton indicates its approval or disapproval of the string in terms of the state at the
end: if the final state it's in is an accepting state , the string is accepted; otherwise
it is rejected.
Adding non-determinism to this computational model means that the automaton is allowed
to change its states in a way that is only partly determined. It may choose to go into any one of
a set of legal states. Non-deterministic finite automata (nfa) are not realistic computational
models, but they give us a simpler way to describe acceptors. In fact, a somewhat remarkable
result we will study is that non-determinism is an inessential feature of computation, in the
sense that every nfa is equivalent to a dfa.
We have a variety of powerful techniques to show that a language is regular. We also have a technique
to show that languages are not regular. This technique is based on a property that all regular
languages have, viz., that every string of
a regular language that is larger than a certain size must have a part that can be repeated an
arbitrary number of times, with the result of this repetition, or pumping , also being a
string in a language. This corresponds intuitively to the fact that to accept a string of this
size, a path through a state diagram of a dfa or nfa must contain a cycle. Repeating the same
cycle any number of times cannot turn a string of a language into one that is not. The
Pumping Lemma states that all regular languages have this property. To prove that a
language is not regular, therefore, we can show that it does not have this property.
The theory of finite automata is rich and elegant, and understanding
them puts us in a better position to understand the effects of adding
more memory. Another reason for studying finite automata is their
applicability to the design of several common types of computer algorithms
and programs, such as a lexical analyzer of a compiler.
Not every context-free language can be recognized by finite automaton,
since some context-free languages are not regular.
Recall, for example, the non-regular language {ww^R : w in {a,b}}.
It would seem that
to recognize this language, a device must be added to a finite automaton
that would "remember" what it scanned in the first part of the string to
check that the second half has the right pattern. The required storage
device need not be a general-purpose one; a simple stack, or "pushdown store"
would be enough. The idea of an automaton with a stack as auxilliary storage
is formalized in the notion of a pushdown automaton , or PDAs.
A PDA does what a NFA does (scans symbols and changes state) and also
"pushes" and "pops" things from the stack. An NFA can be viewed trivially
as a PDA that never operates on its stack; in this sense, PDAs are more
general.
In this module we will show that PDAs are exactly what we need to accept
arbitrary context-free languages. This fact is of mathematical and practical
importance: mathematical because it connects together two views of the same
class of languages; and practical because it lays the foundation for the
study of syntax analysers, like Bison
for "real" context-free languages such as programming languages.
Infinite context=free languages display periodicity in more subtle ways than do regular languages.
To understand this periodicity we examine the structure of derivation trees of derivations from
context-free grammars. This structure allows us to formulate a pumping lemma for context-free
languages. The pumping lemma gives us a proof technique for showing that languages are not context-free.
Finally, we look at closure properties of context-free languages, which give us a proof technique for showing
that langauges are context-free.
By the end of this module, the student should know the following:
These are some problems the student should focus on during this module:
Answer to homework 5 in postscript format. .
Answer to homework 6 in postscript format. .
Answer to homework 7 in postscript format. .
Exam 2 answer sheet (undergrads).
Module 4: Turing Machines and Decidability
Turing machines are not simply one more class of automata. As primitive as they seem to be, no attempt
to strengthen them, e.g. by adding more tapes, has, to this point, been successful. We show results like
this by a simulation method: we simulate the execution of these more exotic versions on the standard
Turing machine defined in the book. So Turing Machines seem to form a stable and maximal
class of computational devices, in terms of the computations they can perform. In fact, we will examine
the widely held view that any reasonable way to formalize the notion of "algorithm", one central idea in
computer science, must be ultimately equivalent to the idea of a Turing Machine.
(By the way, did you know that the word "algorithm" is a corruption of the last part of
of the name of the 9th century Arabic mathematician Abu Abdullah abu
Jafar Muhammad ibn Musa al-Khowarizmi?)
Like the other machines in this course, a Turing machine can be used to
recognize (accept strings in) languages. In the standard definition of a
Turing machine used in the class, strings of a language are accepted if
the machine halts in a final state. The languages accepted by Turing machines
are called recursively enumerable . A Turing machine, unlike the
other classes of machine we have studied, always has the option of evading
a yes-no answer to the question: is a string w in L by failing to
halt . If there is a Turing machine that accepts a language L and halts
on all inputs, then the language is in the class of languages called
recursive .
There are many extensions to the standard Turing machine. Some involve
alternative ways to define accepting conditions for strings. Others involve
adding features to the "hardware", such as multiple heads, multiple tracks,
or a tape that is infinite in both directions. None of these add to the
classes of languages accepted by machines. This is shown by simulating
the operations of these extensions on the standard machine. We also
define a non-deterministic version of the Turing machine, and
look at Turing machines that do other things than accept languages, like
enumerate them.
In this course we addressed the question: what can be computed? Turing machines can
carry out remarkably complex computation tasks. This suggests that with Turing machines
we have reached a natural upper limit on what a computational device can do, that
Turing machines (specifically, the ones that halt on all inputs) provide an accurate
formal definition of algorithm . In other words, nothing will be considered
an algorithm if it can't be represented as a Turing machine. This principle is known
as the Church-Turing thesis . It is a thesis, not a theorem, because it can't
be proved mathematically.
In previous chapters we used pumping lemmas to prove that languages are not regular
or context-free, then defined a more powerful class of machine that accepted them.
If we prove that a language is not recursively enumerable, and the Church-Turing
thesis is correct, then we will be "stuck", in the sense that there won't be
any more powerful machines we can define to bail us out. Computational tasks, like
accepting a language, that can't be solved by Turing machines will therefore be called
undecidable.
Exposing the limitations of Turing machines is a challenging task, since they're so
powerful. The aspect of Turing machines that will expose their limitations is a sort
of introspective ability: we can use them to simulate the actions of other
Turing machines. Then we ask what happens when a Turing machine simulates (an encoding
of) itself! We generate an impossible situation, which suffices to expose a limitation
of computing devices. See if you can see the relationship between this argument, called
the Halting problem, and Cantor's Diagonalization argument. The Halting problem
is the following decision problem: given a Turing machine M and input w, does M halt
on w? No algorithm exists for solving this problem in general (this does not mean
that in specific cases, it might be possible to answer this question). Once the Halting
problem is shown to be unsolvable, a great many other problems can be shown unsolvable
by a proof technique known as reduction. But this is a subject of another course.
By the end of this module, the student should know the following:
These are the problems the student should be able to solve at the end of
this module:
Answer to homework 8 in postscript format. .
This page is under construction. Please excuse our dust.
Welcome to CSE 4083/5083 Formal Languages and Automata Theory, Spring 1998.
(Last update: 4-21-99)
Answer to homework 1 in postscript format. .
For an interesting use of context-free grammars, check out this biology
application. .
Solution for Problem 5:
Let ui = a^ib and vi = ba^i. Then uivi is the palindrone a^ibba^i. For i <> j, the
string uivj is not in P. Therefore, by Corollary 7.2.2, P is not regular.
By the second condition, v consists solely of a's. Pumping v produces the string uv^2w
that contains at least as many a's as b's, and therefore is not in L. So, L is not
regular by the pumping lemma.
The string a^n, where n = k^2, must have a decomposition that satisfies
these three conditions. Consider the length of the string z = uv^2wx^2y
obtained by pumping uvwxy. The length of z = length of uv^2wx^2y =
length uvwxy + length of v + length of x = k^2 + length of v +
length of x, which is less than or equal to k^2 + k. This last value
is strictly less than (k+1)^2, the next perfect square after k^2. This
means that the length of z is strictly greater than k^2 and less than
(k+1)^2, which means it is not a perfect square, hence not in L. This
contradicts the pumping lemma, which means that L is not context free.
Alan Turing
introduced the notion of an abstract machine. Since
then, much work has been conducted identifying, refining and
classifying computing machines in terms of their expressive power. The
most general class of machine is the
Turing Machine .
Although Turing Machines are more general than the machines we've looked at thus far,
their appearence is similar to those machines. A "standard"
Turing machine consists of a finite control,
a tape, and a head that can move in two directions and can be used for reading or writing on
the tape. A deterministic Turing machine is one whose transition function
is a partial function of state/input pairs.