This page is under construction. Please excuse our dust.


Welcome to CSE 4083/5083 Formal Languages and Automata Theory, Spring 1998. (Last update: 4-21-99)

The syllabus for this course is available electronically.

Some supplementary material available on the web.

Module One: Review (Text: Chapters 1 and 2)

This course is about mathematical models of computers and algorithms. The purpose of this module is to review concepts and techniques for the study of theoretical computer science, in particular, of formal languages and machines. We also offer a bit of historical background, emphasizing the long history of humans thinking about languages and machines. For another perspective, try this introduction .

The mathematical study of computation begins by understanding the mathematics of strings of symbols. An alphabet is a finite string of symbols. A string over an alphabet is a finite sequence of symbols from the alphabet. To write a string, we simply juxtapose the symbols; thus, 0001001 is a string over the alphabet {0,1}. A string with no symbols is called the empty string and is denoted by the greek symbol lambda. The set of all finte strings over an alphabet E is denoted by E*. The length of a string is the number of symbols it has. We can define other operations on strings like concatenation and reversal.

A central issue in the theory of computation is the representation of languages by finite specifications. The first example we use is regular expressions . Regular languages are sets of strings that can be expressed as regular expressions.

By the end of this module, the student should be refreshed in the knowledge of the following:

These are the problems the student should be able to solve at the end of this module:

Answer to homework 1 in postscript format. .

Answer to homework 2 in postscript format. .

Module 2: Context-Free Grammars, Languages, and Parsing

Think of yourself as a language processor. There are two parts of processing a language: recognition , telling whether something is a legal sentence of the language, and generation , producing legal structures of the language. A language generater must "start" somewhere, then follow some "rules" that in a finte amount of time produces a "string". The sequence of applications of some rule to produce a string is called a derivation.

Regular expressions can be viewed as language generators; they describe the rules for producing strings using the operations "select", "repeat" and "juxtapose". In this module we shall study a more complex type of language generator, called context-free grammars . Computer programs written in any programming language must satisfy rigid criteria to be syntactically correct and therefore amenable to interpretation by a machine. Context-free grammars are used extensively in modeling the syntax of programming languages. A compiler for a programming language must contain a parser, an algorithm to determine whether a given string is in the language of the grammar, and, if so, to construct the parse tree of the string. A parse tree is a tree, rooted at a node labeled with the start symbol, and whose leaves spell out a string of a language defined by the grammar that produced the tree.

A parser is top-down if the parse tree for the string is constructed top-down, i.e., from the root down. Alternatively, if the parse tree is constructed from the leaves to the root, it is said to be bottom-up . There is no one best way to parse context-free languages, and different methods are sometimes preferable for different grammars.

Context-free grammars can be parsed efficiently. However, to be efficiently parsed, they should be in an efficient format. In Chapter 5, we study ways of transforming an arbitrary context-free grammar into an equivalent one (i.e., one that generates the same language) that can be effectively parsed.

By the end of this module, the student should know the following:

For an interesting use of context-free grammars, check out this biology application. .

These are some problems the student should be able to solve at the end of this module:

Study guide for exam 1.

Answer to homework 3 in postscript format. .

Answer to homework 4 in postscript format. .

Answer guide to exam 1 (graduate students). .

Answer guide to exam 1 (undergraduate students). .

Module 3: Finite Automata and Languages

In this module and the next, we define increasingly powerful models of computation, more and more sophisticated devices for accepting and generating languages. First, we study a very restricted model of computation called a finite automaton . What makes it such a restricted model is that it has no memory outside its fixed central processor. Strings are fed into the device by means of an input tape , divided into squares, with one symbol inscribed in each square. The finite control can be, at one moment, in one of a finite number of states . The control can sense what symbols are on the tape by means of a movable reading head . Initially, the reading head is placed at the left-most square of the tape, and the control is set in a designated initial state . At regular intervals the automaton reads a symbol from the tape and enters a new state. A deterministic finite automaton (dfa) is one in which the state entered is determined only by the current state and symbol just read. After reading one symbol the head moves to the right one square. This process is repeated until the whole string is read. The automaton indicates its approval or disapproval of the string in terms of the state at the end: if the final state it's in is an accepting state , the string is accepted; otherwise it is rejected.

Adding non-determinism to this computational model means that the automaton is allowed to change its states in a way that is only partly determined. It may choose to go into any one of a set of legal states. Non-deterministic finite automata (nfa) are not realistic computational models, but they give us a simpler way to describe acceptors. In fact, a somewhat remarkable result we will study is that non-determinism is an inessential feature of computation, in the sense that every nfa is equivalent to a dfa.

We have a variety of powerful techniques to show that a language is regular. We also have a technique to show that languages are not regular. This technique is based on a property that all regular languages have, viz., that every string of a regular language that is larger than a certain size must have a part that can be repeated an arbitrary number of times, with the result of this repetition, or pumping , also being a string in a language. This corresponds intuitively to the fact that to accept a string of this size, a path through a state diagram of a dfa or nfa must contain a cycle. Repeating the same cycle any number of times cannot turn a string of a language into one that is not. The Pumping Lemma states that all regular languages have this property. To prove that a language is not regular, therefore, we can show that it does not have this property.

The theory of finite automata is rich and elegant, and understanding them puts us in a better position to understand the effects of adding more memory. Another reason for studying finite automata is their applicability to the design of several common types of computer algorithms and programs, such as a lexical analyzer of a compiler.

Not every context-free language can be recognized by finite automaton, since some context-free languages are not regular. Recall, for example, the non-regular language {ww^R : w in {a,b}}. It would seem that to recognize this language, a device must be added to a finite automaton that would "remember" what it scanned in the first part of the string to check that the second half has the right pattern. The required storage device need not be a general-purpose one; a simple stack, or "pushdown store" would be enough. The idea of an automaton with a stack as auxilliary storage is formalized in the notion of a pushdown automaton , or PDAs. A PDA does what a NFA does (scans symbols and changes state) and also "pushes" and "pops" things from the stack. An NFA can be viewed trivially as a PDA that never operates on its stack; in this sense, PDAs are more general.

In this module we will show that PDAs are exactly what we need to accept arbitrary context-free languages. This fact is of mathematical and practical importance: mathematical because it connects together two views of the same class of languages; and practical because it lays the foundation for the study of syntax analysers, like Bison for "real" context-free languages such as programming languages.

Infinite context=free languages display periodicity in more subtle ways than do regular languages. To understand this periodicity we examine the structure of derivation trees of derivations from context-free grammars. This structure allows us to formulate a pumping lemma for context-free languages. The pumping lemma gives us a proof technique for showing that languages are not context-free. Finally, we look at closure properties of context-free languages, which give us a proof technique for showing that langauges are context-free.

By the end of this module, the student should know the following:

These are some problems the student should focus on during this module:

Answer to homework 5 in postscript format. .

Answer to homework 6 in postscript format. .

Answer to homework 7 in postscript format. .

Study guide for exam 2.

Exam 2 answer sheet (grads).

Exam 2 answer sheet (undergrads).

Module 4: Turing Machines and Decidability

Alan Turing introduced the notion of an abstract machine. Since then, much work has been conducted identifying, refining and classifying computing machines in terms of their expressive power. The most general class of machine is the Turing Machine . Although Turing Machines are more general than the machines we've looked at thus far, their appearence is similar to those machines. A "standard" Turing machine consists of a finite control, a tape, and a head that can move in two directions and can be used for reading or writing on the tape. A deterministic Turing machine is one whose transition function is a partial function of state/input pairs.

Turing machines are not simply one more class of automata. As primitive as they seem to be, no attempt to strengthen them, e.g. by adding more tapes, has, to this point, been successful. We show results like this by a simulation method: we simulate the execution of these more exotic versions on the standard Turing machine defined in the book. So Turing Machines seem to form a stable and maximal class of computational devices, in terms of the computations they can perform. In fact, we will examine the widely held view that any reasonable way to formalize the notion of "algorithm", one central idea in computer science, must be ultimately equivalent to the idea of a Turing Machine. (By the way, did you know that the word "algorithm" is a corruption of the last part of of the name of the 9th century Arabic mathematician Abu Abdullah abu Jafar Muhammad ibn Musa al-Khowarizmi?)

Like the other machines in this course, a Turing machine can be used to recognize (accept strings in) languages. In the standard definition of a Turing machine used in the class, strings of a language are accepted if the machine halts in a final state. The languages accepted by Turing machines are called recursively enumerable . A Turing machine, unlike the other classes of machine we have studied, always has the option of evading a yes-no answer to the question: is a string w in L by failing to halt . If there is a Turing machine that accepts a language L and halts on all inputs, then the language is in the class of languages called recursive .

There are many extensions to the standard Turing machine. Some involve alternative ways to define accepting conditions for strings. Others involve adding features to the "hardware", such as multiple heads, multiple tracks, or a tape that is infinite in both directions. None of these add to the classes of languages accepted by machines. This is shown by simulating the operations of these extensions on the standard machine. We also define a non-deterministic version of the Turing machine, and look at Turing machines that do other things than accept languages, like enumerate them.

In this course we addressed the question: what can be computed? Turing machines can carry out remarkably complex computation tasks. This suggests that with Turing machines we have reached a natural upper limit on what a computational device can do, that Turing machines (specifically, the ones that halt on all inputs) provide an accurate formal definition of algorithm . In other words, nothing will be considered an algorithm if it can't be represented as a Turing machine. This principle is known as the Church-Turing thesis . It is a thesis, not a theorem, because it can't be proved mathematically.

In previous chapters we used pumping lemmas to prove that languages are not regular or context-free, then defined a more powerful class of machine that accepted them. If we prove that a language is not recursively enumerable, and the Church-Turing thesis is correct, then we will be "stuck", in the sense that there won't be any more powerful machines we can define to bail us out. Computational tasks, like accepting a language, that can't be solved by Turing machines will therefore be called undecidable.

Exposing the limitations of Turing machines is a challenging task, since they're so powerful. The aspect of Turing machines that will expose their limitations is a sort of introspective ability: we can use them to simulate the actions of other Turing machines. Then we ask what happens when a Turing machine simulates (an encoding of) itself! We generate an impossible situation, which suffices to expose a limitation of computing devices. See if you can see the relationship between this argument, called the Halting problem, and Cantor's Diagonalization argument. The Halting problem is the following decision problem: given a Turing machine M and input w, does M halt on w? No algorithm exists for solving this problem in general (this does not mean that in specific cases, it might be possible to answer this question). Once the Halting problem is shown to be unsolvable, a great many other problems can be shown unsolvable by a proof technique known as reduction. But this is a subject of another course.

By the end of this module, the student should know the following:

These are the problems the student should be able to solve at the end of this module:

Answer to homework 8 in postscript format. .

Answer to homework 9 in postscript format. .

Study guide for final exam. .