Free University of Bolzano/Bozen
Faculty of Computer Science - Master of Science in Computer Science

Theory of Computing

Lectures A.A. 2004/2005

Prof. Diego Calvanese

Teaching material

[M1] Introduction to Automata Theory, Languages, and Computation (2nd edition). J.E. Hopcroft, R. Motwani, J.D. Ullman. Addison Wesley, 2003.

[M2] Lecture Notes for Theory of Computing. Diego Calvanese. 2005. Available as scanned pages in pdf.

[M3] Exercises on Theory of Computing. Available as scanned pages in pdf.

Lectures

Summary

Week	Topics	Monday 10:30-12:30 (lecture)	Wednesday 10:30-12:30 (lecture)	Wednesday 14:00-16:00 (exercise)	Extra (lecture)
#1 Oct. 5	Course introduction	--	Course introduction, basic notions about languages (Lec 1,2)	Formal proofs (Exer 0)
#2 Oct. 10	Finite automata	Deterministic FA (Lec 3,4)	Nondeterministic FA (Lec 5,6)	DFAs and NFAs (Exer 1)	Epsilon-NFAs (Lec 7,8)
#3 Oct. 17	Regular expressions	Regular expressions, from REs to FAs (Lec 9,10)	From DFAs to REs, closure properties for RLs (Lec 11,12)	Epsilon-NFAs and RE (Exer 2)
#4 Oct. 24	Regular languages	Pumping lemma (Lec 13,14)	--	--
#5 Oct. 31	Properties of regular languages	--	Decision properties for RLs, NFA minimization (Lec 15,16)	Regular languages (Exer 3)
#6 Nov. 7	Formal grammars	Formal grammars (Lec 17,18)	Formal grammars (Lec 19,20)	Regular languages (Exer 4)
#6 (extra) Nov. 7	Context free grammars	--	Context free grammars (Lec 21,22)	Formal grammars (Exer 5)
#7 Nov. 14	Properties of context free grammars	Simplifying CFGs, Chomsky normal form (Lec 23,24)	Topics for midterm exam (Exer 6)	Midterm exam (ME)
#8 Nov. 28	Decision problems for CFLs	Pumping lemma and closure properties for CFLs (Lec 25,26)	--	--
#9 Dec. 5	Properties of CFLs	Decision problems for CFLs (Lec 27,28)	Undecidable problems (Lec 29,30)	Context-free grammars and closure properties of CFLs (Exer 7)
#10 Dec. 12	Turing Machines	The Turing Machine (Lec 31,32)	Programming techniques for TMs, multi-tape TMs (Lec 33,34)	Closure properties of CFLs and TMs (Exer 8)
#11 Dec. 19	Decidability and undecidability	Nondeterministic TMs (Lec 35,36)	Closure properties of R.E. languages (Lec 37,38)	Nondeterministic TMs and extensions of TMs (Exer 9)
#12 Jan. 9	Decidability and undecidability	Showing languages to be non-recursive/non-R.E. (Lec 39,40)	Reductions, Rice's theorem (Lec 41,42)	Recursive and R.E. languages (Exer 10)	Rice's theorem, (in)tractable problems (Lec 43,44)
#13 Jan. 16	Computational complexity	NP-completeness (Lec 45,46)	Complexity classes above NP (Lec 47,48)	NP-complete problems (Exer 11)
#14 Jan. 23	Complexity classes	--	Exam topics (Exer 12)	--

Details

Week 1: Course introduction

Lectures 1,2 - 5/10/2005

Topics [M2: Part 1]
- course presentation
- basic definitions about languages
What you should know after the lecture
- the formal meaning of alphabet, string, language

Exercise 0 - 5/10/2005

Review of basic proof techniques [M2: Part 0]
- deductive proofs
- proving equivalences of sets
- proof by contradiction
- proof by induction

Week 2: Finite automata

Lectures 3,4 - 10/10/2005

Topics [M2: Part 2]
- finite state machines
- deterministic finite automata (DFA)
- extended transition function for a DFA
- nondeterministic finite automata (NFA)
What you should know after the lecture
- how a finite-state machine works (informally)
- the formal definitions of a DFA and of an NFA, and of the language they accept
- construct a DFA and an NFA for a given language

Lectures 5,6 - 12/10/2005

Topics [M2: Part 2]
- relationship between DFAs and NFAs (subset construction)
What you should know after the lecture
- how to prove that DFAs and NFAs accept the same class of languages
- how to contstruct a DFA from a given NFA

Exercise 1 - 12/10/2005

Exercises on finite automata [M3: Exercise 1]
- [M1]: Exercises 2.2.2, 2.2.5, 2.2.8, 2.3.1

Lectures 7,8 - 14/10/2005

Topics [M2: Part 2]
- finite automata with epsilon-transitions
- eliminating epsilon-transitions
What you should know after the lecture
- how to prove that epsilon-NFAs and NFAs accept the same class of languages
- how to construct an NFA from a given epsilon-NFA

Week 3: Regular expressions

Lectures 9,10 - 17/10/2005

Topics [M2: Part 3]
- regular expressions
- algebraic laws for regular expressions
- converting a regular expression to a finite automaton
What you should know after the lecture
- what the language generated by a regular expression is
- what algebraic laws hold for regular expressions
- how to prove that each language generated by a R.E. is accepted by some finite automaton
- how to construct a finite automaton from a given R.E.

Lectures 11,12 - 19/10/2005

Topics [M2: Part 3, Part 4]
- converting a finite automaton to a regular expression
- closure properties of regular expressions
What you should know after the lecture
- how to prove that finite automata accept exactly the regular languages
- how to construct a R.E. from a given automaton
- that regular languages are closed under regular and boolean operations

Exercise 2 - 19/10/2005

Exercises on finite automata and regular expressions [M3: Exercise 2]

Week 4: Regular languages

Lectures 13,14 - 24/10/2005

Topics [M2: Part 4]
- pumping lemma for regular languages
- proving languages not to be regular
What you should know after the lecture
- how to prove that a languages is not regular
- what properties can be decided about regular languages

Week 5: Properties of regular languages

Lectures 15,16 - 2/11/2005

Topics [M2: Part 4]
- decision problems for regular languages
- minimization of finite automata
What you should know after the lecture
- what properties about regular languages can be decided, and what is the computational cost
- how to minimize a finite automaton

Exercise 3 - 2/11/2005

Exercises on regular languages [M3: Exercise 3: part a, part b]

Week 6: Formal grammars

Lectures 17,18 - 7/11/2005

Topics [M2: Part 5]
- Chomsky formal grammars
- derivation
What you should know after the lecture
- what a Chomsky formal grammar is
- what the language defined by a formal grammar is

Lectures 19,20 - 9/11/2005

Topics [M2: Part 5]
- classification of formal grammars
- grammars of type 0
- grammars of type 1 (or context-sensitive)
- grammars of type 2 (or context-free)
- grammars of type 3 (or regular)
- epsilon-productions in a grammar
What you should know after the lecture
- how formal grammars are classified
- recognize what the type of a grammar is
- effect of the presence of epsilon-production in a grammar of type 1

Exercise 4 - 9/11/2005

Exercises on regular languages and formal grammars [M3: Exercise 4]

Week 6 (extra): Context free grammars

Lectures 21,22 - 11/11/2005

Topics [M2: Part 5]
- context-free grammars
- parse-trees for context-free grammars
- ambiguous grammars
What you should know after the lecture
- how to construct the parse-tree corresponding to a derivation
- recognize when a context-free grammar is ambiguous

Exercise 5 - 11/11/2005

Exercises on formal grammars [M3: Exercise 5]

Week 7: Midterm exam

Lectures 23,24 - 14/11/2005

Topics [M2: Part 7]
- simplifying context free grammars (eliminating useless symbols, eliminating epsilon-transitions, eliminating unit-productions)
- Chomsky normal form for a CFG
What you should know after the lecture
- how to eliminate useless symbols from a CFG
- how to eliminate epsilon-productions from a CFG
- how to eliminate unit-productions from a CFG
- how to convert a CFG into Chomsky normal form

Exercise 6 - 16/11/2005

Exercises on regular languages, automata, and grammars (preparation for midterm exam) [M3: Exercise 7]

Midterm exam - 23/11/2005

Topics
- regular languages
- automata
- formal grammars

Week 8: Context free languages

Lectures 25,26 - 28/11/2005

Topics [M2: Part 7]
- pumping lemma for context-free languages
- closure properties for context-free languages
What you should know after the lecture
- how to prove that a language is not context-free by applying the pumping lemma
- under which operations contex-free languages are closed and not closed

Week 9: Properties of contex-free languages

Lectures 27,28 - 5/12/2005

Topics [M2: Part 7b]
- complexity of decision problems for context-free languages
  - conversion to Chomsky normal form
  - testing emptiness of a CFL
  - testing membership in a CFL
  - undecidable problems for CFLs
What you should know after the lecture
- which decision problems related to context-free languages can be solved by an algorithm, and what is the computational cost of the algorithm

Lectures 29,30 - 7/12/2005

Topics [M2: Part 8]
- an example for an undecidable problem: the Hello-world problem
- proving undecidability by reduction to an undecidable problem
What you should know after the lecture
- a technique to prove that a problem is undecidable
- how one can exploit an undecidable problem to show that other problems are also undecidable

Exercise 7 - 7/12/2005

Exercises on context-free grammars and closure properties of CFLs [M3: Exercise 7]

Week 10: Turing Machines

Lectures 31,32 - 12/12/2005

Topics [M2: Part 8]
- the Turing Machine
- instantaneous description of a Turing Machine
- recursive enumerable and recursive languages
What you should know after the lecture
- how a Turing Machine is formally defined
- relationship between acceptance by a TM and acceptance by a FA/PDA

Lectures 33,34 - 14/12/2005

Topics [M2: Part 8]
- programming techniques for Turing Machines
  - storage in the state
  - multiple tracks
  - subroutines and procedure calls
- multi-tape Turing Machines
What you should know after the lecture
- how one can program a TM easier by imposing structure on states and tape symbols
- how a multi-tape TM can be simulated by a single-tape TM

Exercise 8 - 14/12/2005

Exercises on closure properties of context-free languages and Turing Machines [M3: Exercise 8]

Week 11: Decidability and undecidability

Lectures 35,36 - 19/12/2005

Topics [M2: Part 8]
- running time of a Turing Machine
- nondeterministic Turing Machines
What you should know after the lecture
- how a nondeterministic TM can be simulated by a multi-tape TM (and hence also by a single-tape TM)
- the cost of simulating a nondeterministic TM by a deterministic TM

Lectures 37,38 - 21/12/2005

Topics [M2: Part 9]
- classes of languages/problems
  - recursive/decidable languages
  - recursively enumerable (R.E.) languages
  - non-R.E. languages
- closure properties of recursive and R.E. languages
What you should know after the lecture
- how languages/problems can be classified
- how to prove closure properties of recursive and R.E. languages

Exercise 9 - 21/12/2005

Exercises on nondeterministic Turing Machines and extensions of Turing Machines [M3: Exercise 09]

Week 12: Decidability and undecidability

Lectures 39,40 - 9/1/2006

Topics [M2: Part 9]
- showing languages to be nonrecursive/non-R.E.
- encoding Turing Machines as binary strings/integers
- enumerating binary strings/Turing Machines
- a non-R.E. language: the diagonalization languages
- a non-recursive language: the universal language
What you should know after the lecture
- how to encode a Turing Machine as a binary string
- how to prove that the diagonalization language is non-R.E.
- how to prove that the universal language is non-recursive

Lectures 41,42 - 11/1/2006

Topics [M2: Part 9, M2: Part 10]
- the notion of reduction between problems/languages
- Rice's theorem
What you should know after the lecture
- what a reduction is
- how to prove Rice's theorem

Exercise 10 - 11/1/2006

Exercises on recursive and R.E. languages [M3: Exercise 10]

Lectures 43,44 - 13/1/2006

Topics [M2: Part 9, M2: Part 10]
- Rice's theorem
- tractable and intractable problems
- the classes P and NP
- a problem in NP: SAT
What you should know after the lecture
- how to prove Rice's theorem
- how the classes P and NP are defined
- how to show a problem to be in NP

Week 13: Computational complexity

Lectures 45,46 - 16/1/2006

Topics [M2: Part 10]
- SAT and CSAT
- poly-time reductions
- NP-hardness and NP-completeness
- Cook's theorem
What you should know after the lecture
- how to polynomially reduce one problem to another
- sketch the proof of Cook's theorem
- how to show a problem to be NP-hard

Lectures 47,48 - 18/1/2006

Topics [M2: Part 10]
- coNP-complete problems
- oracle Turing Machines
- the polynomial hierarchy and PSPACE
- the classes EXPTIME, EXPSPACE, and the exponential hierarchy
What you should know after the lecture
- what an oracle TM is
- how the polynomial hierarchy is defined
- typical problems complete for the classes EXPTIME and EXPSPACE