calculus
by William Shoaff with lots of help
You can download a postscript version of this file (which is prettier) at
The study of algorithms and the resources (time and space) they use. Also, the design of algorithms (data structures, methods, recurring patterns).
From Knuth [2]:
There are many ways to compute; they are all equivalent in some sense.
calculus
For the analysis of algorithms, the RAM model is most often employed. We need to model to discuss the resources consumed or used in the process of executing an algorithm. Note time does not appear to be reusable, but space often is.
We will use the uniform cost model of time and space:
There are Other considerations:
bits to represent natural number
n in binary notation. Thus the uniform model of time and space
may not bias results for large integers (data).
An algorithm solves an instance of a problem. There is, in general, one parameter, the input size, denoted by n, which is used to characterize the problem instance. The input size n is the number of registers needed to hold input (data segment size).
Given n, we'd like to find:
Note that T(n) and S(n) are relations rather than functions. That is, for different input of of the same size n T(n) and S(n) may provide different answers.
Complexities usually not measured exactly: big-
,
and 
notation is used.
This is the longest time (or most space) that the algorithm will use
over all instances of size n.
Often this can be represented by a function f(n) such as f(n)=n2or
.
We write
This is the shortest time (or least space) that the algorithm will use
over all instances of size n.
Often this can be represented by a function f(n) such as f(n)=n2or
.
We write
When the worst and best case performance of an algorithm are the same
we can write
.
Roughly, this says the algorithm
always uses f(n) operations on all instances of size n.
This is the average time (or space) that the algorithm will use over all instances of size n. It depends on the probability distribution of instances of the problem. The average case is very interesting, but we'll delay a full discussion.
This is used when a sequence of operations occur, e.g., inserts and deletes in a tree, where the costs vary depending on the operations and their order. For example, some may take a few steps, some many. The amortized cost is very interesting, but we'll most likely not cover it.
We will mostly be concerned with off-line algorithms where all of the input is known before the algorithm starts.
Collections of problems that required roughly the same amount of resources form complexity classes. Here's a list of the most important.
There are lots of other complexity classes. All the problems we will study will have algorithms that belong to the time-based class P. For space-based problems we'd like the algorithms to belong to C or L. Using linear space or more space is generally considered inefficient.
Often there are large collections of problems that can be solved using the same general techniques or paradigms. A few of the most common are described below.
A straightforward approach to solving a problem based on the problem statement and concepts involved. Brute force algorithms are rarely efficient. Example algorithms include:
Perhaps the most famous algorithmic paradigm, divide-and-conquer is based on partitioning the problem into two or more smaller sub-problems, solving them (using recursion, or if they are simple enough, directly), and combining the sub-problem solutions into a solution for the original problem. Example algorithms include:
Greedy algorithms always make the choice that seems best at the moment. This locally optimal choice is made with the hope that it leads to a globally optimal solution. Some greedy algorithms may not be guaranteed to always produce an optimal solution.
Greedy algorithms are often applied to combinatorial optimization problems.
Richard Bellman [1] is credited for developing dynamic programming. A nutshell definition of dynamic programming is difficult, but to summarize, problems which lend themselves to a dynamic programming attack have the following characteristics:
Dynamic programming algorithms have the following features:
Local search is also applied to combinatorial optimization problems. A local search algorithm starts with some initial solution and iteratively searchs a neighborhood for a better solution. Some general classes of local search methods are:
To analyze algorithms we will need to learn how to count using summataions and recurrence relations. Basic sums and series will be studied next.
operations for some constant c?
public change coinChanger(int n) {
int quarters = n/25;
n %= 25;
int dimes = n/10;
n %= 10;
int nickels = n/5;
int pennies = n % 5;
return (quarters, dimes, nickels, pennies);
}
public double innerProduct(Vector u, Vector v) {
int n = u.length();
double ip = 0.0;
for (int i=0; i $<$ n; i++) { ip += u[i] * v[i]; }
return ip;
}
public double deCasteljau(int n, double[] vertices, float t) {
double temp[] = new double[n];
double oneMinusT = 1.0 - t;
for (int i = 0; i $<$ n; i++) temp[i]=vertices[i]; /* save input */
for (int r = 1; r $<$ n; r++)
for (int i = 0; i $<$= n - r; i++) {
temp[i]= oneMinusT * temp[i] + t * temp[i+1] ;
}
return (temp[0]);
}