The Growth of Functions and Big-O Notation

Big-O notation Purpose: to succinctly state the growth rate of a given function f (n) = O(g (n)) i there exist constants C > 0 and k 0 such that |f (n)| C |g (n)| for every n k f (n) = (g (n)) i there exist constants C > 0 and k 0 such that |f (n)| C |g (n)| for every n k f (n) = (g (n)) i f (n) = O(g (n)) and g (n) = O(f (n)). In this case we say that f and g have the same order of growth. f (n) = o(g (n)) i f (n) = (g (n)) i
f (n) g (n) f (n) g (n)

0 as n as n

Notes: 1. In this course we will almost always be interested in nonnegative functions. In such cases, establishing, e.g. f (n) C g (n), is sucient. 2. A more succinct way to say property P (n) is true for all n > k , where k is a constant is to say property P (n) holds for suciently large n. In this course, as a rule, the student should make the eort to establish and communicate the value of k as part of the solution.

Example 1. Use the above denitions to establish the truth or falsehood of the statement f (n) = B(g (n)), where f (n) = 6n2 2n + 5, g (n) = n3 , and B {O, , , o, }.

Theorem 1. Let p(n) be a polynomial of degree a and q (n) be a polynomial of degree b. Then p(n) = O(g (n)) if and only if a b p(n) = (g (n)) if and only if a b p(n) = (g (n)) if and only if a = b p(n) = o(g (n)) if and only if a < b p(n) = (g (n)) if and only if a > b

Theorem 2. Let f (n), g (n), h(n), and k (n) be nonnegative integer functions for suciently large n. Then f (n) + g (n) = (max(f (n), g (n))) if f (n) = (h(n)) and g (n) = (k (n)), then f (n)g (n) = (h(n)k (n)) Transitivity of big-O. If f (n) = O(g (n)) and g (n) = O(h(n)) then f (n) = O(h(n)). Note: transitivity also holds for big- and big-.

Example 2. Give a succinct expression for the order of growth of f (n)g (n), where f (n) = n log(n4 + 1) + n(log n)2 and g (n) = n2 + 2n 3.

Theorem 2. Let f (x) be a monotone increasing or decreasing Riemann-integrable function over the interval [1, ). Then
n n

f (i) = (
i=1 1

f (x)dx),
n 1

if f is decreasing; and the same holds if f is increasing and f (n) = O( Proof of Theorem 2.

f (x)dx).

Example 3. Give a succinct expression for the order of growth of the sum 1 1 1 + + + . 1 2 n

Example 4. Prove that for every positive integer k and constant > 0, log k n = o(n ).

Growth O(1) O(log n) O(n) O(n log n) O(nb ) (nk ) for every k (bn ) for some b > 1

Terminology constant growth logarithmic growth linear growth log-linear growth polynomial growth superpolynomial growth exponential growth

Example 5. Using the above terminology, discuss the type of growth for the functions 1 n f (n) = nlog n and g (n) = (1 + n ) .

Exercises. Note: in the following problems, all functions are assumed nonnegative. Show all work and provide careful explanation to receive a passing mark. 1. Use the denition of big-O to prove that 3n + 2 log n = O(n). Provide the appropriate C and k constants. 2. Use the denition of big- to prove that n + n log n2 = (5n + 9n log n5 ). Provide the appropriate C and k constants. 3. Prove that f (n) = O(g (n)) if and only if g (n) = (f (n)). 4. Use the denition of big- to prove that f (n) + g (n) = (max(f (n), g (n))).
(n) 5. Prove that if limitn f = C , for some constant C > 0, then f (n) = (g (n)). g (n)

6. Prove that (n + a)b = (nb ), for all real a and b > 0.

(n) 7. Prove that if limitn f = 0, then f (n) = O(g (n)), but g (n) = O(f (n)). g (n)

8. Explain why the statement, The running time of algorithm A is at most (n log n), does not make sense. Re-write the statement so that it makes more sense. 9. Prove or disprove: 2n+1 = O(2n ). 10. Prove or disprove: 22n = O(2n ). 11. Use any techniques or results from lecture to determine a succinct big- expression for the growth of the function log50 (n)n2 + log(n4 )n2.1 + 1000n2 + 100000000n. 12. Suppose g (n) 1 for all n, and that f (n) g (n) + L, for some constant L and all n. Prove that f (n) = O(g (n)). 13. Prove or disprove: if f (n) = O(g (n)) and f (n) 1 and log(g (n)) 1 for suciently large n, then log(f (n)) = O(log(g (n))). 14. Prove or disprove: if f (n) = O(g (n)), then 2f (n) = O(2g(n) ). 15. Prove transitivity of big-O: if f (n) = O(g (n)), then g (n) = O(h(n)), then f (n) = O(h(n)). 16. If g (n) = o(f (n)), then prove that f (n) + g (n) = (f (n)). 17. Given functions f (n) = n and g (n) = n sin(n), explain why f (n) = O(g (n)), and g (n) = O(h(n)). 18. Compute an exact value for 19. Compute an exact value of
n 2 i=1 (3i n i=1

+ 2i + 5). + 2j ).
n(n+1) . 2 n(n+1)(2n+1) . 6

i j =1 (3i

20. Use mathematical induction to prove that 1 + 2 + + n =

21. Use mathematical induction to prove that 1 + 22 + + n2 = 6

22. Use mathematical induction to prove that 1 + 23 + + n3 = [ n(n2+1) ]2 . 23. Use integration by parts to obtain the anti-derivative of f (x) = x2 e3x . 24. Use integration by parts to obtain the anti-derivative of f (x) = x ln x. 25. Use the integral theorem to establish that 1 + 2 + + n = (n2 ). 26. Use the integral theorem to establish that 1 + 4 + 9 + + n2 = (n3 ). 27. Use the integral theorem to establish that 1 + 2k + 3k + + nk = (nk+1 ), where k 1 is an integer. 28. Provide a closed-form expression for the asymptotic growth of log 1 + log 2 + + log n. 29. Show that log(n!) = (n log n). 30. Provide a closed-form expression for the asymptotic growth of n + n/2 + n/3 + + 1. 31. Use the chain rule of dierentiation to compute the derivative of f (x) = 2x + 1. 32. Use the quotient rule of dierentiation to compute the derivative of f (x) = (3x + 5)/(7x2 + x + 11) 33. Use the product rule of dierentiation to compute the derivative of f (x) = x cos(x) + x2 sin(x).
n+1) . 34. Use LHopitals Rule twice to determine limitn n(2 n2 35. Discuss the growth of the function f (n) = 2 2 log n . Polynomial? Superpolynomial? Exponential? Explain and show work.

36. Prove that n! = (2n ). 37. Compare the growth rates of 22 and n!. 38. Discuss the growth of the function f (n) = (log n)log n . Polynomial? Superpolynomial? Exponential? Explain and show work. 39. Discuss the growth of the function f (n) = ( 2)log n . Polynomial? Superpolynomial? Exponential? Explain and show work. 40. Discuss the growth of the function f (n) = (log n)!. 41. Discuss the growth of the function f (n) = n1/ log n . 42. Prove that the running time of an algorithm is (g (n)) i its best-case running time is (g (n)), and its worst-case running time is O(g (n)).
n

Exercise Hints and Answers. 1. C = 6 and k = 1 are sucient (why?). 2. C = 1/45 and k = 1 are sucient (why?). 3. Set up the inequality that the assumption guarantees, and divide both sides by the constant. 4. C1 = 0.5, k1 = 1, and C2 = 2, k2 = 1 are sucient constants (why?).
(n) 5. limitn f = C means that for every > 0, there exists k 0 such that, for all g (n) n k, f (n) | C| < . g (n)

Choose epsilon to be a suciently small multiple of C , and use algebra to establish the needed inequalitites. 6. Use the previous exercise. 7. To prove g (n) = O(f (n)), use a proof by contradiction. In other words, assume the needed C and k did exist, and show how this leads to a contradiction. 8. Big- implies a lower bound. 9. True. 10. False. Use Exercise 7. 11. (n2.1 log n). 12. C = 1 + L and k = 1 are sucient (why?). 13. After taking logarithms, use the result of the previous exercise. 14. False. See Exercise 10. 15. C = C1 C2 and k = max(k1 , k2 ) are sucient for the needed constants. 16. Use Exercise 4. 17. The functions take turns dominating (why?). 18. 19.
n (2n2 2

+ 5n + 13).

n(n+1)(8n+7) (). 6 n(n+1) 2

20. Inductive assumption: 1+2+. . . n = (n+1)(n+2) . 2

for some n 1. Show: 1+2+. . . n+(n+1) =

21. Inductive assumption: 12 + 22 + . . . n2 = n+3) . . . n2 + (n + 1)2 = (n+1)(n+2)(2 . 6 8

n(n+1)(2n+1) 6

for some n 1. Show: 12 + 22 +

22. Inductive assumption: 13 + 23 + . . . n3 = ( n(n2+1) )2 for some n 1. Show: 13 + 23 + n+2) ). . . . n3 + (n + 1)3 = ( (n+1)( 2 23. 24.
e3x (x2 3 1 2 x 2

+ 2x/3 + 2/9 + C ).

1 2 ln(x) 4 x + C.

25. The antiderivative of xm is xm+1 /(m + 1) + C . 26. The antiderivative of xm is xm+1 /(m + 1) + C . 27. The antiderivative of xm is xm+1 /(m + 1) + C . 28. Use the Integral Theorem and integration by parts. 29. Use a logarithm identify to write the expression as a sum. 30. Factor out the n and use the Integral Theorem. 31. f (x) = 1/ 2x + 1. 32. f (x) = (21x2 70x + 28)/(7x2 + x + 11)2 . 33. f (x) = x sin(x) + (x2 + 1) cos(x). 34. 1/2. 35. Polynomial growth but not logarithmic. 36. Use the log-of-ratios technique, and the fact that
n i=1

log(i) = (n log n).

37. The rst function is little- of the second. Use the log-of-ratios technique. 38. Superpolynomial but not exponential growth. Use the log-of-ratios technique against an arbitrary polynomial function. Then use it again against an arbitrary exponential functions. 39. Polynomial (in particular, sub-linear). Use logarithm identities. 40. Superpolynomial. First analyze the growth of log(f (n)). Then consider the growth of 2log(f (n)) which has the same growth as f (n). 41. Constant growth. Take the logarithm of f (n) and analyze its growth.