Knapsack problem

From Wikipedia, the free encyclopedia

Jump to: navigation, search

Example of a one-dimensional (constraint) knapsack problem: which boxes should be chosen

to maximize the amount of money while still keeping the overall weight under or equal to 15
kg? A multiple constrained problem could consider both the weight and volume of the boxes.
Modeling the shapes and sizes would instead constitute a packing problem.
(Solution: if any number of each box is available, then three yellow boxes and three grey boxes; if only the
shown boxes are available, then all but the green box.)

The knapsack problem or rucksack problem is a problem in combinatorial optimization:

Given a set of items, each with a weight and a value, determine the number of each item to
include in a collection so that the total weight is less than a given limit and the total value is
as large as possible. It derives its name from the problem faced by someone who is
constrained by a fixed-size knapsack and must fill it with the most useful items.
The problem often arises in resource allocation with financial constraints. A similar problem
also appears in combinatorics, complexity theory, cryptography and applied mathematics.
The decision problem form of the knapsack problem is the question "can a value of at least V
be achieved without exceeding the weight W?"

Contents
[hide]

1 Definition

2 Computational complexity

3 Dynamic programming solution

o 3.1 Unbounded knapsack problem
o 3.2 0-1 knapsack problem

4 Greedy approximation algorithm

5 Dominance relations to simplify the resolution of the unbounded knapsack problem

o 5.1 Collective dominance
o 5.2 Threshold dominance
o 5.3 Multiple dominance
o 5.4 Modular dominance

6 See also

7 References

8 Further reading

9 External links

[edit] Definition
In the following, we have n kinds of items, 1 through n. Each kind of item i has a value vi and
a weight wi. We usually assume that all values and weights are nonnegative. The maximum
weight that we can carry in the bag is W.
The most common formulation of the problem is the 0-1 knapsack problem, which restricts
the number xi of copies of each kind of item to zero or one. Mathematically the 0-1-knapsack
problem can be formulated as:

maximize

subject to

The bounded knapsack problem restricts the number xi of copies of each kind of item to a
maximum integer value ci. Mathematically the bounded knapsack problem can be formulated
as:

maximize

subject to

The unbounded knapsack problem places no upper bound on the number of copies of each
kind of item.
Of particular interest is the special case of the problem with these properties:

it is a decision problem,

it is a 0-1 problem,

for each kind of item, the weight equals the value: wi = vi.

Notice that in this special case, the problem is equivalent to this: given a set of nonnegative
integers, does any subset of it add up to exactly W? Or, if negative weights are allowed and W
is chosen to be zero, the problem is: given a set of integers, does any subset add up to exactly
0? This special case is called the subset sum problem. In the field of cryptography the term
knapsack problem is often used to refer specifically to the subset sum problem.
If multiple knapsacks are allowed, the problem is better thought of as the bin packing
problem.

[edit] Computational complexity

The knapsack problem is interesting from the perspective of computer science because

there is a pseudo-polynomial time algorithm using dynamic programming

there is a fully polynomial-time approximation scheme, which uses the pseudopolynomial time algorithm as a subroutine

the problem is NP-complete to solve exactly, thus it is expected that no algorithm can
be both correct and fast (polynomial-time) on all cases

many cases that arise in practice, and "random instances" from some distributions, can
nonetheless be solved exactly.

The subset sum version of the knapsack problem is commonly known as one of Karp's 21
NP-complete problems.
There have been attempts to use subset sum as the basis for public key cryptography systems,
such as the Merkle-Hellman knapsack cryptosystem. These attempts typically used some
group other than the integers. Merkle-Hellman and several similar algorithms were later
broken, because the particular subset sum problems they produced were in fact solvable by
polynomial-time algorithms.
One theme in research literature is to identify what the "hard" instances of the knapsack
problem look like[1][2], or viewed another way, to identify what properties of instances in
practice might make them more amenable than their worst-case NP-complete behaviour
suggests.
Several algorithms are freely available to solve knapsack problems, based on dynamic
programming approach[3], branch and bound approach[4] or hybridations of both approaches[5]
[6][7][8]

[edit] Dynamic programming solution

[edit] Unbounded knapsack problem
If all weights (
) are nonnegative integers, the knapsack problem can be
solved in pseudo-polynomial time using dynamic programming. The following describes a
dynamic programming solution for the unbounded knapsack problem.
To simplify things, assume all weights are strictly positive (wi > 0). We wish to maximize
total value subject to the constraint that total weight is less than or equal to W. Then for each
w W, define m[w] to be the maximum value that can be attained with total weight less than
or equal to w. m[W] then is the solution to the problem.
Observe that m[w] has the following properties:

m[0] = 0 (the sum of zero items, i.e., the summation of the empty set)

where vi is the value of the i-th kind of item.

Here the maximum of the empty set is taken to be zero. Tabulating the results from m[0] up
through m[W] gives the solution. Since the calculation of each m[w] involves examining n
items, and there are W values of m[w] to calculate, the running time of the dynamic
programming solution is O(nW). Dividing
by their greatest common
divisor is an obvious way to improve the running time.
The O(nW) complexity does not contradict the fact that the knapsack problem is NPcomplete, since W, unlike n, is not polynomial in the length of the input to the problem. The

length of the input to the problem is proportional to the number, logW, of bits in W, not to W
itself.

[edit] 0-1 knapsack problem

A similar dynamic programming solution for the 0-1 knapsack problem also runs in pseudopolynomial time. As above, assume
are strictly positive integers.
Define m[i,w] to be the maximum value that can be attained with weight less than or equal to
w using items up to i.
We can define m[i,w] recursively as follows:

m[0,w] = 0

m[i,0] = 0

m[i,w] = m[i 1,w]; if wi > w

; if
.

The solution can then be found by calculating m[n,W]. To do this efficiently we can use a
table to store previous computations. This solution will therefore run in O(nW) time and
O(nW) space.

[edit] Greedy approximation algorithm

George Dantzig proposed (1957) a greedy approximation algorithm to solve the unbounded
knapsack problem. His version sorts the items in decreasing order of value per unit of weight,
vi / wi. It then proceeds to insert them into the sack, starting with as many copies as possible
of the first kind of item until there is no longer space in the sack for more. Provided that there
is an unlimited supply of each kind of item, if m is the maximum value of items that fit into
the sack, then the greedy algorithm is guaranteed to achieve at least a value of m / 2.
However, for the bounded problem, where the supply of each kind of item is limited, the
algorithm may be very much further from optimal.

[edit] Dominance relations to simplify the resolution of the

unbounded knapsack problem
Some relations between items are such that quite a lot of items may be useless to consider to
build an optimal solution. These relations are known as Dominance relations. When an item
"i" is known to be dominated by a set of items "J", it can be thrown out of the set of items
usable to build an optimal value. The dominance relations between items allow the size of the
search space to be signicantly reduced. All the dominance relations, enumerated below,

could be derived by the following inequalities:

, and

for some
where

[edit] Collective dominance

The i-th item is collectively dominated by J, written as

iff

and

for some
i.e. = 1. The verification of this dominance is
computationally hard, so it can be used in a dynamic programming approach only.

[edit] Threshold dominance

the i-th item is threshold dominated by J, written as
iff (the above inequalities
hold when
. This is an obvious generalization of the collective dominance by using
instead of single item "i" a compound one, say times item "i". The smallest such defines
the threshold of the item "i", written ti = ( 1)wi.

[edit] Multiple dominance

The item "i" is multiply dominated by "j", written as

, iff

, and

for some
i.e.
. This dominance
could be efficiently used in a preprocessing because it can be detected relatively easily.

[edit] Modular dominance

Let b = the best item, i.e

as
iff
1

for all j The item i is modularly dominated by j, written

, and
i.e. J = {b,j}, = 1,xb = t,xj =

[edit] See also

List of knapsack problems

Packing problem

Cutting stock problem

Continuous knapsack problem

[edit] References
1. ^ Pisinger, D. 2003. Where are the hard knapsack problems? Technical Report
2003/08, Department of Computer Science, University of Copenhagen, Copenhagen,
Denmark.
2. ^ L. Caccetta, A. Kulanoot, Computational Aspects of Hard Knapsack Problems,
Nonlinear Analysis 47 (2001) 55475558.
3. ^ Rumen Andonov, Vincent Poirriez, Sanjay Rajopadhye (2000) Unbounded
Knapsack Problem : dynamic programming revisited European Journal of Operational
Research 123: 2. 168-181 http://dx.doi.org/10.1016/S0377-2217(99)00265-9
4. ^ S. Martello, P. Toth, Knapsack Problems: Algorithms and Computer Implementation
, John Wiley and Sons, 1990
5. ^ S. Martello, D. Pisinger, P. Toth, Dynamic programming and strong bounds for the
0-1 knapsack problem , Manag. Sci., 45:414-424, 1999.
6. ^ Vincent Poirriez, Nicola Yanev, Rumen Andonov (2009) A Hybrid Algorithm for the
Unbounded Knapsack Problem Discrete Optimization
http://dx.doi.org/10.1016/j.disopt.2008.09.004
7. ^ G. Plateau, M. Elkihel, A hybrid algorithm for the 0-1 knapsack problem, Methods
of Oper. Res., 49:277-293, 1985.
8. ^ S. Martello, P. Toth, A mixture of dynamic programming and branch-and-bound for
the subset-sum problem, Manag. Sci., 30:765-771

[edit] Further reading

Garey, Michael R.; David S. Johnson (1979). Computers and Intractability: A Guide
to the Theory of NP-Completeness. W.H. Freeman. ISBN 0-7167-1045-5. A6: MP9,
pg.247.

Martello, Silvano; Paolo Toth (1990). Knapsack Problems: Algorithms and Computer
Implementations. John Wiley & Sons. ISBN 0-471-92420-2.

Kellerer, Hans; U. Pferschy, D. Pisinger (2005). Knapsack Problems. Springer Verlag.

ISBN 3-540-40286-1.

inria-00335065, version 1 -, 00335065 (28 Oct 2008).

http://hal.inria.fr/docs/00/33/50/65/PDF/jvers08.pdf: version 1.

[edit] External links

Lecture slides on the knapsack problem

PYAsUKP: Yet Another solver for the Unbounded Knapsack Problem, with code,
benchmarks and downloadable copies of some papers.

Home page of David Pisinger with downloadable copies of some papers on the
publication list (including "Where are the hard knapsack problems?")

Knapsack Problem solutions in many languages at Rosetta Code

Dynamic Programming algorithm to 0/1 Knapsack problem

0-1 Knapsack Problem in Python

Interactive JavaScript branch-and-bound solver

Retrieved from "http://en.wikipedia.org/wiki/Knapsack_problem"

Categories: Cryptography | NP-complete problems | Operations research | Dynamic
programming | Combinatorial optimization
Views

Article

Discussion

Edit this page

History

Personal tools

Try Beta

Log in / create account

Navigation

Main page

Contents

Featured content

Current events

Random article

Search

Special:Search

Go

Interaction

About Wikipedia

Community portal

Recent changes

Contact Wikipedia

Donate to Wikipedia

Help

Toolbox

What links here

Related changes

Upload file

Special pages

Printable version

Permanent link

Cite this page

Languages

Catal

esky

Deutsch

Espaol

Franais

Search

Italiano

Nederlands

Polski

Portugus

Svenska

Ting Vit

Trke

This page was last modified on 15 April 2010 at 23:53.

Text is available under the Creative Commons Attribution-ShareAlike License;

additional terms may apply. See Terms of Use for details.
Wikipedia is a registered trademark of the Wikimedia Foundation, Inc., a non-profit
organization.

Contact us

Privacy policy

About Wikipedia

Disclaimers