PERFORMANCE-RELATED

ATTRIBUTES
Presented by: Eddie Santillan

1
Correctness
Software is correct if it satisfies the functional
requirements specifications
assuming that specification exists!

If specifications are formal, since programs are formal

objects, correctness can be defined formally
It can be proven as a theorem or disproved by
counterexamples (testing)

2
 The Limits of Correctness
It is an absolute (yes/no) quality
there is no concept of degree of correctness
there is no concept of severity of deviation

What if specifications are wrong?

(e.g., they derive from incorrect requirements or errors in
domain knowledge)

3
Reliability
Reliability
informally, user can rely on it
can be defined mathematically as probability of absence
of failures for a certain time period
Probability of failure-free operation
Depends on hardware/software environment and user
Common metrics include defect density and mean time to
failure

4
Hardware Reliability Metrics
Hardware metrics are not suitable for software since
its metrics are based on notion of component failure
Software failures are often design failures
Often the system is available after the failure has
occurred
Hardware components can wear out
Software Reliability Metrics
Reliability metrics are units of measure for
system reliability
System reliability is measured by counting the
number of operational failures and relating
these to demands made on the system at the
time of failure
A long-term measurement program is required
to assess the reliability of critical systems
Reliability Metrics
Probability of Failure on Demand (POFOD)
POFOD = 0.001
For one in every 1000 requests the service fails per
time unit
Rate of Fault Occurrence (ROCOF)
ROCOF = 0.02
Two failures for each 100 operational time units of
operation
Reliability Metrics
Mean Time to Failure (MTTF)
average time between observed failures (aka MTBF)

Availability = MTBF / (MTBF+MTTR)

MTBF = Mean Time Between Failure
MTTR = Mean Time to Repair

Reliability = MTBF / (1+MTBF)

Time Units
Raw Execution Time
non-stop system
Calendar Time
If the system has regular usage patterns
Number of Transactions
demand type transaction systems
Idealized Situation
Requirements are correct

Reliability
Correctness

10
Usability
Expected users find the system easy to use
Other term: user-friendliness
Rather subjective, difficult to evaluate
Affected mostly by user interface
e.g., visual vs. textual

11
Computer Performance
Computer performance is the amount of work accomplished by
a computer system. Depending on the context, high computer
performance may involve one or more of the following:
Short response for a given piece of work
High throughput (rate of processing work)
Low utilization of computing resources(s)
High availability of the computing system or application
Fast (or highly compact) data compression and decompression
High bandwidth
Short data transmission time
Computer Performance
Computer performance metrics (things to measure)
include
availability response time
channel capacity latency
completion time service time
bandwidth throughput
relative efficiency scalability
performance per watt compression ratio
instruction path length speed up
Performance Tuning
Performance tuning is the improvement of system
performance.
This is typically a computer application, but the same methods
can be applied to economic markets, bureaucracies or other
complex systems.
The motivation for such activity is called a performance
problem, which can be real or anticipated.
Most systems will respond to increased load with some degree
of decreasing performance. A system's ability to accept a higher
load is called scalability, and modifying a system to handle a
higher load is synonymous to performance tuning.
Performance Tuning
Systematic tuning follows these steps:
Assess the problem and establish numeric values that categorize acceptable
behavior.
Measure the performance of the system before modification.
Identify the part of the system that is critical for improving the performance.
This is called the bottleneck.
Modify that part of the system to remove the bottleneck.
Measure the performance of the system after modification.
If the modification makes the performance better, adopt it. If the
modification makes the performance worse, put it back to the way it was.
Performance Equation
The total amount of time (t) required to execute a particular
benchmark program is
T = N*C/f or P = I*f/N
where
P = 1/t is "the performance" in terms of time-to-execute
N is the number of instructions actually executed (the instruction path
length).
f is the clock frequency in cycles per second.
C= 1/I is the average cycles per instruction (CPI) for this benchmark.
I= 1/C is the average instructions per cycle (IPC) for this benchmark.