Investigation Of Mutiara Genset Trouble

By: Maintenance Engineering – VICO Indonesia

1. Background
Started by blackout of Mutiara Central Plant on October 30, 2007, Mutiara gensets was
then frequently trip. We have recorded 8 trip events on Genset#5, 10 trip events on
Genset#3 and 2 trip events on Genset#4. At the last trip of Genset#4, found that the
winding indicate poor insulation resistance leading to refurbishment of this unit (by
Hidup Baru).
The lost of production regarding this problem is the major concern.
To know why is important to eliminate and anticipate the same problem in the future, at
the end to safe our production.
Let’s say MTZ-1900A as genset#2; MTZ-1900B as genset#3; MTZ-1900C as genset#5;
MTZ-1900D as genset#4 and backup Diesel Genset as genset#1.

2. Chronologies
1. Trip (Blackout) on Oct 30, 2007 at 03:06 AM
Normal running : Genset#2,3,4 and 5
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
- Genset#4 Trip indicated under frequency
- Genset#2 Trip indicated nothing, guess it was under frequency
Load shedding system worked, but the response to shed maybe slower than the
overload sensed unit#2 and #4

2. Trip on 31 Oct, 2007 at 01:22 AM

Normal running : Genset#2,3,4 and 5
- Genset#5 Trip indicated Phase over current
- Genset#4 engine S/D due to detonation
Genset#1 was successful to online so the system backed up
Load shedding worked as expected.

3. Trip on Nov 03, 2007 at 01:20 AM

Normal running : Genset#2,3,4 and 5
- Genset#5 Trip indicated Phase Overcurrent
- Genset#4 Trip indicated Phase Overcurrent and Stator RTD over temp
- Genset#3 Trip no indication recorded in GMR
Load shedding worked as expected

4. Trip on Nov 07, 2007 at 23:09 PM

Normal running : Genset#1,2,3 and 5. Genset#4 was being repaired at Hidup Baru.
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
Load Shedding system worked as expected
5. Trip on Nov 10, 2007 at 14:22 PM and 17:25 PM
 Normal running : Genset#1,2,3 and 5
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
Load shedding worked as expected

6. Trip on Nov 14, 2007 at 00:24 AM

Normal running : Genset#1,2,3 and 5
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
Load shedding worked as expected

7. Trip on Nov 16, 2007 at 11:56 AM

Normal running : Genset#1,2,3 and 5
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
Load shedding worked as expected

8. Trip on Nov 23, 2007 at 01:06 AM

Normal running : Genset#1,2,3 and 5
- Genset#5 Trip indicated Loss of Excitation
- Genset#3 Trip indicated Reactive Power Trip
Load shedding worked as expected

3. Description of incidents

At the first of occurrence of this problem is reported by field maintenance that on 30th of
October 2007 Mutiara Central Power black out. The normal condition before S/D was
genset#2,3,4 and 5 run in parallel. Genset#5 was trip at the first time with indication Loss
of Excitation. Genset#3 then (almost simultaneously) trip with indication Reactive
Power. After that, genset#4 S/D with indication underfrequency due to suddenly
overloaded. This is because genset#1 fail to run for backup the loss of genset#3 and 5.
This condition had lead to blackout of MCP. Under this circumstance, the investigation
was then focused to genset#3 and genset#5.

The second, on 31st of Oct 2007, the event shown with phase overcurrent in genset#5 and
genset#4 S/D due to detonation (fuel problem, found the fuel solenoid abnormal
operation). Genset#1 was successfully backup the lost of Genset#5, therefore no blackout
on this event. The normal condition before S/D was genset#2,3,4 and 5 run in parallel.
After this event, maintenance team did corrective action to:
- Adjust fuel regulator differential pressure
- Repair fuel injection solenoid.
- Adjust AVR genset#2 and replace varistor (part of exciter).

The third, on 3rd of Nov 2007. The normal condition before was genset#2,3,4 and 5 run in
parallel. The event was almost the same as before, but, in addition, genset#4 was also trip
phase OC and RTD over temperature and genset#3 trip with no indication. Why no
indication? This is because the memory of GMR was full by alarm of RTD event.
Because maintenance crew install a dummy RTD without disabling the protection
function.

The fourth and next event is almost the same condition, but the most mysterious
problems to find the root causes. The symptoms are genset#5 trip with indication of Loss
of Excitation and Genset#3 with Reactive Power indication. This is almost occurred in
every event since the first blackout except on event when the phase over current existed.

4. Finding

• Informed that there was an AVR replacement work in unit#2 at 29th of Oct 2007.
And after trip on 31st Oct 2007, a report from maintenance crew said that AVR of
unit#2 is re-adjusted.
• At the event of phase overcurrent on 31st of Oct and 03rd of Nov, found that an
excessive of reactive power produced by generator while the active power is
considered in normal range.
• From correspondence with operators informed that power factor hunting at the
moment just before trip happened.
• Found LoE setting of unit#5 and Reactive Power protection setting of unit#3 are
too sensitive.
• At the last event on November 23, 2007, informed that genset#3 and genset#5 trip
with the same indication as before while attempting to parallel genset#1. The
AVR of genset#1 at this occurrence was still in problem.

5. Analysis

By using five-why analysis, let’s define the real causes of the problem.
This is not as simple root cause analysis as we imagine, because many events associated
in this investigation.

Possibility of causes

If we see the events chronology, we can say that the whole events show two different
behaviors. The phase over current and LoE accompanied with excessive reactive power.
But let’s see on “Finding” that the over current event was driven by excessive reactive
power also. This is significantly valuable information! Then, we can say that the over
current symptom is really the excessive reactive power symptoms. We will then consider
the excessive reactive power mainly in our discussion since this symptom happened in
every event.

The Loss of Excitation event basically occurred when a machine loss the field current
produced by exciter or exciter didn’t produce any power. The relay sensed the ‘loss of
excitation’ by seeing the generator impedance from its terminal or by monitoring the
reactive power flow. When reverse reactive powers reach a value limited by the setting
the relay would sense as ‘loss of excitation’. When multiple machines operated in
parallel, a really occurrence of loss of excitation in one machine will cause that machine
to get its excitation from system by absorb reactive power from system to maintain its
terminal voltage constant as the system voltage. In the real loss of excitation problem, we
would see a machine has reverse reactive power (or indicated by lead power factor at cos
phi meter) while the other machines will give more reactive power than normal. In these
events we see that two machines (genset#3 and genset#5) had reverse reactive power.
The fact that informed by operator regarding power factor hunting indicated on cos phi
meter is also strengthen this hypothesis. We then consider by this information that the
Loss of Excitation was only a nuisance trip caused by reactive power or power factor
hunting.

How can Reactive Power swing?

In the parallel operating system, the voltage and speed of the machines is considered as
constant, so that if any disturbance in the system occurred the voltage would drastically
dip, the AVR would response to stabilize the voltage to its system voltage reference.
The excitation system will increase or decrease the field current to maintain the voltage
constant. This process instantly, cause reactive power varies increase or decrease. The
small vary would be tolerable. But, in most system uses induction motors the frequent
event of motor starting and any fault in system would tend to a serious consideration due
to sudden voltage drop that might be happen frequently. In this circumstance, the role of
AVR is very important. The stability of AVR and the precise droop setting in parallel
operating is a curious matter. Once a AVR unstable to response the system changes, it
would affect the others on the parallel operation, cause the power factor (indeed reactive
power) swing or usually called “hunting” among the units.

Let’s start to analyze symptoms we have found in event log.

5.1. Event 30th of Oct 2007

The normal condition was genset#2,3,4 and 5 run in parallel. Genset#5 was trip at the
first time with indication Loss of Excitation. Genset#3 then (almost simultaneously) trip
with indication Reactive Power. After that, genset#2 and 4 S/D with indication
underfrequency due to suddenly overloaded. This is because genset#1 fail to run for
backup the loss of genset#3 and 5. This condition had lead to blackout of MCP
The event sequence is as follow:

Genset#5 Genset#3 Trip Genset#2&4 Trip BLACKOUT

Trip LoE Reactive Power Underfrequency

This two events almost simultenously

Genset#1 Attempt to auto-

Genset#1 fail to online
start and online
We identify 4 critical points.
The five-why analysis is as follow:
The red box indicated the critical points to be evaluated. The brown boxes are the root
causes found as result of this analysis.

MCP Power System Blackout

1ST Why?

Genset#5 Genset#3 Trip Genset#1 fail to online Genset#2&4 Trip

Trip LoE Reactive Power Underfrequency
2ndWhy?
2ndWhy?
nd
2 Why?
Could not synchronize to Suddenly Overloaded
True Loss of Field or exciter system
Power Factor unstable/hunting then
reach the relay pickup setting value 3rdWhy? 3rdWhy?

3rdWhy? Unstable frequency Load Shed not

Unstable voltage working properly
Field winding shorted
Excitation current varies 4thWhy?
4thWhy?
Governor problem Response of relay to trip
4thWhy? AVR problem faster than response of
Sync Check relay Load Shedding system to
AVR sense voltage
problem shed load as required
variations
5thWhy?
5thWhy?
5thWhy? Component damage
Improper tuning Time delay setting of UF
Root causes: relay too short, as found
-Frequent system fault/disturbances 1s. Typically 5s in
-AVR genset#2 problem
unstable common practices.
-Lack of generator data leading to
mistake in relay setting
too
sensitive?

The root causes of event 1 are AVR genset#2 problem and poor setting of the relay. This
guess to AVR genset#2 is strengthened by information of adjustment this unit after trip
on 31st Oct 2007, due to power factor hunting. See Appendix1.

5.2. Event 31st Oct and 03rd Nov 2007

The sequence of event and the five-why analysis is as follow:

 Genset#5, #4 Trip Phase
overcurrent Genset#4 Engine S/D NO
Genset#4 RTD over temp due to detonation BLACKOUT

Genset#1 Attempt to auto- Genset#1 succesful to online

start and online

Genset#5 and 4 Trip Phase

Overcurrent and RTD overtemp
Genset#4 Engine
ST S/D detonation
1 Why?

Phase current increase to level beyond 1ST Why?

the setpoint value (414A).
The trip value as recorded was 430 A Fuel system problem

2ndWhy?
Supply excessive KVAR while KW
remains in normal value 2ndWhy?
Found regulator diff pressure setting too high
rd fuel injection system abnormal operation
3 Why?
Fault or disturbances in the 3rdWhy?
system
Fuel quality degraded
Solenoid fuel injection abnormal operation
4thWhy?
Found genset#4 winding was 4thWhy?
poor insulation
fault
Broken solenoid, loss electrical control
occurs if voltage applied to
Fuel contaminated, poor gas quality
5thWhy?
Root cause: 5thWhy?
Contamination of moisture and dirt
because no filter in air cooling inlet. Root Causes:
Lifetime of solenoid elapsed
Does the PM checklist ask to check this solenoid??
Bad quality gas from well

The basic cause of the engine detonation seems tend to mechanical problem regarding the
fuel system.
The basic cause of overcurrent problem in genset#4 and 5 is the contamination of dirt
and moisture to degrade the genset#4 winding.

5.3. Event on 07th Nov 2007 and after.

In this event, genset#4 was out of service due to poor insulation finding. The normal
condition was genset#1,2,3 and 5 online. Actually several trip events occurred within
these events, but all of with same indication, Loss Excitation of Genset#5 and Reactive
Power on genset#3. Several maintenance actions have been done during these events. But
the same problem still existed. Since the first problem had the same symptom, and the
root cause analysis say that the problem is AVR, then we must focused to find the AVR
problem. Which AVR was got problem?

The last event on 23rd November 2007 provided valuable information. Genset#1 got a
problem with the voltage regulator. This problem was identified by the fact of
uncontrollable overvoltage condition in genset#1 causing this genset out of system.
Because this genset is for backup the power during absence of genset#4, then genset#1
tried to re-tie to system after few observations the voltage was stable. Instantly, genset#3
and 5 tripped with same indication as before. So it’s known that the recent problems
clearly caused by defect of AVR on genset#1. If we see back to first event, it is noted that
genset#1 had an experience fail to auto-online due to AVR fail to synchronize with
system voltage. So this problem actually had existed since the first event.

The sequence of this event is as follow:

Genset#1, 2, 3, 5 was run online Genset#4 was under repair at HB

This two events almost simultenously NO

BLACKOUT
G#1 &2
Genset#5 Genset#3 Trip covered the
Trip LoE Reactive Power load because
the load shed
system work

After genset#1 known got a problem with the voltage, there is additional event as follow:

Genset#1, 2, 3, 5 was run online Genset#4 was under repair at HB

This two events almost simultaneously

NO
BLACKOUT
Genset#1 Genset#1 Genset#5 Genset#3 Trip G#1 &2
Trip attempted to Trip LoE Reactive Power covered the
Overvoltage be ONLINE load because
the load shed
system work
 Genset#5 Genset#3 Trip
Trip LoE Reactive Power

1st Why?
Low impedance or minus
KVAR reach the relay pickup
setting value

2ndWhy?
- Power Factor hunting / excitation
current varies due to voltage variations
- Improper relay setting?

3rdWhy?
AVR defect lead to unstable
response when system fault or got
disturbances

4thWhy?
Found unstable response of
AVR in genset#1.

5thWhy?
Root causes:
- AVR Component damage
- Improper tuning

From the above five-why analysis, we can count several root causes of frequent power
loss in Mutiara Central as follows:

1. AVR in genset#2 and AVR genset#1 had a problem lead to unstable operation
due to system voltage variations.
2. Bad ventilation system of genset#4 absorbed contaminant to winding lead to
degradation of insulation caused short circuit fault
3. The fuel system of engine is not good enough as it is frequently had adjustments
4. The relay setting of LoE, Reactive power and underfrequency still too sensitive
and need to be revised. It is recommended also to see the other protection setting
to be reviewed
6. Recommendations

1. Install filter in cooling air inlet of genset#4

2. Install RTD in genset#3 stator winding and activate the protection in all unit’s
stator winding to prevent winding damage.
3. Review the load shedding system whether it can response faster?
4. Review the setting of protection relay to find any weak point, refer to generator
data provided by Trakindo. The most critical to be reviewed are:
- Loss of Excitation impedance circle of genset#5 need to be narrowed, so less
sensitive than before. As found setting 60 ohm, recommended to reduce to be 54
ohm. Already done.
- Reactive Power Limit setting of genset#3. Refer to generator capability curve.
- Under frequency setting genset#4. Increase time delay to allow load shed working
earlier than relay command to trip.
5. Modify the fuel system design to get better fueling to sustain engine power.
6. Detail task of PM Service to cover fuel injection solenoid verification.
7. Training maintenance personnel on how to adjust AVR correctly.
8. Find out tester or simulator for Automatic voltage Regulator.
 Appendix 1
Email References

1. Email form Pak Zainal regarding adjustment of AVR in genset#2

2. Email from Pak Zainal regarding problem AVR in genset#1

 Appendix2
Relay Setting

The overall setting review is in progress.

The critical setting need to be revised ASAP:

- Setting Reactive Power at genset#3
- Setting Loss Excitation of genset#5

Rubah setting
menjadi “No relays”

Figure1. Rubah setting alarm reaktif power genset#3

Figure2. Setting Impedance Loss of Excitation

Berdasarkan data reaktansi sinkron yang telah didapatkan (Xd=1.5970 ohm; X’d =
0.1278 ohm; X”d = 0.0908 ohm), direkomendasikan setting impedansi LoE dirubah
seperti yang warna merah.
Figure3 Setting Loss of Excitation di GMR
 Appendix2
Event Logger

Event Log Genset#3

Event Log Genset#4
Event log Genset#5