An Intelligent LED Illumination Control System Using Camera

Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

An Intelligent LED Illumination Control System Using Camera

1
Giao Pham Ngoc, 2Suk-Hwan Lee, 3Young Moon Yu, 4Ki-Ryong Kwon
1
Dept. of IT Convergence & Application Eng., Pukyong National University, Busan, South
Korea, E-mail: ngocgiaofet@gmail.com
2
Dept. of Information Security, Tongmyong University, Busan, South Korea, E-mail:
skylee@tu.ac.kr
3
LED-Marine Convergence Technology R&BD Center, Pukyong National University, Busan,
South Korea, E-mail: ymyu@pknu.ac.kr
*4Corresponding Author
Dept. of IT Convergence & Application Eng., Pukyong National University,
Busan, South Korea, E-mail: krkwon@pknu.ac.kr

Abstract
Recently, smart LED lighting systems have developed and implemented in diverse spots because
LED’s characteristic. These systems are often applied by places such as buildings, rooms, garages and
parking lot. However, these LED systems are controlled by light sensors, motion sensor based on
illumination and motion detection. In this paper, we propose an intelligent LED illumination control
system using a camera for buildings, rooms and garages. Our system integrates video processing with
LED system, and only uses a camera to control dimming LEDs based on moving object detection,
determining the position of dimmed LED from the position of moving object and the brightness of
received image from camera.

Keywords: LED control system, LED dimming, Motion detection, Smart home system, Control
dimming, Inverse perspective mapping.

1. Introduction
Nowadays, smart LED lighting systems have been increasing, and researched by many companies
and research centers with advanced countries. At this point, smart lighting LED systems are applied in
spaces such as parking lot [1], smart homes [2,3], and now it can be substituted for the existing lights
because of low cost and durability.
For smart lighting applications in spaces such as buildings, rooms, street so on, the LED light has a
variety of advantages compared with the existing lights. It is easy to interwork with other electronic
modules such as sensors and communicated modules to provide new services, saves energy [1-6]. In
addition, it can be dimmed by pulse width modulation [7] (PWM) or current control to change its
brightness easily. Furthermore, it has a low power characteristic, whose power consumption is much
lower than the existing lights, and a heating problem has been improved compared to the past.
Smart LED lighting systems are the convergence technology of the sensor and light. They have
LED switching action based on sensing data of the sensor [4-6], and use light, motion sensors to
compute illumination of environment, detect moving object. These systems control turn on/off, and
adjust the brightness of LED follow motion of object and brightness of environment. By replacing light
and motion sensors by a camera, we integrated video processing with LED system to control turn
on/off and dim illumination of LED for buildings, rooms and garages. The system controls dimming
LEDs based on video processing to compute the position of dimmed LED from the position of moving
object on image, and determine dimming levels based on the brightness of received image from camera.
Our system includes software and hardware. The hardware includes camera, embedded
board/computer (PC) and LED board. Section 2, we introduce related works. And in section 3, we
describe overview of system, design algorithm data processing, adaptive middleware of proposed

Journal of Convergence Information Technology(JCIT) 208

Volume8, Number12, July 2013
doi:10.4156/jcit.vol8.issue12.25
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

system, data flow diagram and action diagram of system, and implement LED board. The experimental
results are described in section 4.

2. Related works

2.1 General LED control system

In recent years, usage of LED light has been increasing. They are applied in mobile phone, LED
television, lighting and so on. Nowadays, it is used widely in lighting system to substitute fluorescent
lamps which has dis-advantages such as high cost, unstable and difficult to integrate with other devices.
But general LED control systems are often controlled by hand or semi-automatic, and used a device to
control a LED [1-3]. And the simplest LED-based lighting systems often use an LED driver to control
turn on/off all LEDs. These typically, fixed-function devices provide a straightforward for controlling
LEDs, and they offer good power efficiency and do not require software programming. In fact, these
systems only control turn on/off all LEDs without select each individual LED. Thus, it is requirement
for developers to give intelligent LED control system based on LED’s characteristic, and those systems
can be control automatically by software or user can program to change system if it is necessary.

2.2 Vision – based LED control system

For development LED lighting applications, we can consider different visions such as flexibility,
energy saving, reliability, automation and so on. Example, LED control systems can be used for
multiple purposes such as buildings, smart home, and parking lot [1-3] or create custom lighting effects
and scenes for special events and important clients [5]. LED lighting is the only type of lighting that
increases in efficacy when dimmed. When integrated with building automation systems such as day
lighting controls [1-3], occupancy sensors, clocks, and shade control systems, energy efficiency can be
increased. Putting the scheduling and automation of lighting systems under digital control increases the
reliability of lighting systems while relieving the burden on building management and maintenance
organizations.
One of visions – based LED control system is to integrate LED system with other devices or
interfaces such as light sensors, motion sensors or TCP/IP, wireless to automatically control, manage
for applications as smart home [2,3], and green house [5] or energy saving [1-3]. The purpose of
integration LED system with other devices or services is optimal, automatic control and management.
Thus, our system focuses on automatic and simplification of LEDs dimming control using camera
which replaces functions of light and motion sensors in previous systems.

2.3 Originality of proposed system

Our system only uses a camera to replace role of light and motion sensors in previous systems. We
integrate a camera with LED system to automatically control turn on/off and adjust the brightness of
each individual LED follow position of moving object and illumination of environment based on video
processing.
In this system, we program video processing to detect moving object, and then we locate the
position of moving object in real space and map it to the corresponding LED on ceiling based on
coordinates of moving object in image space. Because the brightness of received image is depend on
almost the brightness of environment, our system computes the brightness of dimmed LED based on
brightness of received image. Parameters of position and brightness of LED are transferred to LED
board which has a micro controller can be program to control LED system. User can program to
change operation of system follow other purposes.
LEDs system is designed into LED board with a micro controller, interface RS232/USB, de-
multiplexers and LEDs. The task of LED board is to get parameters of position and brightness of LED
which are sent from computer after video processing step. With designed LED board, system can
control and dim LEDs by micro controller and de-multiplexers.
By using a camera to replace function of light and motion sensors, our system is more simple than
previous system and it can be integrate with other systems depend on user’s purpose. The system is

209
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

operated by video processing, interface programs on computer and program on LED board which we
programmed and developed.

3. Design the proposed system

3.1 Overview of system

Basically, our system includes parts as figure 1: Camera module receives video sequence; the
embedded board or person computer (PC) performs video processing, interface and transfer data to
LED board which consists of LED driver and LED array; LED driver receives data from embedded
board/PC; LED array is displayed following the position of moving object, and the power supplies
energy to embedded board/PC and LED driver, LED array.
Our system operates on embedded board/PC and LED board. The LED driver uses micro controller,
de-multiplexer to control LEDs. The embedded board/PC gets video sequence from camera, and
performs video processing, interfaces between embedded board/PC and LED board, transfers data from
embedded board/PC to LED board. The LED driver receives, analyses data and controls LED array.
Therefore, system has a process on embedded board/PC and a process on LED board. The process on
embedded board/PC includes video processing, interface and transfer data between computer and LED
board. The process on LED board performs to get data from computer and control LEDs.

Figure 1. Overview of system

3.2 Design algorithm data processing for system

The algorithm data processing on embedded board/PC: The system connects to camera. If system
has connection to camera, it will get video sequence and detect moving object [9]. After moving object
detection, system calculates the position of moving object on the floor and corresponding LED on the
top, and then it calculates the brightness (dimming level) of LED based on the brightness of received
image. The position and brightness of LED are built into data message which will be transferred to
LED board. Next, system connects between embedded board/PC and LED board. If having connection,
system will transfer data message to LED board. The LED board receives data message and processes
on LED board.
The algorithm data processing on LED board: the system receives data message which is transferred
from computer then it checks content of data message. If data message is different null, it decodes data
message and checks decrypted data to determine brightness and position of LED. Finally, it sends
control signal to de-multiplexer to control dimming LEDs. The figure 2 describes algorithm data
processing for system.

210
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

(a)

(b)
Figure 2. Algorithm data processing for system; (a) algorithm on embedded board/PC and (b)
algorithm on LED board

211
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

· Moving object detection: There are many approaches for moving object detection in a sequence
video stream. All of them are based on comparing of the current video frame with one from the
previous frames or with something that we call background [8,9]. At here, we performed pre-
processing which consists of reduce noise and convert into grayscale, and used first frame as
background frame; and then we compared background frame and current frame to find different
regions between two frames. If total of different pixels is more than pre-define threshold, we
extract moving object and update background. Algorithm moving object detection is described as
figure 3. S is the total different pixels between two corresponding regions; Ƭ is pre-defined
threshold.

Figure 3. Algorithm moving object detection

· Calculate position of moving object and corresponding LED: After moving object detection, we
have to calculate the position of moving object on the floor to find corresponding LED, and then
control turn on/off. Suppose, the camera is setup on the side of wall while LED array is installed
on the top. In the real space OXYZ with OX is width, OY is length and OZ is height; we have 
is the height of camera, α is the camera angular aperture, β is the angle between optical center of
camera (Ct) and OY horizontal, and the position of moving object on the floor which is attached
OXY space as figure 4(a), has (x, y, 0) coordinates in OXYZ space as figure 4(b).

212
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

(a) (b)
Figure 4. Position of camera and moving object in the real space; (a) position of camera and moving
object in OXY space, and (b) position of camera and moving object in OYZ space

Our purpose is to find LED on top which corresponds to position of moving object in real space
based on the position of moving object on image. To solve that problem, we use inverse perspective
transform [10,11]. Assume that, we receive image has size (h, w) with h is the height of image; w is
the width of image, and the received image is attached O’UV axes as figure 5; and on image space,
moving object has coordinates (u, v). The floor has size (S, L) with S is the width of floor, and L is
the length of floor as figures 4(a) and 5(b). We calculate coordinates of moving object in real space
by inverse perspective transform equations (1) and (2):

 ∝ ∝ ∝ 
 =  ×   −  +  ×  × 
  × −  + (1)
    
  ∝ ∝
y =  ×   −  +  ×  ×   × −  (2)
   

After finding coordinates of moving object, we can find the position of corresponding LED as figure
5.

(a) (b) (c)

Figure 5. Find LED by inverse perspective transform; (a) O’UV axes in image space, (b) OXY axes in
real space and (c) LED array on top view

213
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

· Calculate brightness of LED based on brightness of image: The brightness of LED is dimmed
depending on the brightness of environment when it is day time or night time. Here, we used to
get brightness of received image instead of brightness of environment, and calculate brightness
of LED to control dimming based on brightness of received image [12]. Because the brightness
of received image is depended on the brightness of environment. Suppose the max dimming
value of LED in night time is M, the brightness of received image is Br and dimming value of
LED is D. Br and D values are determined by equations (3) and (4):

 (,,)  (,,)
 = (3)

 =  −  (4)

Max(R, G, B), min(R, G, B) are the highest and lowest color values of received image from
camera. The common M is 255, which corresponds to 8 bits color image. Therefore, the value
range of dimming level is from (255 – Br) to 255, and if Br is zero, we can dim 255 levels.

· LED dimming by PWM: To control brightness of LED, we use pulse width modulation (PWM).
The brightness of LED is depended on PWM duty cycle. If we have  is the period of pulse
sequence, D is brightness of LED, and  is the lifetime of PWM. Relationship between , 
and D is determined by equation (5):

= (5)


Figure 6. PWM signal with 16 levels of voltage value

· Build and transfer data message: the position and brightness of LED are built into string and
transferred to LED board. The structure of data is 6 characters which brightness is stored by 3
characters and position is stored by 3 characters.

3.3 Adaptive middleware of system

Basically, adaptive middleware of system is described as figure 7. When a person enters room,
camera sends video sequence to video processing block to detect moving object and determine the
position of corresponding LED to that moving object, and then it computes the brightness of LEDs.
Output of this block is position and brightness of LEDs. They are continued to send to interface &
transfer block, and built into string. The output of interface & transfer is an encoded message and
transferred to LED board via RS232/USB interface. The LED board gets the encoded message from
computer by AVR micro controller which is integrated, and then it decodes message to determine
position and brightness of LED. Finally, it controls and dims LEDs via de-multiplexers. Figure 8
describes data flow diagram of system, and figure 9 describes action flow diagram on parts of system.

214
An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

Figure 7. Adaptive middleware of system

Figure 8. Data flow diagram

Figure 9. Action flow diagram

215
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

4. Experiment and result

4.1 Implement LED board

(a)

(b)
Figure 10. Implement LED board; (a) block diagram of LED board and (b) hardware structure

As description in section 3, our system is implemented and experimented on computer and LED
board. We designed and implemented LED board as figure 10. The hardware structure of LED board
includes 5 parts:
· AVR micro controller: By using 8 bits microprocessor. The task of this part is to receive the
data message from computer, and decode data message to control other parts.
· De-mux (De-multiplexer): This part gets de-multiplexing signals from AVR micro controller,
and controls LED array
· LED array: Using red, green and blue LEDs. They are displayed following position of moving
object and brightness of image.
· USB-RS232 transceiver: Interface between computer and LED board.
· Power: Supply energy for AVR micro controller and de-mux, LED array. We use battery 5V or
adapter 5V DC with input 220-110 AVC.

216
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

Table 1. Hardware specification of LED board

Module Part name
LED array (1) RGBW led
De-mux (2) CD74HC154, STP16DP05
AVR micro controller (3) Atmega328P
Power (4) Battery 5V or Adapter DC 5V
USB-RS232 transceiver (5) USB cable, Max232

4.2 Experimental results

We used visual studio 2010 to design software which runs on computer, and arduino framework to
design software which runs on LED board. The experimental environment is the lobby of a building.
The camera is installed on the side of wall, and connected to computer. LED array is divided into
blocks which correspond to big LED as figure 7.
When a person enters the central lobby of building, the LED block 2×1 is turned on figure 11(a),
and then person moves towards, LED block 2×2 and 3×2 are turned on figure 11(b) and 11(c). And
when person turns right, LED block 3×3 is turned on figure 11(d). The brightness level of LED is
dimmed by brightness of received image. When person goes to position which has low brightness,
LED will be has high dimming level and reverse.

(a) (b)

(c) (d)
Figure 11. Experimental results; (a) person starts entering central of lobby, (b) and (c) person moves
towards, (d) person turns right

5. Conclusion
In this paper, we design and experiment an automatic LED illumination control system using a
camera, based on moving object detection and the brightness of received image. Our system is
implemented and experimented on LED board. Future, we will implement and experiment with a large
capacity LED system which install on top view as figure 7, and evaluate the level of energy
consumption, and compare with current others.

6. Acknowledgement
This work was supported by the Industrial Strategic Technology Development Program (Project No:
1037416, establishment of infrastructure for LED-marine convergence technology support and
technology development for commercialization) which is funded by the Ministry of Knowledge
Economy (MKE, Korea) and the MSIP (Ministry of Science, ICT & Future Planning), Korea, under

217
 An Intelligent LED Illumination Control System Using Camera
Giao Pham Ngoc, Suk-Hwan Lee, Young Moon Yu, Ki-Ryong Kwon

IT/SW Creative research program supervised by the NIPA (National IT Industry Promotion Agency)"
(NIPA-2013-H0502-13-1023).

References
[1] Insung Hong, Jisung Byun, and Sehyun Park, “Intelligent LED Lighting System with Route
Prediction Algorithm for Parking Garage”, INTELLI 2012: The First International Conference on
Intelligent Systems and Applications, pp. 54-59, 2012.
[2] Zion Hwang, Yoonsik Uhm, Yong Kim, Gwanyeon Kim and Sehyun Park, “Development of LED
Smart Switch with Light-weight Middleware for Location-aware Services in Smart Home”, IEEE
Transactions on Consumer Electronics, IEEE, Vol. 56, No. 3, pp. 1395-1402, August 2010.
[3] Yoonsik Uhm, Insung Hong, Gwanyeon Kim, Byoungjoo Lee and Sehyun Park, “Design and
Implementation of Power-aware LED Light Enabler with Location-aware Adaptive Middleware”,
IEEE Transactions on Consumer Electronics, IEEE, Vol. 56, No. 1, pp. 231-239, February, 2010.
[4] Jia Dongying and Wang Wei, “The Intelligent System for LED lighting Based on STCMCU”
International Conference on Computer and Communication Technologies in Agriculture
Engineering 2010, pp. 445-447, 2010.
[5] Faheem Ijaz, Adeel A.Siddiqui, Byung-Kwan Im and Chankil Lee, “Remote management and
control system for LED based Plant Factory using ZigBee and Internet”, ICACT2012, pp. 942-946,
Feb. 19~22, 2012.
[6] Daeho Kim, Junghoon Lee, Yeongmin Jang and Jaesang Cha, “Smart LED lighting system
implementation using Human tracking US/IR sensor”, ICTC 2011: International Conference on
ITC Convergence 2011, pp. 290-293, September 28~30, 2009.
[7] Ramandeep Singh, Ankur Bhardwaj, Yogesh Chauhan and Rachit Jain, “PWM Strategies in 32-Bit
Micro controller for Interior White LED Down Panel”, International Journal of Computer
Applications, Foundation of Computer Science, Vol. 58, No. 22, pp. 25-32, November 2012.
[8] Olli Heikkila and Janne Silven, “A real-time system for monitoring of cyclists and pedestrians”, In
Proc. of Second IEEE Workshop on Visual Surveillance, pp. 74–81, June 1999
[9] Ya You, Shengrong Gong and Chunping Liu, “Adaptive Moving Object Detection Algorithm
Based on Background Subtraction and Motion Estimation”, IJACT: International Journal of
Advancements in Computing Technology, AICIT, Vol. 5, No. 6, pp. 357-363, 2013.
[10] Minxian Li, Chunxia Zhao, Yingkun Hou and Mingwu Ren, "A New Lane Line Segmentation and
Detection Method based on Inverse Perspective Mapping", JDCTA: International Journal of
Digital Content Technology and its Applications, AICIT, Vol. 5, No. 4, pp. 230-236, 2011.
[11] Anuar Mikdad Muad, Aini Hussain, Salina Abdul Samad, Mohd, Marzuki Mustaffa and
Burhanuddin Yeop Majlis, “Implementation of Inverse Perspective Mapping Algorithm for the
Development of an Automatic Lane Tracking System”, TENCON 2004: The 2004 IEEE Region
10 Conference, pp. 207-211, 2004.
[12] Hyunsuk Cho and Oh-Kyong Kwon, “A Backlight Dimming Algorithm for Low Power and High
Image Quality LCD Applications”, IEEE Transactions on Consumer Electronics, IEEE, Vol. 55,
No. 2, pp. 839-844, May 2009.

218