Super resolution on video

Javid C. Hsueh
Sinze S. Ho
Kuenbu K. Tsai

Tuesday, March 29, 2011

 Super Resolution
• Improve image quality from low-resolution

Tuesday, March 29, 2011

 Super Resolution(SR) on Video

• Rich information between frames

• Reconstruct a better frame by neighbor

Tuesday, March 29, 2011

 Work progress
• Get image stream : by OpenCV
• 2 SR algorithm
• FGSR, FRSR
• GPU speedup investigation
• video player

Tuesday, March 29, 2011

 FGSR
• Fast & General Super-Resolution
• general video
• fast, cheap, acceptable result
• Zhongding  Jiang,    Tien-­‐tsin  Wong,    Hujun  Bao.  Practical  Super-­‐Resolution  
from  Dynamic  Video  Sequences.  In  Proc.  of  IEEE  CVPR2003.

Tuesday, March 29, 2011

 FGSR algorithm
• Predict  High  Resolution(HR)  =  Bicubic(  F  ) i

• Iteratively  refine  Predict  HR  :

F’i+n  =  Bicubic(Fi+n)
In’    =  optical_flow(  F’i+n  )  
HR  =    HR  +  α  *  diff(  HR  ,  In’  )

α  =  1/  temporal  distance+1

Tuesday, March 29, 2011

 FGSR Result
Bicubic FGSR

Tuesday, March 29, 2011

 FRSR
• Fast & Robust Super-Resolution
• global motion video
• slow, better result
• Sina  Farsiu,  Dirk  Robinson,  Michael  Elad,  Peyman  Milanfar.    Fast  and  
Robust  Multi-­‐Frame  Super-­‐Resolution.  In  IEEE  Transactions  on  Image  
Processing2003.

Tuesday, March 29, 2011

 FRSR algorithm
• Predicted  HR=  Median(    optical_flow(F 0~n)    )

• Iteratively  improve  Predicted  HR  :  

F’i  =  optical_flow(Fi)
Greg  =  GradientRegularization(F’i);
HR  =  HR  +  α*Greg;

Tuesday, March 29, 2011

 FRSR Result
Bicubic FRSR

Tuesday, March 29, 2011

 Evaluation
• PSNR: Peak Signal to Noise Ratio
Ground truth FGSR FRSR

Image

PSNR
(dB)
∞ 14.8428 15.1508

Tuesday, March 29, 2011

 Result 2
Ground truth Bicubic

FGSR FRSR

Tuesday, March 29, 2011

 Bicubic interpolation
CPU vs. GPU
-./0$''"1+%!"#$ʠ
#!!"

'#!"

'!!"

&#!"

&!!"
!"#$%&#'(ʠ

%#!"
*+,"
%!!" -+,"

$#!"

$!!"

#!"

!"
('" $%)" %#(" #$%" $!%'" %!')"
)#*+$%'",$ʠ

Tuesday, March 29, 2011

 Conclusion
• key issue : alignment of frames
• Reconstruct the high resolution image by
iteratively refinement

Tuesday, March 29, 2011

 Review project
• insufficient background knowledge
• non-parallelizable algorithm
• too late to write the cuda code

Tuesday, March 29, 2011