Multi-stream video fusion using local principal components analysis

Ravi K. Sharma, Misha Pavel, Todd K. Leen

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations


We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.

Original languageEnglish (US)
Pages (from-to)717-725
Number of pages9
JournalProceedings of SPIE - The International Society for Optical Engineering
Issue number2
StatePublished - 1998
Externally publishedYes
EventProceedings of the 1998 Conference on Infrared Technology and Applications XXIV. Part 1 (of 2) - San Diego, CA, USA
Duration: Jul 19 1998Jul 24 1998

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Multi-stream video fusion using local principal components analysis'. Together they form a unique fingerprint.

Cite this