Carnegie Mellon University
Assorted Pixels : Multi-Sampled Imaging With Structural Models

Shree K Nayar and Srinivasa G. Narasimhan
ECCV, June, 2002.

  • Adobe portable document format (pdf) (538KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Multi-sampled imaging is a general framework for using pixels on an image detector to simultaneously sample multiple dimensions of imaging (space, time, spectrum, brightness, polarization, etc.). The mosaic of red, green and blue spectral filters found in most solid-state color cameras is one example of multi-sampled imaging. We briefly describe how multi-sampling can be used to explore other dimensions of imaging. Once such an image is captured, smooth reconstructions along the individual dimensions can be obtained using standard interpolation algorithms. Typically, this results in a substantial reduction of resolution (and hence image quality). One can extract significantly greater resolution in each dimension by noting that the light fields associated with real scenes have enormous redundancies within them, causing different dimensions to be highly correlated. Hence, multi-sampled images can be better interpolated using local structural models that are learned off- line from a diverse set of training images. The specific type of structural models we use are based on polynomial functions of measured image intensities. They are very effective as well as computationally efficient. We demonstrate the benefits of structural interpolation using three specific applications. These are (a) traditional color imaging with a mosaic of color filters, (b) high dynamic range monochrome imaging using a mosaic of exposure filters, and (c) high dynamic range color imaging using a mosaic of overlapping color and exposure filters.

high dyanmic range, super resolution, multi-sampling


Text Reference
Shree K Nayar and Srinivasa G. Narasimhan, "Assorted Pixels : Multi-Sampled Imaging With Structural Models," ECCV, June, 2002.

BibTeX Reference
   author = "Shree K Nayar and Srinivasa G Narasimhan",
   title = "Assorted Pixels : Multi-Sampled Imaging With Structural Models",
   booktitle = "ECCV",
   month = "June",
   year = "2002",