https://github.com/vsitzmann/deepoptics
Tip revision: defbb975309a6a3f3d2a86b92e82d02156ab213e authored by Vincent Sitzmann on 04 May 2021, 05:27:42 UTC
Updated paper link
Updated paper link
Tip revision: defbb97
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="">
<meta name="author" content="Vincent Sitzmann">
<!-- <link rel="shortcut icon" href="../../assets/ico/favicon.ico"> -->
<title>Deepoptics</title>
<!-- Bootstrap core CSS -->
<link href="bootstrap.min.css" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="offcanvas.css" rel="stylesheet">
</head>
<body>
<div class="container">
<div class="jumbotron">
<h2>End-to-end optimization of optics and image processing</h2>
<p class="abstract">Joint optimization of whole computer vision pipelines for true domain-specific imaging</p>
<p iclass="authors"><a href="http://stanford.edu/~sitzmann/">Vincent Sitzmann*</a>, <a href="http://stanford.edu/~stevend2/">Steven Diamond*</a>, <a href="https://www.cs.ubc.ca/~evanpeng/">Yifan Peng*</a>, <a href="">Xiong Dun</a>, <a href="http://web.stanford.edu/~boyd">Stephen Boyd</a>, <a href="">Wolfgang Heidrich</a>, <a href="">Felix Heide</a>, <a href="https://stanford.edu/~gordonwz/">Gordon Wetzstein</a></p>
<p><a class="btn btn-primary" href="https://github.com/vsitzmann/deepoptics">Code</a> <a class="btn btn-primary" href="https://drive.google.com/file/d/19jSB7d52fyXcFbzGo63Z6zzj70hs5s2Q/view">Paper</a></p>
</div>
<img src="img/teaser.jpg" style="width:100%; margin-right:-20px; margin-top:-10px;">
<hr>
<div>
<p>In typical cameras the optical system is designed first; once it is fixed, the parameters in the image processing algorithm are tuned to get good image reproduction. In contrast to this sequential design approach, we consider joint optimization of an optical system (for example, the physical shape of the lens) together with the parameters of the reconstruction algorithm. We build a fully-differentiable simulation model that maps the true source image to the reconstructed one. The model includes diffractive light propagation, depth and wavelength-dependent effects, noise and nonlinearities, and the image post-processing. We jointly optimize the optical parameters and the image processing algorithm parameters so as to minimize the deviation between the true and reconstructed image, over a large set of images. We implement our joint optimization method using autodifferentiation to efficiently compute parameter gradients in a stochastic optimization algorithm. We demonstrate the efficacy of this approach by applying it to achromatic extended depth of field and snapshot super-resolution imaging.</p>
</div>
<div class="section">
<h3>Submission Video</h3>
<iframe width="1025" height="560" src="https://www.youtube.com/embed/iJdsxXOfqvw" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
</div>
<div class="section">
<h3>Related work</h3>
<hr>
<p>Check out our related work on differentiable image reconstruction:</p>
<p><a href="https://arxiv.org/abs/1705.08041">Unrolled optimization with Deep Priors</a></p>
<p><a href="https://arxiv.org/abs/1701.06487">Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data</a></p>
</div>
<h3>Bibtex</h3>
<hr>
<div class="bibtexsection">
@article{sitzmann2018end,
title={End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging},
author={Sitzmann, Vincent
and Diamond, Steven
and Peng, Yifan
and Dun, Xiong
and Boyd, Stephen
and Heidrich, Wolfgang
and Heide, Felix
and Wetzstein, Gordon},
journal={ACM Transactions on Graphics (TOG)},
volume={37},
number={4},
pages={114},
year={2018},
publisher={ACM}
}
</div>
<!-- <div style="padding-top:30px"> -->
<div>
<h3>References</h3>
<hr>
<p>For full details, please refer to our paper.</p>
<div class="list-group">
<a href="https://drive.google.com/file/d/19jSB7d52fyXcFbzGo63Z6zzj70hs5s2Q/view" class="list-group-item">
<img src="img/paper_thumbnails.png" style="width:100%; margin-right:-20px; margin-top:-10px;">
<h3 class="list-group-item-heading">End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging</h4>
<p class="list-group-item-text">Vincent Sitzmann*, Steven Diamond*, Yifan Peng*, Xiong Dun, Stephen Boyd, Wolfgang Heidrich, Felix Heide, Gordon Wetzstein</p>
</a>
</div>
<p>Send feedback and questions to <a href="http://web.stanford.edu/~sitzmann/">Vincent Sitzmann</a></p>
</div>
</div><!--/.container-->
</body>
</html>
