https://github.com/MarkMoHR/virtual_sketching
Tip revision: 958efe45a9120b9d467ba7701efba28c11e38f8f authored by Your Name on 21 August 2021, 08:13:15 UTC
Added bilibili links
Added bilibili links
Tip revision: 958efe4
index.html
<!doctype html>
<html lang="en">
<!-- === Header Starts === -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>General Virtual Sketching Framework for Vector Line Art</title>
<link href="./assets/bootstrap.min.css" rel="stylesheet">
<link href="./assets/font.css" rel="stylesheet" type="text/css">
<link href="./assets/style.css" rel="stylesheet" type="text/css">
</head>
<!-- === Header Ends === -->
<body>
<!-- === Home Section Starts === -->
<div class="section">
<!-- === Title Starts === -->
<div class="title">
<b>General Virtual Sketching Framework for Vector Line Art</b>
</div>
<!-- === Title Ends === -->
<div class="author">
<a href="http://mo-haoran.com/" target="_blank">Haoran Mo</a><sup>1</sup>,
<a href="https://esslab.jp/~ess/en/" target="_blank">Edgar Simo-Serra</a><sup>2</sup>,
<a href="http://cse.sysu.edu.cn/content/2537" target="_blank">Chengying Gao</a><sup>*1</sup>,
<a href="https://changqingzou.weebly.com/" target="_blank">Changqing Zou</a><sup>3</sup>,
<a href="http://cse.sysu.edu.cn/content/2523" target="_blank">Ruomei Wang</a><sup>1</sup>
</div>
<div class="institution">
<sup>1</sup>Sun Yat-sen University,
<sup>2</sup>Waseda University,
<br>
<sup>3</sup>Huawei Technologies Canada
</div>
<br>
<div class="institution">
Accepted by <a href="https://s2021.siggraph.org/" target="_blank">ACM SIGGRAPH 2021</a>
</div>
<div class="link">
<a href="https://esslab.jp/publications/HaoranSIGRAPH2021.pdf" target="_blank">[Paper]</a>
<a href="https://github.com/MarkMoHR/virtual_sketching" target="_blank">[Code]</a>
</div>
<div class="teaser">
<img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/teaser6.png" style="width: 100%;">
<br>
<br>
<font size="3">
Given clean line drawings, rough sketches or photographs of arbitrary resolution as input, our framework generates the corresponding vector line drawings directly. As shown in (b), the framework models a virtual pen surrounded by a dynamic window (red boxes), which moves while drawing the strokes. It learns to move around by scaling the window and sliding to an undrawn area for restarting the drawing (bottom example; sliding trajectory in blue arrow). With our proposed stroke regularization mechanism, the framework is able to enlarge the window and draw long strokes for simplicity (top example).
</font>
</div>
</div>
<!-- === Home Section Ends === -->
<!-- === Overview Section Starts === -->
<div class="section">
<div class="title">Abstract</div>
<div class="body">
Vector line art plays an important role in graphic design, however, it is tedious to manually create.
We introduce a general framework to produce line drawings from a wide variety of images,
by learning a mapping from raster image space to vector image space.
Our approach is based on a recurrent neural network that draws the lines one by one.
A differentiable rasterization module allows for training with only supervised raster data.
We use a dynamic window around a virtual pen while drawing lines,
implemented with a proposed aligned cropping and differentiable pasting modules.
Furthermore, we develop a stroke regularization loss
that encourages the model to use fewer and longer strokes to simplify the resulting vector image.
Ablation studies and comparisons with existing methods corroborate the efficiency of our approach
which is able to generate visually better results in less computation time,
while generalizing better to a diversity of images and applications.
</div>
<div class="link">
<a href="https://esslab.jp/publications/HaoranSIGRAPH2021.pdf" target="_blank">[Paper]</a>
<a href="https://dl.acm.org/doi/abs/10.1145/3450626.3459833" target="_blank">[Paper (ACM)]</a>
<a href="https://markmohr.github.io/files/SIG2021/SketchVectorization_SIG2021_supplemental.pdf" target="_blank">[Supplementary]</a>
<a href="https://github.com/MarkMoHR/virtual_sketching" target="_blank">[Code]</a>
<a href="https://drive.google.com/drive/folders/1-hi2cl8joZ6oMOp4yvk_hObJGAK6ELHB?usp=sharing" target="_blank">[All Precomputed Results]</a>
<!-- <a href="" target="_blank">[Presentation (TBD)]</a> -->
</div>
</div>
<!-- === Overview Section Ends === -->
<!-- === Result Section Starts === -->
<div class="section">
<div class="title">Method</div>
<br>
<div class="body">
<p style="text-align:center; font-size:23px; font-weight:bold">Framework Overview<p>
<img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/framework6.png" width="100%">
<br>
<br>
<font size="4">
Our framework generates the parametrized strokes step by step in a recurrent manner.
It uses a dynamic window (dashed red boxes) around a virtual pen to draw the strokes,
and can both move and change the size of the window.
(a) Four main modules at each time step: aligned cropping, stroke generation, differentiable rendering and differentiable pasting.
(b) Architecture of the stroke generation module.
(c) Structural strokes predicted at each step;
movement only is illustrated by blue arrows during which no stroke is drawn on the canvas.
</font>
<br>
<br>
<p style="text-align:center; font-size:23px; font-weight:bold">
Overall Introduction
<p>
<p style="text-align:center; font-size:20px">
(Or watch on <a href="https://www.bilibili.com/video/BV1gM4y1V7i7/" target="_blank">Bilibili</a>)
<br>
👇
<p>
<!-- Adjust the frame size based on the demo (EVERY project differs). -->
<div style="position: relative; padding-top: 50%; text-align: center;">
<iframe src="https://www.youtube.com/embed/gXk3TMceByY" frameborder=0
style="position: absolute; top: 1%; left: 5%; width: 90%; height: 100%;"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
</div>
</div>
</div>
<!-- === Result Section Ends === -->
<!-- === Result Section Starts === -->
<div class="section">
<div class="title">Results</div>
<div class="body">
Our framework is applicable to a diversity of image types, such as clean line drawing images, rough sketches and photographs.
<p style="margin-top: 10pt; text-align:center; font-size:23px; font-weight:bold">Vectorization<p>
<table width="100%" style="margin: 0pt auto; text-align: center; border-collapse: separate; border-spacing: 5pt;">
<tr>
<td width="45%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/clean/muten.png" width="100%"></td>
<td width="10%"></td>
<td width="45%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/clean/muten-black-full-simplest.gif" width="100%"></td>
</tr>
</table>
<br>
<p style="margin-top: 10pt; text-align:center; font-size:23px; font-weight:bold">Rough sketch simplification<p>
<table width="100%" style="margin: 0pt auto; text-align: center; border-collapse: separate; border-spacing: 5pt;">
<tr>
<td width="26%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/rough/rocket.png" width="100%"></td>
<td width="26%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/rough/rocket-blue-simplest.gif" width="100%"></td>
<td width="4%"></td>
<td width="14%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/rough/penguin.png" width="100%"></td>
<td width="14%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/rough/penguin-blue-simplest.gif" width="100%"></td>
</tr>
</table>
<br>
<p style="margin-top: 10pt; text-align:center; font-size:23px; font-weight:bold">Photograph to line drawing<p>
<table width="100%" style="margin: 0pt auto; text-align: center; border-collapse: separate; border-spacing: 5pt;">
<tr>
<td width="23%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/face/1390_input.png" width="100%"></td>
<td width="23%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/face/face-blue-1390-simplest.gif" width="100%"></td>
<td width="8%"></td>
<td width="23%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/face/1190_input.png" width="100%"></td>
<td width="23%"><img src="https://cdn.jsdelivr.net/gh/mark-cdn/CDN-for-works@1.4/files/SIG21/gifs/face/face-blue-1190-simplest.gif" width="100%"></td>
</tr>
</table>
<br>
<p style="margin-top: 10pt; text-align:center; font-size:23px; font-weight:bold">
More Results
<p>
<p style="text-align:center; font-size:20px">
(Or watch on <a href="https://www.bilibili.com/video/BV1pv411N7Yx/" target="_blank">Bilibili</a>)
<br>
👇
<p>
<!-- Adjust the frame size based on the demo (EVERY project differs). -->
<div style="position: relative; padding-top: 50%; text-align: center;">
<iframe src="https://www.youtube.com/embed/Pr6mK9ddXkQ" frameborder=0
style="position: absolute; top: 1%; left: 5%; width: 90%; height: 100%;"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
</div>
<br>
<div class="link">
<a href="https://drive.google.com/drive/folders/1-hi2cl8joZ6oMOp4yvk_hObJGAK6ELHB?usp=sharing" target="_blank">
[Download Our Precomputed Output Results (7MB)]</a>
</div>
</div>
</div>
<!-- === Result Section Ends === -->
<!-- === Result Section Starts === -->
<div class="section">
<div class="title">Presentations</div>
<div class="body">
<p style="margin-top: 10pt; text-align:center; font-size:23px; font-weight:bold">
3-5 minute presentation
<p>
<p style="text-align:center; font-size:20px">
(Or watch on <a href="https://www.bilibili.com/video/BV1S3411q7VX/" target="_blank">Bilibili</a>)
<br>
👇
<p>
<!-- Adjust the frame size based on the demo (EVERY project differs). -->
<div style="position: relative; padding-top: 50%; text-align: center;">
<iframe src="https://www.youtube.com/embed/BSJN1ixacts" frameborder=0
style="position: absolute; top: 1%; left: 5%; width: 90%; height: 100%;"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
</div>
<br>
<div class="link">
👉 15-20 minute presentation:
<a href="https://youtu.be/D_U4e1qh5qc" target="_blank">[YouTube]</a>
<a href="https://www.bilibili.com/video/BV1uU4y1E7Wg/" target="_blank">[Bilibili]</a>
</div>
<div class="link">
👉 30-second fast forward:
<a href="https://youtu.be/d0EbSU_EeFg" target="_blank">[YouTube]</a>
<a href="https://www.bilibili.com/video/BV1vq4y1M7j1/" target="_blank">[Bilibili]</a>
</div>
</div>
</div>
<!-- === Result Section Ends === -->
<!-- === Reference Section Starts === -->
<div class="section">
<div class="bibtex">BibTeX</div>
<pre>
@article{mo2021virtualsketching,
title = {General Virtual Sketching Framework for Vector Line Art},
author = {Mo, Haoran and Simo-Serra, Edgar and Gao, Chengying and Zou, Changqing and Wang, Ruomei},
journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2021)},
year = {2021},
volume = {40},
number = {4},
pages = {51:1--51:14}
}
</pre>
<br>
<div class="bibtex">Related Work</div>
<div class="citation">
<div class="comment">
Jean-Dominique Favreau, Florent Lafarge and Adrien Bousseau.
<strong>Fidelity vs. Simplicity: a Global Approach to Line Drawing Vectorization</strong>. SIGGRAPH 2016.
[<a href="https://www-sop.inria.fr/reves/Basilic/2016/FLB16/fidelity_simplicity.pdf">Paper</a>]
[<a href="https://www-sop.inria.fr/reves/Basilic/2016/FLB16/">Webpage</a>]
<br><br>
</div>
<div class="comment">
Mikhail Bessmeltsev and Justin Solomon.
<strong>Vectorization of Line Drawings via PolyVector Fields</strong>. SIGGRAPH 2019.
[<a href="https://arxiv.org/abs/1801.01922">Paper</a>]
[<a href="https://github.com/bmpix/PolyVectorization">Code</a>]
<br><br>
</div>
<div class="comment">
Edgar Simo-Serra, Satoshi Iizuka and Hiroshi Ishikawa.
<strong>Mastering Sketching: Adversarial Augmentation for Structured Prediction</strong>. SIGGRAPH 2018.
[<a href="https://esslab.jp/~ess/publications/SimoSerraTOG2018.pdf">Paper</a>]
[<a href="https://esslab.jp/~ess/en/research/sketch_master/">Webpage</a>]
[<a href="https://github.com/bobbens/sketch_simplification">Code</a>]
<br><br>
</div>
<div class="comment">
Zhewei Huang, Wen Heng and Shuchang Zhou.
<strong>Learning to Paint With Model-based Deep Reinforcement Learning</strong>. ICCV 2019.
[<a href="https://openaccess.thecvf.com/content_ICCV_2019/papers/Huang_Learning_to_Paint_With_Model-Based_Deep_Reinforcement_Learning_ICCV_2019_paper.pdf">Paper</a>]
[<a href="https://github.com/megvii-research/ICCV2019-LearningToPaint">Code</a>]
<br><br>
</div>
</div>
</div>
<!-- === Reference Section Ends === -->
</body>
</html>
