NPRportrait 1.0:A three-level benchmark for non-photorealistic rendering of portraits

Recently,there has been an upsurge of activity in image-based non-photorealistic rendering(NPR),and in particular portrait image stylisation,due to the advent of neural style transfer(NST).However,the state of performance evaluation in this field is poor,especially compared to the norms in the computer vision and machine learning communities.Unfortunately,the task of evaluating image stylisation is thus far not well defined,since it involves subjective,perceptual,and aesthetic aspects.To make progress towards a solution,this paper proposes a new structured,threelevel,benchmark dataset for the evaluation of stylised portrait images.Rigorous criteria were used for its construction,and its consistency was validated by user studies.Moreover,a new methodology has been developed for evaluating portrait stylisation algorithms,which makes use of the different benchmark levels as well as annotations provided by user studies regarding the characteristics of the faces.We perform evaluation for a wide variety of image stylisation methods(both portrait-specific and general purpose,and also both traditional NPR approaches and NST)using the new benchmark dataset.

Example based painting generation

We present an approach for generating paintings on photographic images with the style encoded by the example paintings and adopt representative brushes extracted from the example paintings as the painting primitives. Our system first divides the given photographic image into several regions on which we synthesize a grounding layer with texture patches extracted from the example paintings. Then, we paint those regions using brushes stochastically chosen from the brush library, with further brush color and shape perturbations. The brush direction is determined by a direction field either constructed by a convenient user interactive manner or synthesized from the examples. Our approach offers flexible and intuitive user control over the painting process and style.

A survey of digital paper-cutting

Paper-cutting is one of the most fantastic Chinese folk arts. With the aid of computers, digital paper-cutting has become a popular topic in the field of NPR research. This paper will make a comprehensive survey of related references in four as- pects-image-based paper-cutting, paper-cutting based on computer graphics, paper architectures and paper-cutting animation and it shows that the imitation of the masters＇ artistic styles of paper-cutting will be the trend of future study.

Technical illustration based on 3D CSG models

This paper presents an automatic non-photorealistic rendering approach to generating technical illustration from 3D models. It first decomposes the 3D object into a set of CSG primitives, and then performs the hidden surface removal based on the prioritized list, in which the rendition order of CSG primitives is sorted out by depth. Then, each primitive is illustrated by the pre-defined empirical lighting model, and the system mimics the stroke-drawing by user-specified style. In order to artistically and flexibly modulate the illumination, the empirical lighting model is defined by three major components: parameters of multi-level lighting intensities, parametric spatial occupations for each lighting level, and an interpolation method to calculate the lighting distribution over primitives. The stylized illustration is simulated by a grid-based method, in which we ‘fill’ the desirable pictorial units into the spatial occupation of CSG primitives, instead of “pixel-by-pixel” painting. This region-by-region shading facilitates the simulation of illustration styles.

Stroke Based Painterly Rendering with Mass Data through Auto Warping Generation

Painting is done according to the artist’s style.The most representative of the style is the texture and shape of the brush stroke.Computer simulations allow the artist’s painting to be produced by taking this stroke and pasting it onto the image.This is called stroke-based rendering.The quality of the result depends on the number or quality of this stroke,since the stroke is taken to create the image.It is not easy to render using a large amount of information,as there is a limit to having a stroke scanned.In this work,we intend to produce rendering results using mass data that produces large amounts of strokes by expanding existing strokes through warping.Through this,we have produced results that have higher quality than conventional studies.Finally,we also compare the correlation between the amount of data and the results.

Real-time flow-based video abstraction using OpenCL

A non-photorealistic rendering technique is a method to show various effects different from those of realistic image generation.Of the various techniques,flow-based image abstraction displays the shape and color features well and performs a stylistic visual abstraction.But real-time rendering is impossible when CPU is used because it applies various filtering and iteration methods.In this paper,we present real-time processing methods of video abstraction using open open computing language(OpenCL),technique of general-purpose computing on graphics processing units(GPGPU).Through the acceleration of general-purpose computing(GPU),16 frame-per-second(FPS)or greater is shown to process video abstraction.

A simple,stroke-based method for gesture drawing

Background Gesture drawing is a type of fluid,fast sketch with loose and roughly drawn lines that captures the motion and feeling of a subject.Although style transfer methods,which are able to learn a style from an input image and apply it to a secondary image,can reproduce many styles,they are currently unable to produce the flowing strokes of gesture drawings.Method In this paper,we present a method for producing gesture drawings that roughly depict objects or scenes with loose dancing contours and frantic textures.By following a gradient field,our method adapts stroke-based painterly rendering algorithms to produce long curved strokes.A rough,overdrawn appearance is created through a progressive refinement.In addition,we produce rough hatch strokes by altering the stroke direction.These add optional shading to gesture drawings.Results A wealth of parameters provide users the ability to adjust the output style,from short and rapid strokes to long and fluid strokes,and from swirling to straight lines.Potential stylistic outputs include pen-and-ink and colored pencil.We present several generated gesture drawings and discuss the application of our method to video.Conclusion Our stroke-based rendering algorithm produces convincing gesture drawings with numerous controllable parameters,permitting the creation of a variety of styles.

GPU-based 3D model rendering of chinese ink painting

In computer graphics, non-photorealistic rendering(NPR) is an important branch. NPR technology is to achieve a variety of artistic effects through the computer, such as oil painting, cartoon, watercolor and other effects. The purpose of this paper is to automatically convert the 3D model into two-dimensional Chinese ink painting effect on graphics processing unit(GPU), and has improved the traditional algorithm that has some lacks of rendering effect. The algorithm is divided into two major steps: contour line rendering, and interior rendering. For contour line rendering, on the basis of the traditional extraction of contours, this paper adds self-bold, particle diffusion and other steps. For interior rendering, on the basis of the traditional layered rendering of diffuse lighting, two methods are added. The first method is based on the mean filtering, and the filter kernel is stratified by the principle of percentage-closer soft shadows(PCSS). The second method is Noise texture mapping, to obtain an ink diffusion effect.

Free form deformation based face cartoon stylization

A novel free form based face cartoon stylization method is presented in this paper. First, a face cartoon library with marked feature points is constructed. And then select the input image as the target image and an appropriate cartoon image from cartoon library as the reference image, apply the deformation between the corresponding feature points of the images to the target image. Finally, we apply an image stylization process to the result image. As an alternative method, we also choose an appropriate cartoon image as the target image and the input image as the reference image to apply the free from deformation. The experimental results show that our method is straightforward and quick with diversified styles, delivering more infection.

Controllable multi-domain semantic artwork synthesis

We present a novel framework for the multidomain synthesis of artworks from semantic layouts.One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis.To address this problem,we propose a dataset called ArtSem that contains 40,000 images of artwork from four different domains,with their corresponding semantic label maps.We first extracted semantic maps from landscape photography and used a conditional generative adversarial network(GAN)-based approach for generating high-quality artwork from semantic maps without requiring paired training data.Furthermore,we propose an artwork-synthesis model using domain-dependent variational encoders for high-quality multi-domain synthesis.Subsequently,the model was improved and complemented with a simple but effective normalization method based on jointly normalizing semantics and style,which we call spatially style-adaptive normalization(SSTAN).Compared to the previous methods,which only take semantic layout as the input,our model jointly learns style and semantic information representation,improving the generation quality of artistic images.These results indicate that our model learned to separate the domains in the latent space.Thus,we can perform fine-grained control of the synthesized artwork by identifying hyperplanes that separate the different domains.Moreover,by combining the proposed dataset and approach,we generated user-controllable artworks of higher quality than that of existing approaches,as corroborated by quantitative metrics and a user study.

Video-based running water animation in Chinese painting style

This paper presents a novel algorithm for synthesizing animations of running water, such as water- falls and rivers, in the style of Chinese paintings, for applications such as cartoon making. All video frames are first registered in a common coordinate system, simultaneously segmenting the water from background and computing optical flow of the water. Taking artists＇ advice into account, we produce a painting structure to guide painting of brush strokes. Flow lines are placed in the water following an analysis of variance of optical flow, to cause strokes to be drawn where the water is flowing smoothly, rather than in turbulent areas： this allows a few moving strokes to depict the trends of the water flows. A variety of brush strokes is then drawn using a template determined from real Chinese paintings. The novel contributions of this paper are： a method for painting structure generation for flows in videos, and a method for stroke placement, with the necessary temporal coherence.

Real-Time Watercolor for Animation

We present algorithms that allow for reat-time rendering of 3D-scenes with a watercolor painting appearance,Our approach provides an appropriate simplification of the visual complexity,initates characteristic natural effects of watercolor,and provides two essential painting techniques,the wet-on-wet and the wet-on-dry painting.We concentrate on efficient algorithms based on image space processing rather than on an exact simulation.This allows for the real-time rendering of 3D-scenes.During an animation a high frame-to-frame coherence can be achieved due to a stable segmentation scheme.Finally,we seamlessly integrate a smooth illumination into the watercolor renderings nsing information from the 3D-scenc.

Estimating reflectance and shape of objects from a single cartoon-shaded image

Although many photorealistic relighting methods provide a way to change the illumination of objects in a digital photograph, it is currently difficult to relight digital illustrations having a cartoon shading style. The main difference between photorealistic and cartoon shading styles is that cartoon shading is characterized by soft color quantization and nonlinear color variations that cause noticeable reconstruction errors under a physical reflectance assumption, such as Lambertian reflection. To handle this non-photorealistic shading property, we focus on shading analysis of the most fundamental cartoon shading technique. Based on the color map shading representation, we propose a simple method to determine the input shading as that of a smooth shape with a nonlinear reflectance property. We have conducted simple ground-truth evaluations to compare our results to those obtained by other approaches.

Expressive line drawings of human faces from range images

We propose a novel technique to extract features from a range image and use them to produce a 3D pen-and-ink style portrait similar to a traditional artistic drawing. Unlike most previous template-based, component-based or example-based face sketching methods, which work from a frontal photograph as input, our system uses a range image as input. Our method runs in real-time for models of moderate complexity, allowing the pose and drawing style to be modified interactively. Portrait drawing in our system makes use of occluding contours and suggestive contours as the most important shape cues. However, current 3D feature line detection methods require a smooth mesh and cannot be reliably applied directly to noisy range images. We thus present an improved silhouette line detection algorithm. Feature edges related to the significant parts of a face are extracted from the range image, connected, and smoothed, allowing us to construct chains of line paths which can then be rendered as desired. We also incorporate various portrait-drawing principles to provide several simple yet effective non- photorealistic portrait renderers such as a pen-and-ink shader, a hatch shader and a sketch shader. These are able to generate various life-like impressions in different styles from a user-chosen viewpoint. To obtain satisfactory results, we refine rendered output by smoothing changes in line thickness and opacity. We are careful to provide appropriate visual cues to enhance the viewer＇s comprehension of the human face. Our experimental results demonstrate the robustness and effectiveness of our approach, and further suggest that our approach can be extended to other 3D geometric objects.

Automated pebble mosaic stylization of images

Digital mosaics have usually used regular tiles, simulating historical tessellated mosaics. In this paper, we present a method for synthesizing pebble mosaics, a historical mosaic style in which the tiles are rounded pebbles. We address both the tiling problem,of distributing pebbles over the image plane so as to approximate the input image content, and the problem of geometry, creating a smooth rounded shape for each pebble. We adopt simple linear iterative clustering(SLIC)to obtain elongated tiles conforming to image content,and smooth the resulting irregular shapes into shapes resembling pebble cross-sections. Then, we create an interior and exterior contour for each pebble and solve a Laplace equation over the region between them to obtain height-field geometry. The resulting pebble set approximates the input image while representing full geometry that can be rendered and textured for a highly detailed representation of a pebble mosaic.

Stroke Style Analysis for Painterly Rendering

We propose a novel method that automatically analyzes stroke-related artistic styles of paintings. A set of adaptive interfaces are also developed to connect the style analysis with existing painterly rendering systems, so that the specific artistic style of a template painting can be effectively transferred to the input photo with minimal effort. Different from conventional texture-synthesis based rendering techniques that focus mainly on texture features, this work extracts, analyzes and simulates high-level style features expressed by artists＇ brush stroke techniques. Through experiments, user studies and comparisons with ground truth, we demonstrate that the proposed style-orientated painting framework can significantly reduce tedious parameter adjustment, and it allows amateur users to efficiently create desired artistic styles simply by specifying a template painting.

Feature-preserving color pencil drawings from photographs

Color pencil drawing is well-loved due to its rich expressiveness.This paper proposes an approach for generating feature-preserving color pencil drawings from photographs.To mimic the tonal style of color pencil drawings,which are much lighter and have relatively lower saturation than photographs,we devise a lightness enhancement mapping and a saturation reduction mapping.The lightness mapping is a monotonically decreasing derivative function,which not only increases lightness but also preserves input photograph features.Color saturation is usually related to lightness,so we suppress the saturation dependent on lightness to yield a harmonious tone.Finally,two extremum operators are provided to generate a foreground-aware outline map in which the colors of the generated contours and the foreground object are consistent.Comprehensive experiments show that color pencil drawings generated by our method surpass existing methods in tone capture and feature preservation.

Procedural modeling of water caustics and foamy water for cartoon animation

We propose a method for procedural modeling and animation of cartoon water effects such as water caustics,foamy wake,and longshore currents.In our method we emulate the visual abstraction of these cartoon effects by the use of Voronoi diagrams and the motion abstraction by designing relevant controlling mechanisms corresponding to each effect.Our system enables the creation of cartoon effects withminimal intervention from the animator.Through high-level initial specification,the effects are animated procedurally in the style of hand-drawn cartoons.

Stylized strokes for coherent line drawings

Temporal coherence is one of the central challenges for rendering a stylized line. It is especially difficult for stylized contours of coarse meshes or nonuniformly sampled models, because those contours are polygonal feature edges on the models with no continuous correspondences between frames. We describe a novel and simple technique for constructing a 2D brush path along a 3D contour. We also introduce a 3D parameter propagation and re-parameterization procedure to construct stroke paths along the 2D brush path to draw coherently stylized feature lines with a wide range of styles. Our method runs in real-time for coarse or non-uniformly sampled models, making it suitable for interactive applications needing temporal coherence.

Real-Time Image and Video Artistic Style Rendering System Based on GPU

Aiming at the practical engineering application of video stylization,in this paper, a GPU-based video art stylization algorithm is proposed, and areal-time video art stylization rendering system is implemented. The four mostcommon artistic styles including cartoon, oil painting, pencil painting and watercolorpainting are realized in this system rapidly. Moreover, the system makesgood use of the GPU’s parallel computing characteristics, transforms the videostylized rendering algorithm into the texture image rendering process, acceleratesthe time-consuming pixel traversal processing in parallel and avoids the loop processingof the traditional CPU. Experiments show that the four art styles achievedgood results, and the system has a good interactive experience.