Full Papers Program

Nestor Salamon, Marcel Lancelle, Elmar Eisemann

Light painting is an artform, where a light source is moved during a long-exposure shot, creating trails resembling a stroke on a canvas. It is very difficult to perform because the light source needs to be moved at the intended speed and along a precise trajectory. Additionally, images can be corrupted by the person moving the light. We propose computational light painting, which avoids such artifacts and is easy to use. Taking a video of the moving light as input, a virtual exposure allows us to draw the intended light positions in a post-process. We support animation, as well as 3D light sculpting, with high-quality results.

R. Arora, I. Darolia, V. P. Namboodiri, K. Singh, A. Bousseau

A hallmark of early stage design is a number of quick-and-dirty sketches capturing design inspirations, model variations and alternate viewpoints of a visual concept. We present SketchSoup, a workflow that allows designers to explore the design space induced by such sketches. We take an unstructured collection of drawings as input, along with a small number of user-provided correspondences as input. We register them using a multi-image matching algorithm, and present them as a 2D interpolation space. By morphing sketches in this space, our approach produces plausible visualizations of shape and viewpoint variations despite the presence of sketch distortions that would prevent standard camera calibration and 3D reconstruction. In addition, our interpolated sketches can serve as inspiration for further drawings, which feed back into the design space as additional image inputs. SketchSoup thus fills a significant gap in the early ideation stage of conceptual design by allowing designers to make better informed choices before proceeding to more expensive 3D modelling and prototyping. From a technical standpoint, we describe an end-to-end system that judiciously combines and adapts various image processing techniques to the drawing domain—where the images are dominated not by colour, shading and texture, but by sketchy stroke contours.

B. Steiner, E. Mousavian, F. M. Saradj, M. Wimmer, P. Musialski

Traditionally, building floor plans are designed by architects with their usability, functionality and architectural aesthetics in mind; however, the structural properties of the distribution of load-bearing walls and columns are usually not taken into account at this stage. In this paper, we propose a novel approach for the design of architectural floor plans by integrating structural layout analysis directly into the planning process. In order to achieve this, we introduce a planning tool which interactively enforces checks for structural stability of the current design, and which on demand proposes how to stabilize it if necessary. Technically, our solution contains an interactive architectural modelling framework as well as a constrained optimization module where both are based on respective architectural rules. Using our tool, an architect can predict already in a very early planning stage whose designs are structurally sound such that later changes due to stability reasons can be prevented. We compare manually computed solutions with optimal results of our proposed automated design process in order to show how much our proposed system can help architects to improve the process of laying out structural models optimally.

Laszlo Szirmay-Kalos, Iliyan Georgiev, Milán Magdics, Balázs Molnár, Dávid Légrády

This paper presents a new stochastic particle model for efficient and unbiased Monte Carlo rendering of heterogeneous participating media. We randomly add and remove material particles to obtain a density with which free flight sampling and transmittance estimation are simple while material particle properties are simultaneously modified to maintain the true expectation of the radiance. We show that meeting this requirement may need the introduction of light particles with negative energy and materials with negative extinction, and provide an intuitive interpretation for such phenomena. Unlike previous unbiased methods, the proposed approach does not require a-priori knowledge of the maximum medium density that is typically difficult to obtain for procedural models. However, the method can benefit from an approximate knowledge of the density, which can usually be acquired on-the-fly with little extra cost and can greatly reduce the variance of the proposed estimators. The introduced mechanism can be integrated in participating media renderers where transmittance estimation and free flight sampling are building blocks. We demonstrate its application in a multiple scattering particle tracer, in transmittance computation, and in the estimation of the inhomogeneous air-light integral.

Pascal Weber, Johannes Hanika, Carsten Dachsbacher

We present a new technique called Multiple Vertex Next Event Estimation, which outperforms current direct lighting techniques in forward scattering, optically dense media with the Henyey-Greenstein phase function. Instead of a one-segment connection from a vertex within the medium to the light source, an entire sub path of arbitrary length is created. This is done by perturbation of a seed path within the Monte Carlo context. Our technique was integrated in a Monte Carlo renderer, combining random walk path tracing with Multiple Vertex Next Event Estimation via multiple importance sampling for an unbiased result. We evaluate this new technique against standard Next Event Estimation and show that it significantly reduces noise and increases performance of multiple scattering renderings in highly anisotropic, optically dense media. Additionally, we propose an extension to spot light sources and discuss performance implications of memory-heavy heterogeneous media.

Binh-Son Hua, Adrien Gruson, Derek Nowrouzezahrai, Toshiya Hachisuka

The most common solutions to the light transport problem rely on either Monte Carlo (MC) integration or density estimation methods, such as uni- & bi-directional path tracing or photon mapping. Recent gradient-domain extensions of MC approaches show great promise; here, gradients of the final image are estimated numerically (instead of the image intensities themselves) with coherent paths generated from a deterministic shift mapping. We extend gradient-domain approaches to light transport simulation based on density estimation. As with previous gradient-domain methods, we detail important considerations that arise when moving from a primal- to gradient-domain estimator. We provide an efficient and straightforward solution to these problems. Our solution supports stochastic progressive density estimation, so it is robust to complex transport effects. We show that gradient-domain photon density estimation converges faster than its primal-domain counterpart, as well as being generally more robust than gradient-domain uni- & bi-directional path tracing for scenes dominated by complex transport.

Stefan Lienhard, Cheryl Lau, Pascal Müller, Peter Wonka, Mark Pauly

We introduce design transformations for rule-based procedural models, e.g., for buildings and plants. Given two or more procedural designs, each specified by a grammar, a design transformation combines elements of the existing designs to generate new designs. We introduce two technical components to enable design transformations. First, we extend the concept of discrete rule switching to rule merging, leading to a very large shape space for combining procedural models. Second, we propose an algorithm to jointly derive two or more grammars, called grammar co-derivation. We demonstrate two applications of our work: we show that our framework leads to a larger variety of models than previous work, and we show fine-grained transformation sequences between two procedural models.

Torsten Hädrich, Oliver Deussen, Bedrich Benes, Sören Pirk

We present a novel system for the interactive modeling of developmental climbing plants with an emphasis on efficient control and plausible physics response. A plant is represented by a set of connected anisotropic particles that respond to the surrounding environment and to their inner state. Each particle stores biological and physical attributes that drive growth and plant adaptation to the environment such as light sensitivity, wind interaction, and physical obstacles. This representation allows for the efficient modeling of external effects that can be induced at any time without prior analysis of the plant structure. In our framework we exploit this representation to provide powerful editing capabilities that allow to edit a plant with respect to its structure and its environment while maintaining a biologically plausible appearance. Moreover, we couple plants with Lagrangian fluid dynamics and model advanced effects, such as the breaking and bending of branches. The user can thus interactively drag and prune branches or seed new plants in dynamically changing environments. Our system runs in real-time and supports up to 20 plant instances with 25k branches in parallel. The effectiveness of our approach is demonstrated through a number of interactive experiments, including modeling and animation of different species of climbing plants on complex support structures.

James Gain, Harry Long, Guillaume Cordonnier, Marie-Paule Cani

One challenge in portraying large-scale natural scenes in virtual environments is specifying the attributes of plants, such as species, size and placement, in a way that respects the features of natural ecosystems, while remaining computationally tractable and allowing user design. To address this, we combine ecosystem simulation with a distribution analysis of the resulting plant attributes to create biome-specific databases, indexed by terrain conditions, such as temperature, rainfall, sunlight and slope. For a specific terrain, interpolated entries are drawn from this database and used to interactively synthesize a full ecosystem, while retaining the fidelity of the original simulations. A painting interface supplies users with semantic brushes for locally adjusting ecosystem age, plant density and variability, as well as optionally picking from a palette of precomputed distributions. Since these brushes are keyed to the underlying terrain properties a balance between user control and real-world consistency is maintained. Our system can be be used to interactively design ecosystems up to 5 km × 5 km in extent, or to automatically generate even larger ecosystems in a fraction of the time of a full simulation, while demonstrating known properties from plant ecology such as succession, self-thinning, and underbrush, across a variety of biomes.

Yeara Kozlov, Derek Bradley, Moritz Baecher, Bernhard Thomaszewski, Thabo Beeler, Markus Gross

Oftentimes facial animation is created separately from overall body motion. Since convincing facial animation is challenging enough in itself, artists tend to create and edit the face motion in isolation. Or if the face animation is derived from motion capture, this is typically performed in a mo-cap booth while sitting relatively still. In either case, recombining the isolated face animation with body and head motion is non-trivial and often results in an uncanny result if the body dynamics are not properly reflected on the face (e.g. the bouncing of facial tissue when running).

We tackle this problem by introducing a simple and intuitive system that allows to add physics to facial blendshape animation. Unlike previous methods that try to add physics to face rigs, our method preserves the original facial animation as closely as possible. To this end, we present a novel simulation framework that uses the original animation as per-frame rest-poses without adding spurious forces. As a result, in the absence of any external forces or rigid head motion, the facial performance will exactly match the artist-created blendshape animation. In addition we propose the concept of blendmaterials to give artists an intuitive means to account for changing material properties due to muscle activation. This system allows to automatically combine facial animation and head motion such that they are consistent, while preserving the original animation as closely as possible. The system is easy to use and readily integrates with existing animation pipelines.

Jaewon Song, Roger Blanco i Ribera, Kyungmin Cho, Mi You, J.P. Lewis, Byungkuk Choi, Junyong Noh

Camille Schreck, Damien Rohmer, Stefanie Hahmann

We propose an efficient method to model paper tearing in the context of interactive modeling. The method uses geometrical information to automatically detect potential starting points of tears. We further introduce a new hybrid geometrical and physical-based method to compute the trajectory of tears while procedurally synthesizing high resolution details of the tearing path using a texture based approach. The results obtained are compared with real paper and with previous studies on the expected geometric paths of paper that tears.

S. Oesau, F. Lafarge, P. Alliez

We present a method for planar shape detection and regularization from raw point sets. The geometric modelling and processing of man‐made environments from measurement data often relies upon robust detection of planar primitive shapes. In addition, the detection and reinforcement of regularities between planar parts is a means to increase resilience to missing or defect‐laden data as well as to reduce the complexity of models and algorithms down the modelling pipeline. The main novelty behind our method is to perform detection and regularization in tandem. We first sample a sparse set of seeds uniformly on the input point set, and then perform in parallel shape detection through region growing, interleaved with regularization through detection and reinforcement of regular relationships (coplanar, parallel and orthogonal). In addition to addressing the end goal of regularization, such reinforcement also improves data fitting and provides guidance for clustering small parts into larger planar parts. We evaluate our approach against a wide range of inputs and under four criteria: geometric fidelity, coverage, regularity and running times. Our approach compares well with available implementations such as the efficient random sample consensus–based approach proposed by Schnabel and co‐authors in 2007.We present a method for planar shape detection and regularization from raw point sets. The geometric modelling and processing of man‐made environments from measurement data often relies upon robust detection of planar primitive shapes. In addition, the detection and reinforcement of regularities between planar parts is a means to increase resilience to missing or defect‐laden data as well as to reduce the complexity of models and algorithms down the modelling pipeline. The main novelty behind our method is to perform detection and regularization in tandem. We first sample a sparse set of seeds uniformly on the input point set, and then perform in parallel shape detection through region growing, interleaved with regularization through detection and reinforcement of regular relationships (coplanar, parallel and orthogonal).

Y.-P. Cao, T. Ju, J. Xu, S.-M. Hu

Sharp edges are important shape features and their extraction has been extensively studied both on point clouds and surfaces. We consider the problem of extracting sharp edges from a sparse set of colour-and-depth (RGB-D) images. The noise-ridden depth measurements are challenging for existing feature extraction methods that work solely in the geometric domain (e.g. points or meshes). By utilizing both colour and depth information, we propose a novel feature extraction method that produces much cleaner and more coherent feature lines. We make two technical contributions. First, we show that intensity edges can augment the depth map to improve normal estimation and feature localization from a single RGB-D image. Second, we designed a novel algorithm for consolidating feature points obtained from multiple RGB-D images. By utilizing normals and ridge/valley types associated with the feature points, our algorithm is effective in suppressing noise without smearing nearby features.

M. Maňák, L. Jirkovsky, I. Kolingerova,

The Connolly surface defines the boundary between a molecular structure and its environment. Its shape depends on the radius of the probe used to inspect the structure. The exploration of surface features is of great interest among chemists because it helps them to better understand and describe processes in the molecular structure. To help chemists better explore these features, we have combined two things together: a fast extraction of Connolly surfaces from a Voronoi diagram of atoms and a fast visualization based on GPU ray casting. Not only the surface but also the volume description is provided by the diagram. This enables to distinguish surface cavities one from another and compute their properties, e.g. the approximate volume, the maximal filling sphere or the maximal probe that can escape from the cavity to the outer environment. Cavities can be filtered out by applying restrictions to these properties. Views behind the surface and surface clipping improve the perception of the complex internal structure. The surface is quickly recomputed for any probe radius, so interactive changes of the probe radius show the development of cavities, especially how and where they merge together or with the outer environment.

Riccardo Roveri, Cengiz Oztireli, Markus Gross

Understanding and generating sampling patterns are fundamental problems for many applications in computer graphics. Ideally, point patterns should conform to the problem at hand with spatially adaptive density and correlations. Although there exist excellent algorithms that can generate point distributions with spatially adaptive density or anisotropy, the pair-wise correlation model, blue noise being the most common, is assumed to be constant throughout the space. Analogously, by relying on possibly modulated pair-wise difference vectors, the analysis methods are designed to study only such spatially constant correlations. In this paper, we present the first techniques to analyze and synthesize point patterns with adaptive density and correlations. This provides a comprehensive framework for understanding and utilizing general point sampling. Starting from fundamental measures from stochastic point processes, we propose an analysis framework for general distributions, and a novel synthesis algorithm that can generate point distributions with spatio-temporally adaptive density and correlations based on a locally stationary point process model. Our techniques also extend to general domains. We illustrate the utility of the new techniques on analysis and synthesis of real-world distributions, image and video sampling, stippling and geometry sampling.

T. Stuyck, F. Da, S. Hadap, P. Dutré

This paper presents a realistic digital oil painting system, specifically targeted at the real-time performance on highly resource-constrained portable hardware such as tablets and iPads. To effectively use the limited computing power, we develop an efficient adaptation of the shallow water equations that models all the characteristic properties of oil paint. The pigments are stored in a multi-layered structure to model the peculiar nature of pigment mixing in oil paint. The user experience ranges from thick shape-retaining strokes to runny diluted paint that reacts naturally to the gravity set by tablet orientation. Finally, the paint is rendered in real time using a combination of carefully chosen efficient rendering techniques. The virtual lighting adapts to the tablet orientation, or alternatively, the front-facing camera captures the lighting environment, which leads to a truly immersive user experience. Our proposed features are evaluated via a user study. In our experience, our system enables artists to quickly try out ideas and compositions anywhere when inspiration strikes, in a truly ubiquitous way. They do not need to carry expensive and messy oil paint supplies.

S. Frey and T. Ertl

We present an approach to adaptively select time steps from time-dependent volume data sets for an integrated and comprehensive visualization. This reduced set of time steps not only saves cost, but also allows to show both the spatial structure and temporal development in one combined rendering. Our selection optimizes the coverage of the complete data on the basis of a minimum-cost flow-based technique to determine meaningful distances between time steps. As both optimal solutions of the involved transport and selection problem are prohibitively expensive, we present new approaches that are significantly faster with only minor deviations. We further propose an adaptive scheme for the progressive incorporation of new time steps. An interactive volume raycaster produces an integrated rendering of the selected time steps, and their computed differences are visualized in a dedicated chart to provide additional temporal similarity information. We illustrate and discuss the utility of our approach by means of different data sets from measurements and simulation.

Xi Zhao, Myung Geol Choi, Taku Komura

In this paper, we propose a novel approach for the classification and retrieval of interactions between human characters and objects. We propose to use the interaction bisector surface (IBS) between the body and the object as a feature of the interaction. We define a multiresolution representation of the body structure, and we compute a correspondence matrix hierarchy that describes which parts of the character’s skeleton take part in the composition of the IBS and how much they contribute to the interaction. Keyframes of the interactions are extracted based on the evolution of the IBS and used to align the query interaction with the interaction in the database by dynamic time warping(DTW). Through the experiment results, we show that our approach outperforms existing techniques in motion classification and retrieval, which implies that the contextual information plays a significant role for scene and interaction description. Our method also shows better performance than techniques that compute the distance between the joint positions and feature points of the objects, or the relative transformation between the local coordinate systems of the body parts an the object, thanks to the fact our method does not require aligning the objects. Our method can be applied for automatic recognition of human action and surveillance.

Rene Weller, Nicole Debowski, Gabriel Zachmann

We define a novel geometric predicate and a class of objects that enables us to prove a linear bound on the number of intersecting polygon pairs for colliding 3D objects in that class. Our predicate is relevant both in theory and in practice: it is easy to check and it needs to consider only the geometric properties of the individual objects – it does not depend on the configuration of a given pair of objects. In addition, it characterizes a practically relevant class of objects: we checked our predicate on a large database of real-world 3D objects and the results show that it holds for all but the most pathological ones. Our proof is constructive in that it is the basis for a novel collision detection algorithm that realizes this linear complexity also in practice. Additionally, we present a parallelization of this algorithm with a worst-case running time that is independent of the number of polygons. Our algorithm is very well suited not only for rigid but also for deformable and even topology-changing objects, because it does not require any complex data structures or pre-processing. We have implemented our algorithm on the GPU and the results show that it is able to find in real-time all colliding polygons for pairs of deformable objects consisting of more than 200k triangles, including self-collisions.

L. Gao, S.-Y. Chen, Y.-K. Lai, S. Xia

Shape interpolation has many applications in computer graphics such as morphing for computer animation. In this paper, we propose a novel data-driven mesh interpolation method. We adapt patch-based linear rotational invariant coordinates to effectively represent deformations of models in a shape collection, and utilize this information to guide the synthesis of interpolated shapes. Unlike previous data-driven approaches, we use a rotation/translation invariant representation which defines the plausible deformations in a global continuous space. By effectively exploiting the knowledge in the shape space, our method produces realistic interpolation results at interactive rates, outperforming state-of-the-art methods for challenging cases. We further propose a novel approach to interactive editing of shape morphing according to the shape distribution. The user can explore the morphing path and select example models intuitively and adjust the path with simple interactions to edit the morphing sequences. This provides a useful tool to allow users to generate desired morphing with little effort. We demonstrate the effectiveness of our approach using various examples.

Tobias Günther, Markus Gross

Traditionally, vector field visualization is concerned with 2D and 3D flows. Yet, many concepts can be extended to general dynamical systems, including the higher-dimensional problem of modeling the motion of finite-sized objects in fluids. In the steady case, the trajectories of these so-called inertial particles appear as tangent curves of a 4D or 6D vector field. These higher-dimensional flows are difficult to map to lower-dimensional spaces, which makes their visualization a challenging problem. The approach we follow is to focus on vector field topology, which allows to study asymptotic particle behavior. As recent work on the 2D case has shown, both extraction and classification of isolated critical points depend on the underlying particle model. In this paper, we aim for a model-independent classification technique, which we apply to two different particle models in not only 2D, but also 3D cases. We show that the classification can be done by performing an eigenanalysis of the spatial derivatives’ velocity subspace of the higher-dimensional 4D or 6D flow. We construct glyphs that depict not only the types of critical points, but also encode the directional information given by the eigenvectors. We show that the eigenvalues and eigenvectors of the inertial phase space have sufficient symmetries and structure so that they can be depicted in 2D or 3D, instead of 4D or 6D.

Tobias Günther, Holger Theisel, Markus Gross

Displaying geometry in flow visualization is often accompanied by occlusion problems, making it difficult to perceive information that is relevant in the respective application. Balancing occlusion avoidance and the selection of meaningful geometry was recognized to be a view-dependent, global optimization problem. The recent opacity optimization technique solves a bounded-variable least-squares problem, which minimizes energy terms for the reduction of occlusion, background clutter, adding smoothness and regularization. The original technique operates on an object-space discretization and was shown for line and surface geometry. Recently, it has been extended to volumes, where it was solved locally per ray by dropping the smoothness energy term and replacing it by pre-filtering the importance measure. In this paper, we pick up the idea of splitting the opacity optimization problem into two smaller problems. The first problem is a minimization with analytic solution, and the second problem is a smoothing of the obtained minimizer in object-space. Thereby, the minimization problem can be solved locally per pixel, making it possible to consistently combine all geometry types (points, lines and surfaces) in a single optimization framework. We call this decoupled opacity optimization and apply it to a number of steady 3D vector fields.

S.-H. Kim, Y.-W. Tai, J.-Y. Lee, J. Park, I. S. Kweon

In this paper, we present a new framework to determine up front orientations and detect salient views of 3D models. The salient viewpoint to human preferences is the most informative projection with correct upright orientation. Our method utilizes two Convolutional Neural Network (CNN) architectures to encode category-specific information learnt from a large number of 3D shapes and 2D images on the web. Using the first CNN model with 3D voxel data, we generate a CNN shape feature to decide natural upright orientation of 3D objects. Once a 3D model is upright-aligned, the front projection and salient views are scored by category recognition using the second CNN model. The second CNN is trained over popular photo collections from internet users. In order to model comfortable viewing angles of 3D models, a category-dependent prior is also learnt from the users. Our approach effectively combines category-specific scores and classical evaluations to produce a data-driven viewpoint saliency map. The best viewpoints from the method are quantitatively and qualitatively validated with more than 100 objects from 20 categories. Our thumbnail images of 3D models are the most favoured among those from different approaches.

Philipp Herholz, Felix Haase, Marc Alexa

We define Voronoi cells and centroids based on heat diffusion. These heat cells and heat centroids coincide with the common definitions in Euclidean spaces. On curved surfaces they compare favorably with definitions based on geodesics: they are smooth and can be computed in a stable way with a single linear solve. We analyze the numerics of this approach and can show that diffusion diagrams converge quadratically against the smooth case under mesh refinement, which is better than other common discretization of distance measures in curved spaces. By factorizing the system matrix in a preprocess, computing Voronoi diagrams or centroids amounts to just back-substitution. We show how to localize this operation so that the complexity is linear in the size of the cells and not the underlying mesh. We provide several example applications that show how to benefit from this approach.

C. Brandt, L. Scandolo, E. Eisemann, K. Hildebrandt

We propose a framework for the spectral processing of tangential vector fields on surfaces. The basis is a Fourier-type representation of tangential vector fields that associates frequencies with tangential vector fields. To implement the representation for piecewise constant tangential vector fields on triangle meshes, we introduce a discrete Hodge–Laplace operator that fits conceptually to the prominent cotan discretization of the Laplace–Beltrami operator. Based on the Fourier representation, we introduce schemes for spectral analysis, filtering and compression of tangential vector fields. Moreover, we introduce a spline-type editor for modelling of tangential vector fields with interpolation constraints for the field itself and its divergence and curl. Using the spectral representation, we propose a numerical scheme that allows for real-time modelling of tangential vector fields.

J. El Sayeh Khalil, A. Munteanu, L. Denis, P. Lambert, R. Van de Walle

This paper presents a novel wavelet-based transform and coding scheme for irregular meshes. The transform preserves geometric features at lower resolutions by adaptive vertex sampling and retriangulation, resulting in more accurate subsampling and better avoidance of smoothing and aliasing artefacts. By employing octree-based coding techniques, the encoding of both connectivity and geometry information is decoupled from any mesh traversal order, and allows for exploiting the intra-band statistical dependencies between wavelet coefficients. Improvements over the state of the art obtained by our approach are three-fold: (1) improved rate–distortion performance over Wavemesh and IPR for both the Hausdorff and root mean square distances at low-to-mid-range bitrates, most obvious when clear geometric features are present while remaining competitive for smooth, feature-poor models; (2) improved rendering performance at any triangle budget, translating to a better quality for the same runtime memory footprint; (3) improved visual quality when applying similar limits to the bitrate or triangle budget, showing more pronounced improvements than rate–distortion curves.

Joep Moritz, Stuart James, Tom Haines, Tobias Ritschel, Tim Weyrich

The most common, easiest and most resource-friendly way to add spatial variation of appearance to computer-generated images is texture mapping: a frontal image of a material is captured with a camera and projected onto the virtual geometry. As the image is of finite size and virtual worlds are large, the image typically needs to be tiled, which can lead to distracting and unnatural repetitions. While these are typically removed manually by expert users, we here suggest a way to make every texture-like photo a tileable texture. To this end, we first devise a measure of perceived stationarity, that captures how tileable a texture is. Second, and core of this work, is an algorithm to prescribe a desired stationarity on a given texture exemplar by texture re-synthesis. Using a perceptual linearization of texture stationarity makes controlling the tileability trade-off between uniformity and spatial variation intuitive. A typical application is texturing-without- repetition from casual photos in a mobile interactive application such as a computer game.

Martin Kolář, Kurt Debattista, Alan Chalmers

This paper presents the results of a user study which quantifies the relative and absolute quality of example-based texture synthesis algorithms. In order to allow such evaluation, a list of texture properties is compiled, and a minimal representative set of textures is selected to cover these. Six texture synthesis methods are compared against each other and a reference on a selection of twelve textures by non-expert participants (N = 67). Results demonstrate certain algorithms successfully solve the problem of texture synthesis for certain textures, but there are no satisfactory results for other types of texture properties. The presented textures and results make it possible for future work to be subjectively compared, thus facilitating the development of future texture synthesis methods.

Yang Zhou, Huajie Shi, Dani Lischinski, Minglun Gong, Johannes Kopf, Hui Huang

Many interesting real-world textures are inhomogeneous and/or anisotropic. An inhomogeneous texture is one where various visual properties exhibit significant changes across the texture’s spatial domain. Examples include perceptible changes in surface color, lighting, local texture pattern and/or its apparent scale, and weathering effects, which may vary abruptly, or in a continuous fashion. An anisotropic texture is one where the local patterns exhibit a preferred orientation, which also may vary across the spatial domain. While many example-based texture synthesis methods can be highly effective when synthesizing uniform (stationary) isotropic textures, synthesizing highly non-uniform textures, or ones with spatially varying orientation, is a considerably more challenging task, which so far has remained under explored. In this paper, we propose a new method for automatic analysis and controlled synthesis of such textures. Given an input texture exemplar, our method generates a source guidance map comprising: (i) a scalar progression channel that attempts to capture the low frequency spatial changes in color, lighting, and local pattern combined, and (ii) a direction field that captures the local dominant orientation of the texture. Having augmented the texture exemplar with this guidance map, users can exercise better control over the synthesized result by providing easily specified target guidance maps, which are used to constrain the synthesis process.

Karl Haubenwallner, Hans-Peter Seidel, Markus Steinberger

In this paper, we show that genetic algorithms (GA) can be used to control the output of procedural modeling algorithms. We propose an efficient way to encode the choices that have to be made during a procedural generation as a hierarchical genome representation. In combination with mutation and reproduction operations specifically designed for controlled procedural modeling, our GA can evolve a population of individual models close to any high-level goal. Possible scenarios include a volume that should be filled by a procedurally grown tree or a painted silhouette that should be followed by the skyline of a procedurally generated city. These goals are easy to set up for an artist compared to the tens of thousands of variables that describe the generated model and are chosen by the GA. Previous approaches for controlled procedural modeling either use Reversible Jump Markov Chain Monte Carlo (RJMCMC) or Stochastically-Ordered Sequential Monte Carlo (SOSMC) as workhorse for the optimization. While RJMCMC converges slowly, requiring multiple hours for the optimization of larger models, it produces high quality models. SOSMC shows faster convergence under tight time constraints for many models, but can get stuck due to choices made in the early stages of optimization. Our GA shows faster convergence than SOSMC and generates better models than RJMCMC in the long run.

Jan Beneš, Tom Kelly, Filip Děchtěrenko, Jaroslav Křivánek, Pascal Mueller

H. Hua

It is a challenge for shape grammars to incorporate spatial hierarchy and interior connectivity of buildings in early design stages. To resolve this difficulty, we developed a bi-directional procedural model: the forward process constructs the derivation tree with production rules, while the backward process realizes the tree with shapes in a stepwise manner (from leaves to the root). Each inverse-derivation step involves essential geometric-topological reasoning. With this bi-directional framework, design constraints and objectives are encoded in the grammar-shape translation. We conducted two applications. The first employs geometric primitives as terminals and the other uses previous designs as terminals. Both approaches lead to consistent interior connectivity and a rich spatial hierarchy. The results imply that bespoke geometric-topological processing helps shape grammar to create plausible, novel compositions. Our model is more productive than hand-coded shape grammars, while it is less computation-intensive than evolutionary treatment of shape grammars.

Sheldon Andrews, Marek Teichmann, Paul Kry

J. Huang, Q. Wang, M. Fratarcangeli, K. Yan, C. Pelachaud

Inverse kinematics (IK) equations are usually solved through approximated linearizations or heuristics. These methods lead to character animations that are unnatural looking or unstable because they do not consider both the motion coherence and limits of human joints. In this paper, we present a method based on the formulation of multi-variate Gaussian distribution models (MGDMs), which precisely specify the soft joint constraints of a kinematic skeleton. Each distribution model is described by a covariance matrix and a mean vector representing both the joint limits and the coherence of motion of different limbs. The MGDMs are automatically learned from the motion capture data in a fast and unsupervised process. When the character is animated or posed, a Gaussian process synthesizes a new MGDM for each different vector of target positions, and the corresponding objective function is solved with Jacobian-based IK. This makes our method practical to use and easy to insert into pre-existing animation pipelines. Compared with previous works, our method is more stable and more precise, while also satisfying the anatomical constraints of human limbs. Our method leads to natural and realistic results without sacrificing real-time performance.

Y. Wang, G. Li, Z. Zeng, H. He

Compactly representing time-varying geometries is an important issue in dynamic geometry processing. This paper proposes a framework of sparse localized decomposition for given animated meshes by analyzing the variation of edge lengths and dihedral angles (LAs) of the meshes. It first computes the length and dihedral angle of each edge for poses and then evaluates the difference (residuals) between the LAs of an arbitrary pose and their counterparts in a reference one. Performing sparse localized decomposition on the residuals yields a set of components which can perfectly capture local motion of articulations. It supports intuitive articulation motion editing through manipulating the blending coefficients of these components. To robustly reconstruct poses from altered LAs, we devise a connection-map-based algorithm which consists of two steps of linear optimization. A variety of experiments show that our decomposition is truly localized with respect to rotational motions and outperforms state-of-the-art approaches in precisely capturing local articulated motion.

Or Litany, Emanuele Rodola, Alex Bronstein, Michael Bronstein

We propose an efficient procedure for calculating partial dense intrinsic correspondence between deformable shapes performed entirely in the spectral domain. Our technique relies on the recently introduced partial functional maps formalism and on the joint approximate diagonalization (JAD) of the Laplace-Beltrami operators previously introduced for matching non-isometric shapes. We show that a variant of the JAD problem with an appropriately modified coupling term (surprisingly) allows to construct quasi-harmonic bases localized on the latent corresponding parts. This circumvents the need to explicitly compute the unknown parts by means of the cumbersome alternating minimization used in the previous approaches, and allows performing all the calculations in the spectral domain with constant complexity independent of the number of shape vertices. We provide an extensive evaluation of the proposed technique on standard non-rigid correspondence benchmarks and show state-of-the-art performance in various settings, including partiality, clutter, and the presence of topological noise.

Dorian Nogneng, Maks Ovsjanikov

We consider the problem of non-rigid shape matching, and specifically the functional maps framework that was recently proposed to find correspondences between shapes. A key step in this framework is to formulate descriptor preservation constraints that help to encode the information (e.g., geometric or appearance) that must be preserved by the unknown map. In this paper, we show that considering descriptors as linear operators acting on functions through multiplication, rather than as simple scalar-valued signals, allows to extract significantly more information from a given descriptor and ultimately results in a more accurate functional map estimation. Namely, we show that descriptor preservation constraints can be formulated via commutativity with respect to the unknown map, which can be conveniently encoded by considering relations between matrices in the discrete setting. As a result, when the vector space spanned by the descriptors has a dimension smaller than that of the reduced basis, our optimization may still provide a fully-constrained system leading to accurate point-to-point correspondences, while previous methods might not. We demonstrate on a wide variety of experiments that our approach leads to significant improvement for functional map estimation by helping to reduce the number of necessary descriptor constraints by an order of magnitude, even given an increase in the size of the reduced basis.

D. Ceylan, M. Dang, N. J. Mitra, B. Neubert, M. Pauly

Understanding patterns of variation from raw measurement data remains a central goal of shape analysis. Such an understanding reveals which elements are repeated, or how elements can be derived as structured variations from a common base element. We investigate this problem in the context of 3D acquisitions of buildings. Utilizing a set of template models, we discover geometric similarities across a set of building elements. Each template is equipped with a deformation model that defines variations of a base geometry. Central to our algorithm is a simultaneous template matching and deformation analysis that detects patterns across building elements by extracting similarities in the deformation modes of their matching templates. We demonstrate that such an analysis can successfully detect structured variations even for noisy and incomplete data.

Radek Danecek, Endri Dibra, Cengiz Oztireli, Remo Ziegler, Markus Gross

3D garment capture is an important component for various applications such as free-view point video, virtual avatars, online shopping, and virtual cloth fitting. Due to the complexity of the deformations, capturing 3D garment shapes requires controlled and specialized setups. A viable alternative is image-based garment capture. Capturing 3D garment shapes from a single image, however, is a challenging problem and the current solutions come with assumptions on the lighting, camera calibration, complexity of human or mannequin poses considered, and more importantly a stable physical state for the garment and the underlying human body. In addition, most of the works require manual interaction and exhibit high run-times. We propose a new technique that overcomes these limitations, making garment shape estimation from an image a practical approach for dynamic garment capture. Starting from synthetic garment shape data generated through physically based simulations from various human bodies in complex poses obtained through Mocap sequences, and rendered under varying camera positions and lighting conditions, our novel method learns a mapping from rendered garment images to the underlying 3D garment model. This is achieved by training Convolutional Neural Networks (CNN-s) to estimate 3D vertex displacements from a template mesh with a specialized loss function. We illustrate that this technique is able to recover the global shape of dynamic 3D garments from a single image under varying factors such as challenging human poses, self occlusions, various camera poses and lighting conditions, at interactive rates. Improvement is shown if more than one view is integrated. Additionally, we show applications of our method to videos.

Liwen Hu, Derek Bradley, Hao Li, Thabo Beeler

Physical simulation has long been the approach of choice for generating realistic hair animations in CG. A constant drawback of simulation, however, is the necessity to manually set the physical parameters of the simulation model in order to get the desired dynamic behavior. To alleviate this, researchers have begun to explore methods for reconstructing hair from the real world and even to estimate the corresponding simulation parameters through the process of inversion. So far, however, these methods have had limited applicability, because dynamic hair capture can only be played back without the ability to edit, and solving for simulation parameters can only be accomplished for static hairstyles, ignoring the dynamic behavior. We present the first method for capturing dynamic hair and automatically determining the physical properties for simulating the observed hairstyle in motion. Since our dynamic inversion is agnostic to the simulation model, the proposed method applies to virtually any hair simulation technique, which we demonstrate using two state-of-the-art hair simulation models. The output of our method is a fully simulation-ready hairstyle, consisting of both the static hair geometry as well as its physical properties. The hairstyle can be easily edited by adding additional external forces, changing the head motion, or re-simulating in completely different environments, all while remaining faithful to the captured hairstyle.

T. Inglis, M.-L. Eckert, J. Gregson, N. Thuerey

We apply a novel optimization scheme from the image processing and machine learning areas, a fast Primal-Dual method, to achieve controllable and realistic fluid simulations. While our method is generally applicable to many problems in fluid simulations, we focus on the two topics of fluid guiding and separating solid-wall boundary conditions. Each problem is posed as an optimization problem and solved using our method, which contains acceleration schemes tailored to each problem. In fluid guiding, we are interested in partially guiding fluid motion to exert control while preserving fluid characteristics. With our method, we achieve explicit control over both large-scale motions and small-scale details which is valuable for many applications, such as level-of-detail adjustment (after running the coarse simulation), spatially varying guiding strength, domain modification, and resimulation with different fluid parameters. For the separating solid-wall boundary conditions problem, our method effectively eliminates unrealistic artefacts of fluid crawling up solid walls and sticking to ceilings, requiring few changes to existing implementations. We demonstrate the fast convergence of our Primal-Dual method with a variety of test cases for both model problems.

Graham Fyffe, Koki Nagano, Loc Huynh, Shunsuke Saito, Jay Busch, Andrew Jones, Hao Li, Paul Debevec

We present a fully automatic and parallelizable technique for passive facial performance capture that directly reconstructs dynamic details on a common face topology from multi-view stereo. We present a multi-view stereo reconstruction technique that directly produces a complete high-fidelity head model with consistent facial mesh topology. While existing techniques decouple the process of point cloud estimation and facial tracking, our framework jointly optimizes for stereo constraints and a consistent mesh parameterization. Our method is therefore free from drifts and fully parallelizable for dynamic facial performance capture. We produce highly detailed facial geometries with artist-quality UV maps. Our facial model also includes secondary elements such as eyeballs, mouth pockets, nostrils, and the back of the head. Our approach consists of warping an initial textured template model to match the multi-view input images of the subject, while satisfying cross-view stereo constraints and cross-subject and pose consistencies using a combination of 2D landmark detection, optical flow, and a combination of surface and volumetric Laplacian deformation model. We process high-resolution input images and assume flat illumination condition to ensure photometric consistency during warping. Since no scene flow is ever computed between frames, our method can be trivially parallelized by processing each frame independently. Accurate rigid head pose can also be extracted using a PCA-based dimension reduction and denoising scheme. We demonstrate highly accurate performance capture results of challenging head motions and complex facial expressions around eye and mouth regions. While the quality of our results are on par with the current state-of-the-art, our approach can be fully parallelized, does not suffer from drift, and does not require the construction of a complex template model for tracking.

Amit Haim Bermano, Markus Billeter, Daisuke Iwai, Anselm Grundhöfer

We propose the first system for live dynamic augmentation of human faces. Using projector-based illumination, we alter the appearance of human performers during novel performances. The key challenge of live augmentation is latency — an image is generated according to a specific pose, but is displayed on a different facial configuration by the time it is projected. Therefore, our system aims at reducing latency during every step of the process, from capture, through processing, to projection. Using infrared illumination, an optically and computationally aligned high-speed camera detects facial orientation as well as expression. The estimated expression blendshapes are mapped onto a lower dimensional space, and the facial motion and non-rigid deformation are estimated, smoothed and predicted through adaptive Kalman filtering. Finally, the desired appearance is generated interpolating precomputed offset textures according to time, global position, and expression. We have evaluated our system through an optimized CPU and GPU prototype, and demonstrated successful low latency augmentation for different performers and performances, in various speeds. In contrast to existing methods, the presented system is the first method which fully supports dynamic facial projection mapping without the requirement of any physical tracking markers and incorporates facial expressions.

Martin Klaudiny, Steven McDonagh, Derek Bradley, Thabo Beeler, Kenny Mitchell

We present a real-time multi-view facial capture system facilitated by synthetic training imagery. Our method is able to achieve high-quality markerless facial performance capture in real-time from multi-view helmet camera data, employing an actor specific regressor. The regressor training is tailored to specified actor appearance and we further condition it for the expected illumination conditions and the physical capture rig by generating the training data synthetically. In order to leverage the information present in live imagery, which is typically provided by multiple cameras, we propose a novel multi-view regression algorithm that uses multi-dimensional random ferns. We show that higher quality can be achieved by regressing on multiple video streams than previous approaches that were designed to operate on only a single view. Furthermore, we evaluate possible camera placements and propose a novel camera configuration that allows to mount cameras outside the field of view of the actor, which is very beneficial as the cameras are then less of a distraction for the actor and allow for an unobstructed line of sight to the director and other actors. Our new real-time facial capture approach has immediate application in on-set virtual production, in particular with the ever-growing demand for motion-captured facial animation in visual effects and video games.

Teofilo Dutra, Ricardo Marques, Joaquim Bento Cavalcante-Neto, Creto A. Vidal, Julien Pettre

Most recent crowd simulation algorithms equip agents with a synthetic vision component for steering. They offer promising perspectives through a more realistic simulation of the way humans navigate according to their perception of the surrounding environment. In this paper, we propose a new perception/motion loop to steering agents along collision free trajectories that significantly improves the quality of vision-based crowd simulators. In contrast with previous solutions – which make agents avoid collisions in a purely reactive way – we suggest exploring the full range of possible adaptations and retaining the locally optimal one. To this end, we introduce a cost function, based on perceptual variables, which estimates an agent’s situation considering both the risks of future collision and a desired destination. We then compute the partial derivatives of that function with respect to all possible motion adaptations. The agent then adapts its motion by following the gradient. This paper has thus two main contributions: the definition of a general purpose control scheme for steering synthetic vision-based agents, and the proposition of cost functions for evaluating the perceived danger of the current situation. We demonstrate improvements in several cases.

Timo von Marcard, Bodo Rosenhahn, Michael Black, Gerard Pons-Moll

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables motion capture using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall.

Z. Ren, P. Charalambous, J. Bruneau, Q. Peng, J. Pettré

Crowd simulators are commonly used to populate movie or game scenes in the entertainment industry. Even though it is crucial to consider the presence of groups for the believability of a virtual crowd, most crowd simulations only take into account individual characters or a limited set of group behaviors. We introduce a unified solution that allows for simulations of crowds that have diverse group properties such as social groups, marches, tourists and guides, etc. We extend the Velocity Obstacle approach for agent-based crowd simulations by introducing Velocity Connection; the set of velocities that keep agents moving together while avoiding collisions and achieving goals. We demonstrate our approach to be robust, controllable, and able to cover a large set of group behaviors.

Sema Berkiten, Maciej Halber, Justin Solomon, Chongyang Ma, Hao Li, Szymon Rusinkiewicz

The visual richness of computer graphics applications is frequently limited by the difficulty of obtaining high-quality, detailed 3D models. This paper proposes a method for realistically transferring details (specifically, displacement maps) from existing high-quality 3D models to simple shapes that may be created with easy-to-learn modeling tools. Our key insight is to use metric learning to find a combination of geometric features that successfully predicts detail-map similarities on the source mesh, and use the learned feature combination to drive the detail transfer. The latter uses a variant of multi-resolution non-parametric texture synthesis, augmented by a high-frequency detail transfer step in texture space. We demonstrate that our technique can successfully transfer details among a variety of shapes including furniture and clothing.

Nico Schertler, Manfred Buchroithner, Stefan Gumhold

Quantitative analysis of cave systems represented as 3D models is becoming more and more important in the field of cave sciences. One open question is the rigorous identification of chambers in a data set, which has a deep impact on subsequent analysis steps such as size calculation. This affects the international recognition of a cave since especially record-holding caves bear significant tourist attraction potential. In the past, chambers have been identified manually, without any clear definition or guidance. While experts agree on core parts of chambers in general, their opinions may differ in more controversial areas. Since this process is heavily subjective, it is not suited for objective quantitative comparison of caves. Therefore, we present a novel fully-automatic curve skeleton-based chamber recognition algorithm that has been derived from requirements from field experts. We state the problem as a binary labeling problem on a curve skeleton and find a solution through energy minimization. A thorough evaluation of our results with the help of expert feedback showed that our algorithm matches real-world requirements very closely and is thus suited as the foundation for any quantitative cave analysis system.

F. Reichl, J. Weiss, R. Westermann

We describe how the pipeline for 3D online reconstruction using commodity depth and image scanning hardware can be made scalable for large spatial extents and high scanning resolutions. Our modified pipeline requires less than 10% of the memory that is required by previous approaches at similar speed and resolution. To achieve this, we avoid storing a 3D distance field and weight map during online scene reconstruction. Instead, surface samples are binned into a high-resolution binary voxel grid. This grid is used in combination with caching and deferred processing of depth images to reconstruct the scene geometry. For pose estimation, GPU ray-casting is performed on the binary voxel grid. A one-to-one comparison to level-set ray-casting in a distance volume indicates slightly lower pose accuracy. To enable unlimited spatial extents and store acquired samples at the appropriate level of detail, we combine a hash map with a hierarchical tree representation.

Yoonsang Lee, Taesoo Kwon

Nicolas Bonneel, James Tompkin, Oliver Wang, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, Hanspeter Pfister

Visual formats have advanced beyond single-view images and videos: 3D movies are commonplace, researchers have developed multi-view navigation systems, and VR is helping to push light field cameras to mass market. However, editing tools for these media are still nascent, and even simple filtering operations, like color correction or stylization, are problematic: naively applying image filters per frame or per view rarely produces satisfying results due to time and space inconsistencies. We address this issue with a method that preserves and stabilizes filter effects, while being agnostic to the inner working of the filter. Our algorithm captures filter effects in the gradient domain, then uses input frame gradients as a reference to impose temporal and spatial consistency. Our least-squares formulation adds minimal overhead compared to processing the data naively. Further, when filter cost is high, we introduce a filter transfer strategy that reduces the number of per-frame filtering computations by an order of magnitude, with only a small reduction in visual quality. We demonstrate our algorithm on several camera array formats including stereo videos and light fields.

Q. Zheng, C. Zheng

Rendering with full lens model can offer images with photorealistic lens effects, but it leads to high computational costs. This paper proposes a novel camera lens model, NeuroLens, to emulate the imaging of real camera lenses through a data-driven approach. The mapping of image formation in a camera lens is formulated as imaging regression functions (IRFs), which map input rays to output rays. IRFs are approximated with neural networks, which compactly represent the imaging properties and support parallel evaluation on a graphics processing unit (GPU). To effectively represent spatially varying imaging properties of a camera lens, the input space spanned by incident rays is subdivided into multiple subspaces and each subspace is fitted with a separate IRF. To further raise the evaluation accuracy, a set of neural networks is trained for each IRF and the output is calculated as the average output of the set. The effectiveness of the NeuroLens is demonstrated by fitting a wide range of real camera lenses. Experimental results show that it provides higher imaging accuracy in comparison to state-of-the-art camera lens models, while maintaining the high efficiency for processing camera rays.

Oliver Nalbach, Hans-Peter Seidel, Tobias Ritschel

This paper proposes a pipeline to accurately acquire, efficiently reproduce and intuitively manipulate phosphorescent appearance. In contrast to common appearance models, a model of phosphorescence needs to account for temporal change (decay) and previous illumination (saturation). For reproduction, we propose a rate equation that can be efficiently solved in combination with other illumination in a mixed integro-differential equation system. We describe an acquisition system to measure spectral coefficients of this rate equation for actual materials. Our model is evaluated by comparison to photographs of actual phosphorescent objects. Finally, we propose an artist-friendly interface to control the behavior of phosphorescent materials by specifying spatio-temporal appearance constraints.

Mickael Ribardiere, Benjamin Bringier, Daniel Meneveaux, Lionel Simonot

This paper focuses on microfacet reflectance models, and more precisely on the definition of a new and more general distribution function, which includes both Beckmann’s and GGX distributions widely used in the computer graphics community. Therefore, our model makes use of an additional parameter γ, which controls the distribution function slope and tail height. It actually corresponds to a bivariate Student’s t-distribution in slopes space and it is presented with the associated analytical formulation of the geometric attenuation factor derived from Smith representation. We also provide the analytical derivations for importance sampling isotropic and anisotropic materials. As shown in the results, this new representation offers a finer control of a wide range of materials, while extending the capabilities of fitting parameters with captured data.

Guillaume Loubet, Fabrice Neyret

Sebastian Maisch, Timo Ropinski

Rendering translucent materials in real time is usually done by using surface diffusion and/or (translucent) shadow maps. The downsides of these approaches are, that surface diffusion cannot handle translucency effects that show up when rendering thin objects, and that translucent shadow maps are only available for point light sources. Furthermore, translucent shadow maps introduce limitations to shadow mapping techniques exploiting the same maps. In this paper we present a novel approach for rendering translucent materials at interactive frame rates. Our approach allows for an efficient calculation of translucency with native support for general illumination conditions, especially area and environment lighting, at high accuracy. The proposed technique’s only parameter is the used diffusion profile, and thus it works out of the box without any parameter tuning. Furthermore, it can be used in combination with any existing surface diffusion techniques to add translucency effects. Our approach introduces Spatial Adjacency Maps that depend on precalculations to be done for fixed meshes. We show that these maps can be updated in real time to also handle deforming meshes and that our results are of superior quality as compared to other well known real-time techniques for rendering translucency.

Kumar Moneish, Vineet Gandhi, Rémi Ronfard, Michael Gleicher

Recordings of stage performances are easy to capture with a high-resolution camera, but are difficult to watch because the actors faces are too small. We present an approach to automatically create a split screen video that transforms these recordings to show both the context of the scene as well as close-up details of the actors. Given a static recording of a stage performance and tracking information about the actors positions, our system generates videos showing a focus+context view based on computed close-up camera motions using crop-and zoom. The key to our approach is to compute these camera motions such that they are cinematically valid close-ups and to ensure that the set of views of the different actors are properly coordinated and presented. We pose the computation of camera motions as convex optimization that creates detailed views and smooth movements, subject to cinematic constraints such as not cutting faces with the edge of the frame. Additional constraints link the close up views of each actor, causing them to merge seamlessly when actors are close. Generated views are placed in a resulting layout that preserves the spatial relationships between actors. We demonstrate our results on a variety of staged theater performances.

Nicholas Waldin, Manuela Waldner, Ivan Viola

Drawing the user’s gaze to an important item in an image or a graphical user interface is a common challenge. Usually, some form of highlighting is used, such as a clearly distinct color or a border around the item. Flicker can also be very salient, but is often perceived as annoying. In this paper, we explore high-frequency flicker (60 to 72 Hz) to guide the user’s attention in an image. At such high frequencies, the critical flicker frequency (CFF) threshold is reached, which makes the flicker appear to fuse into a stable signal. However, the CFF is not uniform across the visual field, but is higher in the peripheral vision at normal lighting conditions. We show that high-frequency flicker, using personalized attributes like patch size and luminance, can be easily detected by observers in the peripheral vision, but is hardly visible in the foveal vision when users directly look at the flickering patch. We demonstrate that this property can be used to draw the user’s attention to important image regions using a standard active stereo computer monitor. In an uncalibrated visual search task, users could in a crowded image easily spot the specified search targets flickering with very high frequency. They also reported that high-frequency flicker was distracting when they had to attend to another Region, while it was hardly annoying when looking at the flickering Region itself.

C. Harvey, K. Debattista, T. Bashford-Rogers, A. Chalmers

A major challenge in generating high-fidelity virtual environments (VEs) is to be able to provide realism at interactive rates. The high-fidelity simulation of light and sound is still unachievable in real time as such physical accuracy is very computationally demanding. Only recently has visual perception been used in high-fidelity rendering to improve performance by a series of novel exploitations; to render parts of the scene that are not currently being attended to by the viewer at a much lower quality without the difference being perceived. This paper investigates the effect spatialized directional sound has on the visual attention of a user towards rendered images. These perceptual artefacts are utilized in selective rendering pipelines via the use of multi-modal maps….

J. Tsukamoto, D. Iwai, K. Kashima

This paper proposes a novel shadow removal technique for cooperative projection system based on spatiotemporal prediction. In our previous work, we proposed a distributed feedback algorithm, which is implementable in cooperative projection environments subject to data transfer constraints between components. A weakness of this scheme is that the compensation is conducted in each pixel independently. As a result, spatiotemporal information of the environmental change cannot be utilized even if it is available. In view of this, we specifically investigate the situation where some of the projectors are occluded by a moving object whose one-frame-ahead behaviour is predictable. In order to remove the resulting shadow, we propose a novel error propagating scheme that is still implementable in a distributed manner and enables us to incorporate the prediction information of the obstacle. It is demonstrated theoretically and experimentally that the proposed method significantly improves the shadow removal performance in comparison to the previous work.