OctreeBTFs – A compact, seamless and distortion-free reflectance representation

Publication date: November 2017
Source:Computers & Graphics, Volume 68
Author(s): Stefan Krumpen, Michael Weinmann, Reinhard Klein
Conventional reflectance acquisition techniques rely on the reconstruction of a closed surface geometry and a subsequent surface parametrization that is required to store the reflectance data parametrized over the surface. Among the main drawbacks of this approach is the need for a suitable surface parametrization which is particularly challenging for inaccuracies in the reconstruction or holes occurring due to missing observations. An inappropriate parametrization, in turn, leads to artifacts such as distortion effects and visible seams which severely decrease the visual quality of the digitized object appearance. Furthermore, standard reflectance representations are either compact and less expressive or not compact and expressive. In this paper, we introduce a compact, accurate, seamless and distortion-free volumetric reflectance representation to address these issues. This novel representation named OctreeBTFs is based on storing surface reflectance in terms of Apparent BRDFs in a grid structure that is adapted to the underlying object geometry. The local coordinate systems required to store and render the local reflectance behavior of anisotropic surface reflectance characteristics are computed based on a novel, biologically inspired local approach, instead of computing them based on the texture coordinates obtained from the uv-parameterization. The resulting octree-based data structure results in a more compact BTF representation that can be rendered in real-time with multiple light sources. Finally, our data structure can also be applied to other reflectance representations or to store other information such as physical properties.

Full article: OctreeBTFs – A compact, seamless and distortion-free reflectance representation

Sketch recognition with few examples

Publication date: December 2017
Source:Computers & Graphics, Volume 69
Author(s): Kemal Tugrul Yesilbek, T. Metin Sezgin
Sketch recognition is the task of converting hand-drawn digital ink into symbolic computer representations. Since the early days of sketch recognition, the bulk of the work in the field focused on building accurate recognition algorithms for specific domains, and well defined data sets. Recognition methods explored so far have been developed and evaluated using standard machine learning pipelines and have consequently been built over many simplifying assumptions. For example, existing frameworks assume the presence of a fixed set of symbol classes, and the availability of plenty of annotated examples. However, in practice, these assumptions do not hold. In reality, the designer of a sketch recognition system starts with no labeled data at all, and faces the burden of data annotation. In this work, we propose to alleviate the burden of annotation by building systems that can learn from very few labeled examples, and large amounts of unlabeled data. Our systems perform self-learning by automatically extending a very small set of labeled examples with new examples extracted from unlabeled sketches. The end result is a sufficiently large set of labeled training data, which can subsequently be used to train classifiers. We present four self-learning methods with varying levels of implementation difficulty and runtime complexities. One of these methods leverages contextual co-occurrence patterns to build verifiably more diverse set of training instances. Rigorous experiments with large sets of data demonstrate that this novel approach based on exploiting contextual information leads to significant leaps in recognition performance. As a side contribution, we also demonstrate the utility of bagging for sketch recognition in imbalanced data sets with few positive examples and many outliers.

Full article: Sketch recognition with few examples

Perception of noise and global illumination: Toward an automatic stopping criterion based on SVM

Publication date: December 2017
Source:Computers & Graphics, Volume 69
Author(s): Nawel Takouachet, Samuel Delepoulle, Christophe Renaud, Nesrine Zoghlami, João Manuel R.S. Tavares
Unbiased global illumination methods based on stochastical techniques provide photorealistic images. However, they are prone to noise that can only be reduced by increasing the number of processed samples. The problem of finding the number of samples that are required in order to ensure that most observers cannot perceive any noise is still an open issue. In this article, we address this problem focusing on visual perception of noise. However, rather than using known perceptual models, we investigate the use of learning approaches classically used in the field of Artificial Intelligence. Hence, we propose to use such approaches to create a model which is able to learn which image highlights perceptual noise. The learning is performed through the use of a database of examples based on experimentations of noise perception with human users. This model can then be used in any progressive stochastic global illumination method in order to find the visual convergence threshold of different parts of an input image.

Full article: Perception of noise and global illumination: Toward an automatic stopping criterion based on SVM

Humans are easily fooled by digital images

Publication date: November 2017
Source:Computers & Graphics, Volume 68
Author(s): Victor Schetinger, Manuel M. Oliveira, Roberto da Silva, Tiago J. Carvalho
Digital images are everywhere, from social media to news and scientific papers. This paper describes an extensive user study to evaluate the ability of an average individual to spot edited images. By design, our study avoids lucky guesses. After observing an image, subjects were asked if it is authentic or not. Whenever a subject indicated that an image has been altered, (s)he had to provide evidence to support the answer by pointing at the suspected region in the image. We collected 17,208 individual answers from 393 volunteers, using 177 images selected from public forensic databases. Our results indicate that the average individual is not good at distinguishing original from edited images, answering correctly on 58% of all images, and only identifying the modified ones 46.5% of the time. This performance is superior to random guessing, but poor compared to results achieved by computational techniques.

Full article: Humans are easily fooled by digital images

Visual analytics of time-varying multivariate ionospheric scintillation data

Publication date: November 2017
Source:Computers & Graphics, Volume 68
Author(s): Aurea Soriano-Vargas, Bruno C. Vani, Milton H. Shimabukuro, João F. G. Monico, Maria Cristina F. Oliveira, Bernd Hamann
We present a clustering-based interactive approach to multivariate data analysis, motivated by the specific needs of scintillation data. Ionospheric scintillation is a rapid variation in the amplitude and/or phase of radio signals traveling through the ionosphere. This spatial and time-varying phenomenon is of great interest since it affects the reception quality of satellite signals. Specialized receivers at strategic regions can track multiple variables related to this phenomenon, generating a database of observations of regional ionospheric scintillation. We introduce a visual analytics solution to support analysis of such data, keeping in mind the general applicability of our approach to similar multivariate data analysis situations. Taking into account typical user questions, we combine visualization and data mining algorithms that satisfy these goals: (i) derive a representation of the variables monitored that conveys their behavior in detail, at multiple user-defined aggregation levels; (ii) provide overviews of multiple variables regarding their behavioral similarity over selected time periods; (iii) support users when identifying representative variables for characterizing scintillation behavior. We illustrate the capabilities of our proposed framework by presenting case studies driven directly by questions formulated by collaborating domain experts.

Full article: Visual analytics of time-varying multivariate ionospheric scintillation data

Anatomical augmented reality with 3D commodity tracking and image-space alignment

Publication date: December 2017
Source:Computers & Graphics, Volume 69
Author(s): Armelle Bauer, Debanga Raj Neog, Ali-Hamadi Dicko, Dinesh K. Pai, François Faure, Olivier Palombi, Jocelyne Troccaz
This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of the current user. Using a single Microsoft V2.0 Kinect (later on referenced as the Kinect), we animate in real-time a user-specific model of internal anatomy according to the user’s motion and we superimpose it onto the user’s color map. Users can visualize their anatomy moving as if they where looking inside their own bodies in real-time. A new calibration procedure to set up and attach a user-specific anatomy to the Kinect body tracking skeleton is introduced. At calibration time, the bone lengths are estimated using a set of poses. By using Kinect data as input, the practical limitation of skin correspondence in prior work is overcome. The generic 3D anatomical model is attached to the internal anatomy registration skeleton, and warped on the depth image using a novel elastic deformer subject to a closest-point registration force and anatomical constraints. The noise in Kinect outputs precludes direct display of realistic human anatomy. Therefore, to enforce anatomical plausibility, a novel filter to reconstruct plausible motions based on fixed bones lengths as well as realistic angular degrees of freedom (DOFs) and limits are introduced. Anatomical constraints, applied to the Kinect body tracking skeleton joints, are used to maximize the physical plausibility of the anatomy motion while minimizing the distance to the raw data. At run-time, a simulation loop is used to attract the bones toward the raw data. Skinning shaders efficiently drag the resulting anatomy to the user’s tracked motion. Our user-specific internal anatomy model is validated by comparing the skeleton with segmented MRI images. A user study is established to evaluate the believability of the animated anatomy. As an extension of [1], we also propose an image-based algorithm that corrects accumulated inaccuracy of the system steps: motion capture, anatomy transfer, image generation and animation. These inaccuracies show up as occlusion and self-occlusion misalignments of the anatomy regions when superimposed between them and on top of the color map. We also show that the proposed work can efficiently reduce these inaccuracies.

Full article: Anatomical augmented reality with 3D commodity tracking and image-space alignment

Local Moebius transformations applied to omnidirectional images

Publication date: November 2017
Source:Computers & Graphics, Volume 68
Author(s): Leonardo Souto Ferreira, Leonardo Sacht, Luiz Velho
This work presents a new method to transform omnidirectional images based on a combination of Moebius transformations in the complex plane and weighting functions that restrict the action of these mappings to regions of interest. The transformations are calculated based on the specification of the image of three points and the weighting functions are designed to achieve specific goals such as local zoom or straight line rectification. Since no optimization or numerical methods are involved, our implementation of the proposed method can be upgraded to reach real-time performance. We provide a user interface and present many results that illustrate the potential of the proposed technique.

Full article: Local Moebius transformations applied to omnidirectional images

Posture-based and action-based graphs for boxing skill visualization

Publication date: December 2017
Source:Computers & Graphics, Volume 69
Author(s): Yijun Shen, He Wang, Edmond S.L. Ho, Longzhi Yang, Hubert P.H. Shum
Automatic evaluation of sports skills has been an active research area. However, most of the existing research focuses on low-level features such as movement speed and strength. In this work, we propose a framework for automatic motion analysis and visualization, which allows us to evaluate high-level skills such as the richness of actions, the flexibility of transitions and the unpredictability of action patterns. The core of our framework is the construction and visualization of the posture-based graph that focuses on the standard postures for launching and ending actions, as well as the action-based graph that focuses on the preference of actions and their transition probability. We further propose two numerical indices, the Connectivity Index and the Action Strategy Index, to assess skill level according to the graph. We demonstrate our framework with motions captured from different boxers. Experimental results demonstrate that our system can effectively visualize the strengths and weaknesses of the boxers.

Full article: Posture-based and action-based graphs for boxing skill visualization

Disparity map estimation and view synthesis using temporally adaptive triangular meshes

Publication date: November 2017
Source:Computers & Graphics, Volume 68
Author(s): Guilherme P. Fickel, Cláudio R. Jung
This paper presents a new method for spatio-temporally coherent disparity map estimation and view interpolation for multiview linear camera arrays based on 2D domain triangulation. In the first frame of the sequence, a 3D mesh is computed for each camera, leading to a spatially coherent view interpolation. For the remaining frames of the sequence, a new scheme is proposed to update the 3D mesh dynamically, by moving, deleting and inserting vertices based on the optical flow and the previously computed disparity map. With this approach it is possible to relate triangles of the mesh across time, and a combination of Hidden Markov Models (HMMs) applied to time-persistent triangles with the Kalman Filter applied to the vertices produces temporally coherent disparity maps and interpolated views. Experimental results indicate that our approach was able to generate visually coherent in-between interpolated views for challenging, real-world videos with natural lighting and camera movement. Also, quantitative evaluations using objective video quality metrics show that our interpolated videos are typically better than competitive approaches.

Full article: Disparity map estimation and view synthesis using temporally adaptive triangular meshes