On-the-fly multi-scale infinite texturing from example

1. Summary

This paper proposes a method for on-the-fly non-periodic infinite texturing of surfaces based on a single image. 

- Pattern repetition is avoided by defining patches within each texture whose content can be changed at run-time.

- Multi-scale is managed by using one input image per represented scale.  

- Undersampling artifacts are avoided by accounting for fine-scale features while colors are transferred between scales. 

- It allows for relief-enhanced rendering and provides a tool for intuitive creation of height maps. This is done using an adhoc local descriptor that measures feature self-similarity in order to propagate height values provided by the use for a few selected texels only. 

It is easy to implement on GPU because it is patch-based system.

2. Features

- Real-time non-periodic infinite surface texturing is achieved by modifying texture tiles at runtime, yet neither introducing visual artifacts nor requiring heavy computations. The memory consumption is also reduced compared to state-of-the-art tiling. 

- Multi-scale texturing consistently blends colors between multiple layers of texture: the color transfer mechanism avoids popping and ghosting artifacts by respecting the features of each scale.

- Relief enhancement is integrated as a height map texture, which is kept consistent with the color texture. 

3. Overview

 In this section, The process for mono-scale texturing will be described. 

3.1 Tile clustering and content selection

 As an input, it is given a set of tiles which are processed separately before the rendering stage. To break the repetition effect, the content of input will be randomly modified on-the-fly. Therefore, A tile partitioning is  pre-computed composed of patches with a set of interchangeable contents. All contents for a given patch have the same shape, but correspond to different locations in the tile.  At rendering, each time a tile is repeated the content of its patches is randomly selected. 

3.2 Fragmentation for multi-scale extension

 For rendering, the colors of two consecutive layers are blended according to the viewing distance. It requires colors to be transferred between scales. To minimize visual inconsistency such as popping, ghosting and blocky artifacts, the color is transferred on pre-computed fragments not on individual texels. Fragments are merely small pieces of textures, whose colors can be modified without introducing seams.  

3.3 Color conversion

 Fragments capture small features with a homogeneous color. which is decisive for multi-scale color transfer. In contrast, patches must encompass several texture features in order to break the repetition while keeping visual feature arrangements. Though fragment and patches have different properties, they have to be correlated each other, because content exchange which is performed per patch and color transfer which is performed per fragment will be combined at rendering. In addition, they share a common objective: hiding seams. It is achieved by exploiting fact that seams are less perceivable in regions of high contrast: fragments are computed such that their borders match high contrasts. 

A dedicated color space, that we call principal variation color space, is designed to meet several objective: to facilitate the building of fragments, and the measure of contrasts and seams. It allows to represent colors compactly and improve multi-scale anti-aliasing. It separates a few dominant colors from local variations for each texture tile. It is used throughout the process, and colors are converted back only at rendering. 

3.4 Relief enhancement

 If a height map is provided for the relief enhancement, it must be considered for the design of patches because content exchange may introduce geometric seams. Since patches are based on fragments which follow dominant colors, we introduce relief at the very beginning, as an additional coordinate to color channels.