November 9, 2024
10 Hello Image
Efficient image processing is crucial for any application, given the increasing resolutions and growing data volumes. Even small optimizations in this pipeline can save hours, days, or even weeks of processing time. Additionally, flexible access to image conversion and manipulation utilities across various languages and environments is a valuable feature.
The Image interface is a central component of image handling in Tellusim. It supports loading and saving 2D, 3D, Cube, 2D Array, and Cube Array images, as well as format/type conversion, component/region extraction, scaling, and mipmap generation. All heavy operations are optimized with SIMD and multithreading. The extension system allows for custom format and conversion operation support. The Python API ensures compatibility with popular libraries such as NumPy, PyTorch, and Pillow. The Image interface is simple to use and fully compatible with all supported programming languages.
The following Python snippets showcase basic image operations that can be useful for batch image processing.
Getting image information, including Exif metadata, without loading image content:
# fast operation without content loading
image.info("image.png")
print(image.description)
Performing basic operations with image:
# load Image from file image.load("image.png") print(image.description) # swap red and blue components image.swap(0, 2) # rotate image by 90 degrees CCW image = image.getRotated(-1) # convert image to RGBA format image = image.toFormat(FormatRGBAu8n) # crop image image = image.getRegion(Region(40, 150, 64, 94)) # upscale image using default Cubic filter image = image.getResized(image.size * 4) # create mipmap chain using default mipmap filter image = image.getMipmapped(Image.FilterMip, Image.FlagGamma) # save image image.save("test_basic.dds") print(image.description)
The ImageSampler interface provides access to individual pixels of a specific image layer, mipmap, or face. A high-order Catmull-Rom filter is available for high-quality image resampling when needed. The following snippet demonstrates how to create a simple procedural image:
# create new image
image.create2D(FormatRGBu8n, 512, 256)
# create image sampler from the first image layer
sampler = ImageSampler(image)
# fill image
color = ImageColor(255)
for y in range(image.height):
for x in range(image.width):
v = ((x ^ y) & 255) / 32.0
color.r = int(math.cos(Pi * 1.0 + v) * 127.5 + 127.5)
color.g = int(math.cos(Pi * 0.5 + v) * 127.5 + 127.5)
color.b = int(math.cos(Pi * 0.0 + v) * 127.5 + 127.5)
sampler.set2D(x, y, color)
# save image
image.save("test_xor.png")
print(image.description)
Conversions between panorama and cube formats are straightforward. The following example converts an RGB cube image to a panoramic projection:
# create Cube image
image.createCube(FormatRGBu8n, 128)
print(image.description)
# clear image
for face in range(0, 6, 3):
ImageSampler(image, Slice(Face(face + 0))).clear(ImageColor(255, 0, 0))
ImageSampler(image, Slice(Face(face + 1))).clear(ImageColor(0, 255, 0))
ImageSampler(image, Slice(Face(face + 2))).clear(ImageColor(0, 0, 255))
# convert to 2D panorama
# it will be horizonal cross without Panorama flag
image = image.toType(Image.Type2D, Image.FlagPanorama)
image.save("test_panorama.png")
print(image.description)
Our CPU texture encoders are fast and deliver excellent compression quality. Only a single function call is needed to compress a texture to any BC or ASTC format. An Async interface can be supplied to the function for more precise thread control. By default, the compressors utilize all available CPU cores:
# load and resize Image image.load("image.png") image = image.getResized(image.size * 2) # create mipmaps image = image.getMipmapped() # compress image to BC1 format image_bc1 = image.toFormat(FormatBC1RGBu8n) image_bc1.save("test_bc1.dds") print(image_bc1.description) # compress image to BC7 format image_bc7 = image.toFormat(FormatBC7RGBAu8n) image_bc7.save("test_bc7.dds") print(image_bc7.description) # compress image to ASTC4x4 format image_astc44 = image.toFormat(FormatASTC44RGBAu8n) image_astc44.save("test_astc44.ktx") print(image_astc44.description) # compress image to ASTC8x8 format image_astc88 = image.toFormat(FormatASTC88RGBAu8n) image_astc88.save("test_astc88.ktx") print(image_astc88.description)
Python buffer protocol support simplifies data sharing between Tellusim and other Python frameworks. The following snippet demonstrates modifying image content using NumPy operations:
# load image and convert to float32 format image.load("image.png") image = image.toFormat(FormatRGBf32) # create array with specified dimension and format array = numpy.zeros(shape = ( image.width, image.height, 3 ), dtype = numpy.float32) # copy image data into the array image.getData(array) # set inverted data into the image image.setData(1.0 - array) # save inverted image image.save("test_numpy.dds") print(image.description)
The following file formats can be loaded directly: ASTC, BMP, BW, CUR, DDS, DEM, EXR, HDR, HGT, ICO, IMAGE, JPEG, KTX, LA, PBM, PGM, PNG, PPM, PSD, RGB, RGBA, SGI, TGA, and TIFF. The list of supported saving formats excludes only DEM and HGT files. Any other formats can be added via a C++ plugin and will function as native formats.
While it is easy to create such scripts in Python or other supported languages, this can also be avoided by using the Tellusim Image Processing Tool. The command-line options work as a pipeline of operations on the loaded images, dramatically simplifying batch processing. For example, the following command loads all images in the directory, resizes them, creates gamma mipmaps, encodes them to ASTC55, and saves all images with an “_astc” postfix and a .ktx extension:
ts_image *.jpg -scale 0.5 -mipmaps gamma -format astc55rgbau8n -p _astc -e ktx
For GravityMark, we used the following script to convert NASA Topo maps to the appropriate dimensions and formats:
#!/bin/bash SRC=world.topo.bathy.200412 SIZE=8192 SCALE=0.5 NAME=color mkdir -p $NAME.$SIZE ts_image -v -create rgbu8n 86400 43200 \ $SRC.3x21600x21600.A1.png -insert 0 0 -remove \ $SRC.3x21600x21600.A2.png -insert 0 21600 -remove \ $SRC.3x21600x21600.B1.png -insert 21600 0 -remove \ $SRC.3x21600x21600.B2.png -insert 21600 21600 -remove \ $SRC.3x21600x21600.C1.png -insert 43200 0 -remove \ $SRC.3x21600x21600.C2.png -insert 43200 21600 -remove \ $SRC.3x21600x21600.D1.png -insert 64800 0 -remove \ $SRC.3x21600x21600.D2.png -insert 64800 21600 -remove \ -s $SCALE -cube -mipmaps gamma \ -clone -push -face 0 -o $NAME.$SIZE/$NAME.0.jpg -remove -pop \ -clone -push -face 1 -o $NAME.$SIZE/$NAME.1.jpg -remove -pop \ -clone -push -face 2 -o $NAME.$SIZE/$NAME.2.jpg -remove -pop \ -clone -push -face 3 -o $NAME.$SIZE/$NAME.3.jpg -remove -pop \ -clone -push -face 4 -o $NAME.$SIZE/$NAME.4.jpg -remove -pop \ -clone -push -face 5 -o $NAME.$SIZE/$NAME.5.jpg -remove -pop \ -clone -push -format bc1rgbu8n -o $NAME.$SIZE/$NAME.bc1.ktx -remove -pop \ -clone -push -format etc2rgbu8n -o $NAME.$SIZE/$NAME.etc2.ktx -remove -pop \ -clone -push -format astc66rgbau8n -o $NAME.$SIZE/$NAME.astc.ktx -remove -pop
The Image Processing Tool supports fast GPU compression to BC and ASTC formats. To use this, you simply need to specify the “gpu” flag for the “format” operation:
We will compare the performance and quality of the CPU and GPU encoders in the next post. Stay tuned.