Imagine working with special cameras that capture light your eyes can't even see—ultraviolet rays that cause sunburn or infrared heat signatures that reveal hidden writing. Or perhaps using specialized cameras sensitive enough to distinguish subtle color variations in paint that look just right under specific lighting. Scientists and engineers do this every day—and the resulting data files are so large, they're drowning in it.
A new compression format called Spectral JPEG XL might finally solve this growing problem in scientific visualization and computer graphics. Researchers Alban Fichet and Christoph Peters of Intel Corporation detailed the format in a recent paper published in the Journal of Computer Graphics Techniques (JCGT). It tackles a serious bottleneck for industries working with these specialized images. These spectral files can contain 30, 100, or more data points per pixel, causing file sizes to balloon into multi-gigabyte territory—making them unwieldy to store and analyze.
When we think of digital images, we typically imagine files that store just three colors: red, green, and blue (RGB). This works well for everyday photos, but capturing the true color and behavior of light requires much more detail. Spectral images aim for this higher fidelity by recording light's intensity not just in broad RGB categories, but across dozens or even hundreds of narrow, specific wavelength bands. This detailed information primarily spans the visible spectrum and often extends into near-infrared and near-ultraviolet regions crucial for simulating how materials interact with light accurately.
Unlike standard RGB images with their three channels, these files store information across numerous channels, each representing the intensity of light within a very specific, narrow band of wavelengths. The paper discusses working with spectral images containing 31 distinct channels and even shows examples with as many as 81 spectral bands.
These channels often need to capture a much wider range of brightness values than typical photos. To handle this, spectral images frequently use high-precision formats like 16-bit or 32-bit floating-point numbers for each channel, enabling High Dynamic Range (HDR) data capture. This is a far cry from standard 8-bit images and is key for accurately representing things like the intense brightness of light sources alongside darker scene elements.
Exploring a world beyond RGB
Why would anyone need this level of wavelength detail in an image? There are many reasons. Car manufacturers want to predict exactly how paint will look under different lighting. Scientists use spectral imaging to identify materials by their unique light signatures. And rendering specialists need it to accurately simulate real-world optical effects like dispersion (rainbows from prisms, for example) and fluorescence.
For instance, past Ars Technica coverage has highlighted how astronomers analyzed spectral emission lines from a gamma-ray burst to identify chemicals in the explosion, how physicists reconstructed original colors in pioneering 19th-century photographs, and how multispectral imaging revealed hidden, centuries-old text and annotations on medieval manuscripts like the Voynich Manuscript, sometimes even uncovering the identities of past readers or scribes through faint surface etchings.
The current standard format for storing this kind of data, OpenEXR, wasn't designed with these massive spectral requirements in mind. Even with built-in lossless compression methods like ZIP, the files remain unwieldy for practical work as these methods struggle with the large number of spectral channels.
Spectral JPEG XL utilizes a technique used with human-visible images, a math trick called a discrete cosine transform (DCT), to make these massive files smaller. Instead of storing the exact light intensity at every single wavelength (which creates huge files), it transforms this information into a different form.
Think of it like this: When you look at a rainbow's color transition, you don't need to record every possible wavelength to understand what you see. The DCT works by converting these smooth wavelength patterns into a set of wave-like patterns (frequency coefficients) that, when added together, re-create the original spectral information.
To borrow an analogy from music, it's roughly similar to how MP3 compression works for audio—it selectively preserves frequencies humans most readily perceive, discarding subtle details most listeners won’t miss. Similarly, Spectral JPEG XL preserves the key patterns that define how light interacts with materials, discarding the less significant details.
Importantly, it then applies a weighting step, dividing higher-frequency spectral coefficients by the overall brightness (the DC component), allowing less important data to be compressed more aggressively. That is then fed into the codec, and rather than inventing a completely new file type, the method uses the compression engine and features of the standardized JPEG XL image format to store the specially prepared spectral data.
Making spectral images easier to work with
According to the researchers, the massive file sizes of spectral images have reportedly been a real barrier to adoption in industries that would benefit from their accuracy. Smaller files mean faster transfer times, reduced storage costs, and the ability to work with these images more interactively without specialized hardware.
The results reported by the researchers seem impressive—with their technique, spectral image files shrink by 10 to 60 times compared to standard OpenEXR lossless compression, bringing them down to sizes comparable to regular high-quality photos. They also preserve key OpenEXR features like metadata and high dynamic range support.
While some information is sacrificed in the compression process—making this a "lossy" format—the researchers designed it to discard the least noticeable details first, focusing compression artifacts in the less important high-frequency spectral details to preserve important visual information.
Of course, there are some limitations. Translating these research results into widespread practical use hinges on the continued development and refinement of the software tools that handle JPEG XL encoding and decoding. Like many cutting-edge formats, the initial software implementations may need further development to fully unlock every feature. It's a work in progress.
And while Spectral JPEG XL dramatically reduces file sizes, its lossy approach may pose drawbacks for some scientific applications. Some researchers working with spectral data might readily accept the trade-off for the practical benefits of smaller files and faster processing. Others handling particularly sensitive measurements might need to seek alternative methods of storage.
For now, the new technique remains primarily of interest to specialized fields like scientific visualization and high-end rendering. However, as industries from automotive design to medical imaging continue generating larger spectral datasets, compression techniques like this could help make those massive files more practical to work with.
