Core Image

Core Image is a pixel-accurate, near-realtime, non-destructive image processing technology in Mac OS X. Implemented as part of the QuartzCore framework of Mac OS X 10.4 and later, Core Image provides a plugin-based architecture for applying filters and effects within the Quartz graphics rendering layer.^[1] The framework was later added to iOS in iOS 5.^[2]

Overview

Core Image abstracts the pixel-level manipulation process required when applying a filter to an image, making it simple for applications to implement image transformation capabilities without extensive coding. In a simple implementation, Core Image applies a single Image Filter to the pixel data of a given source to produce the transformed image. Each Image Filter specifies a single transform or effect, either built into Core Image or loaded from a plugin called an Image Unit. Combined with preset or user-defined input parameters, the filter can be applied to the original pixel data without modifying it, thereby providing non-destructive image editing.^[3]^[4]

Like Photoshop, Core Image can apply multiple filters to the same image source. Instead of applying a series of filters individually, Core Image assembles a dynamic instruction pipeline so that only one calculation needs to be applied to the pixel data to achieve a cumulative effect. Applying the pixel operations associated with multiple filters can be achieved simultaneously and without a significant increase in processing time. Regardless of the number of filters, Core Image assembles the code for this instruction pipeline with a just-in-time compiler, which is executed by either the CPU or graphics card's GPU, whichever can perform the calculation faster.^[5]

Filters are written in the Core Image Kernel Language, which shares a subset of commands with OpenGL Shading Language (GLSL).^[6] When a compatible GPU is available, the Core Image compiler writes the instruction pipeline using GLSL, handling buffers and states transparently. Although GPU rendering is preferred^{[citation needed]}, the compiler can operate in a CPU fallback mode, generating commands suitable for the current CPU architecture instead.^[7] CPU fallback uses the vector processing capabilities of the current CPU or CPUs, and it is multi-processor aware. Thus, Core Image performance depends on the GLSL capabilities of the GPU or the processing power of the CPU. With a supported GPU, most effects can be rendered in realtime or near-realtime.^[8]

History and implementation

Core Image was introduced with Mac OS X 10.4.^[9] Early examples of its use can be found in the ripple effect in Dashboard, and Apple's professional digital photography application, Aperture.^[10] Starting with Mac OS X 10.5, any application that implements Image Kit can utilize Core Image.^[11] Preview and iPhoto are common examples.

In 2011, Apple added Core Image to iOS in iOS 5.0.^[2]

The Xcode Tools include Core Image Fun House and Quartz Composer; both utilize Core Image.

The Core Image plugin architecture was inspired by that of Core Audio.^[12]

Pixel accuracy

All pixel processing provided by an Image Unit is performed in a pre-multiplied alpha (RGBA) color space, storing four color channels: red, green, blue, and transparency (alpha). Each color channel is represented by a 32-bit, floating point number. This provides exceptional color depth, far greater than can be perceived by the human eye, as each pixel is represented by a 128-bit vector (four 32-bit color channels). For color spaces of lower bit-depth, the floating-point calculation model employed by Core Image provides exceptional performance, which is useful when processing multiple images or video frames.^[3]^[13]

Supported graphics processors

Any programmable GPU that supports the required OpenGL Shader (GLSL) commands is Core Image capable. Apple has used the following graphics cards to support Core Image GPU processing in Mac OS X 10.4 and Aperture, so the following list could be considered an example of minimum requirements:^[10]^[14]

ATI Mobility Radeon 9700 (Mobility Radeon 9600 is also capable)
ATI Radeon 9550, 9600, 9650, 9600 XT, 9600 Pro, 9700 Pro, 9800 XT, and 9800 Pro
ATI Radeon X600 XT, X600 Pro, X800 XT, X850 XT, X1600, X1900 XT
ATI Radeon HD 2400 XT, HD 2600 Pro, and HD 2600 XT
NVIDIA GeForce FX 5200 Ultra, FX Go 5200
NVIDIA GeForce 6600, 6600 LE, 6800 Ultra DDL, and 6800 GT DDL
NVIDIA GeForce 7300 GT, 7600 GT, and 7800 GT
NVIDIA GeForce 8600M GT & 8800 GT
NVIDIA GeForce 9400M & 9600M GT
NVIDIA GeForce GT 120 & GT 130
NVIDIA GeForce GTX 285
NVIDIA Quadro FX 4500 & FX 5600
NVIDIA Quadro FX 4800
NVIDIA Quadro 4000
NVIDIA GeForce 320M
NVIDIA GeForce GT 330M
NVIDIA GeForce GT 640M
NVIDIA GeForce GT 650M
NVIDIA GeForce GTX 660M
NVIDIA GeForce GTX 675M
NVIDIA GeForce GTX 680MX
NVIDIA GeForce GT 750M
NVIDIA GeForce GT 755M
NVIDIA GeForce GTX 775M
NVIDIA GeForce GTX 780M
Intel GMA 900, GMA 950, or GMA X3100
Intel HD Graphics 3000(Sandy Bridge)
Intel HD Graphics 4000(Ivy Bridge)
Intel HD Graphics 5000(Haswell)
Intel Iris 5100(Haswell)
Intel Iris Pro 5200(Haswell)
Intel HD Graphics 5300(Broadwell)
Intel HD Graphics 6000(Broadwell)
Intel Iris 6100(Broadwell)

Note that any GPU capable of handling Core Image instructions is also Quartz Extreme capable. The requirements for Core Image are greater than those of Quartz Extreme.^[14]

Built-in filters

macOS includes many built-in filters, including the following ones. Mac OS X 10.4 introduced over 100 of them, and Mac OS X 10.5 added to the list.^[3]^[15]

Blurs, like Median, Gaussian, Motion, and Noise
Color Adjustments & Effects, like Exposure, Gamma Adjust, and Sepia Tone
Composition Operations, like Addition and Multiply Blends, or Hard Light
Distortions, like Pinch, Circle Splash, and Vortex
Generators, like Star Shine and Lenticular Halo
Geometry Adjustments, such as Crop, Scale, Rotate, and Affine transformation
Halftone, Tile, and Posterize filters
Transitions, such as Dissolve and Ripple

References

^ "Mac Dev Center - Introduction to Core Image Programming Guide". Retrieved September 20, 2009.
^ ^a ^b "iOS 5.0 API Diffs". Retrieved September 14, 2012.
^ ^a ^b ^c "Apple - Developer - Developing with Core Image". Archived from the original on October 6, 2009. Retrieved September 20, 2009.
^ "Mac Dev Center - Introduction to Core Image Programming Guide - Filter Clients and Filter Creators". Retrieved September 20, 2009.
^ "ArsTechnica - Mac OS X 10.4 Tiger - Page 15". Retrieved September 20, 2009.
^ "Mac Dev Center - Core Image Kernel Language Reference - Introduction". Retrieved September 20, 2009.
^ "Mac Dev Center - Core Image Programming Guide - Core Image Concepts - Core Image and the GPU". Retrieved September 20, 2009.
^ "ArsTechnica - Mac OS X 10.4 Tiger - Page 15". Retrieved April 17, 2007.
^ "Mac Dev Center - Core Image Programming Guide - Core Image Concepts". Retrieved September 20, 2009.
^ ^a ^b "Apple - Aperture - Tech Specs". Retrieved September 20, 2009.
^ "Mac Dev Center - Image Kit Programming Guide - Introduction to Image Kit Programming Guide". Retrieved September 20, 2009.
^ Singh 2006, p. 97.
^ "Mac Dev Center - Core Image Programming Guide - Core Image Concepts - Color Components and Premultiplied Alpha". Retrieved September 20, 2009.
^ ^a ^b "Mac OS X 10.4 - Requirements for Quartz Extreme and Core Image Graphics". Archived from the original on November 9, 2014. Retrieved September 20, 2009.
^ "Core Image Filter Reference". Archived from the original on May 19, 2024. Retrieved July 24, 2024.

Bibliography

Singh, Amit (June 19, 2006). Mac OS X Internals: A Systems Approach. Addison-Wesley Professional. ISBN 978-0-13-270226-3.

External links

Core Image Programming Guide – Mac OS X Developer Library
Apple Previews Mac OS X "Tiger" - announcement of Mac OS X 10.4, including Core Image
Filterpedia - open source Swift app demonstrating almost every available built-in CoreImage filter along with custom GLSL and Metal based filters by Simon Gladman.