GPU acceleration on Mac and Windows

What runs on your computer

All models run through ONNX Runtime, the same engine on every platform. ONNX Runtime hands the work to an execution provider: the CPU, Core ML on a Mac, or DirectML on Windows. GPU acceleration only moves the search model, the heaviest part of indexing. Faces stay on the processor whatever you choose.

Should you turn it on?

• Yes, to try it, if you're about to index a large library (thousands of photos) and want to keep using your computer meanwhile, or want less fan noise and battery drain.
• No need for small libraries or day-to-day scans: after the first index, Aubora only reads new photos, and the processor handles that quickly.
• Turn it off if anything looks wrong. The processor path is the reference.

On a Mac: Core ML

Which Macs

Macs with Apple Silicon (M1, M2, M3, M4 and later). On Intel Macs the option is disabled: the GPU version of the model must be compiled and cached on disk, and the ONNX Runtime build available for Intel Macs can't keep that cache, so it would recompile at every launch.

Turn it on

1. Open Settings → Advanced → Acceleration.
2. Turn on Use GPU acceleration (CoreML).
3. Aubora downloads the GPU version of the search model once (about 600 MB, in 16-bit precision), then compiles it for your Mac's GPU. That takes a few minutes; a progress note shows each step, and you can keep using Aubora meanwhile.
4. The compiled model is cached on disk, so later launches skip the compile.

What to expect

• Speed: about the same as the processor. In our tests on an M4, both took about 170 ms per photo for the search model.
• Load on your Mac: much lower. On the processor, indexing keeps several cores busy; on the GPU it drops to roughly one core. The rest of the Mac stays responsive, and the fans and battery are under less load.
• Only for large scans: loading the model onto the GPU takes a moment at the start of a scan (the status shows "Loading AI models (CoreML…)"). So Aubora uses the GPU only for scans of at least 2,000 photos by default; smaller scans stay on the processor. You can change the threshold right under the toggle: Use the GPU only for scans of at least … photos.

Why not the Neural Engine?

Apple's Neural Engine gave wrong results for this model on some Macs during our tests, without any error message. Aubora therefore asks Core ML to use the GPU and CPU only, in the "ML Program" format with fixed input sizes: the configuration that matched the processor's results on every photo we tested.

On Windows: DirectML

Which PCs

An x64 PC running Windows 11 (what Aubora supports; DirectML itself works from Windows 10 version 1903) with a DirectX 12 graphics card. According to Microsoft, that covers:

• NVIDIA GeForce GTX 600 series and newer;
• AMD Radeon HD 7000 series (GCN) and newer;
• Intel integrated graphics from 4th-generation Core (Haswell) and newer.

On Windows on ARM (Snapdragon laptops) the option is disabled: there's no DirectML build of ONNX Runtime for ARM yet. Support for these PCs' graphics and AI chips is planned. In a virtual machine there's usually no DirectX 12 graphics card visible, so Aubora stays on the processor.

Turn it on

1. Open Settings → Advanced → Acceleration.
2. Turn on Use GPU acceleration (DirectML).
3. Aubora downloads the GPU version of the search model once, then checks it on your graphics card against the processor (see below).
4. It applies from the next scan, whatever its size.

What to expect

• Speed: with a dedicated graphics card, the search model should run faster than on the processor. Integrated graphics gain less. We're measuring it on real hardware during the beta.
• Laptops with two graphics chips (an integrated one and a dedicated one): DirectML uses Windows' default adapter, usually the one driving the main display. Aubora doesn't let you pick the other one yet.

How Aubora keeps the results correct

A GPU computes with 16-bit numbers where the processor uses 32-bit ones, and GPU drivers sometimes compute a model wrongly without reporting an error. Aubora guards against both:

• Checked against the processor. When it loads the model on the GPU, Aubora runs the same input on the GPU and on the processor and compares the two results. If they don't match closely (a cosine similarity under 0.99), it drops the GPU and stays on the processor, and tells you.
• 16-bit is enough for search. On a real library, the 16-bit model's results matched the 32-bit model's (cosine similarity 1.0) and returned the same top 10 for every search tested. Your search results don't change.
• Automatic fallback. If the download, the compile, or the check fails, Aubora keeps working on the processor. Nothing is lost, and your index stays valid.

Turn it off

Turn off the toggle in Settings → Advanced → Acceleration. On a Mac, Aubora offers to also delete the cached GPU model to free the disk space it uses; if you keep it, turning the GPU back on is instant. Scans then run on the processor again.

Messages you may see

Questions

Still stuck? What to do if you're stuck