CARELESS on MacOS GPU/Metal

Is it possible to use the TensorFlow Apple SIlicon GPU implementation for mac with CARELESS?

@aaronfinke, Apple has a plugin that makes the Apple Silicon GPU available to TensorFlow. It’s not something I have in my tests or documentation, but I’m happy to support it if it’s a feature that users want. I have to say though that I just tried it on an M3 Pro. While TensorFlow picks up and uses the GPU just fine, it’s actually slower with the GPU than without (~8 iterations / s with vs 15 without GPU on my hewl ssad benchmark). You can make a new conda env to try it out:

conda create -yn metal python=3.12
conda activate metal
pip install -u pip
pip install careless
pip install tensorflow-metal

You can confirm it worked by running careless devices:

(metal) PC103400:hewl_ssad kmdalton$ careless devices
Careless version 0.5.4
###############################################
# TensorFlow can access the following devices #
###############################################
 - CPU: /physical_device:CPU:0
 - GPU: /physical_device:GPU:0

You should see a GPU device listed if everything went to plan.

Thanks, I got it working too with some noodling. I also found the GPU run to be slower than the CPU (same Mac specs as you). I was hoping for some performance benefit… c’est la vie.

Yeah I’m honestly not sure whether to expect it to run faster on metal or not. I don’t know enough about Apple CPUs. Sorry it wasn’t a big performance win, but thanks for trying it out!

I’m back to looking at this after @rahewitt 's talk at the DIALS workshop. With Claude’s assistance I have mapped the ML parts of TensorFlow to PyTorch/MPS which is much better optimized for Apple Silicon/Metal than TensorFlow, which has many issues still. Initial tests look good, and I’m getting proper GPU acceleration. Will report back once I have done more tests.

Hi @aaronfinke, there is a complete pytorch port of careless on this branch: GitHub - rs-station/careless at pytorch-migration · GitHub. That might make a better starting point for you.

1 Like

That is to say, I do plan to port careless to PyTorch, but it hasn’t hit main yet. It’d be great if we could coordinate efforts!

1 Like

ok I’ll check out that branch, and fork careless so you can see what I’ve come up with. I’ll run some benchmarks too

1 Like

With this fork I get ~2.2x speedup vs CPU on my Macbook pro M3 with the HEWL test dataset