Is it possible to use the TensorFlow Apple SIlicon GPU implementation for mac with CARELESS?
@aaronfinke, Apple has a plugin that makes the Apple Silicon GPU available to TensorFlow. It’s not something I have in my tests or documentation, but I’m happy to support it if it’s a feature that users want. I have to say though that I just tried it on an M3 Pro. While TensorFlow picks up and uses the GPU just fine, it’s actually slower with the GPU than without (~8 iterations / s with vs 15 without GPU on my hewl ssad benchmark). You can make a new conda env to try it out:
conda create -yn metal python=3.12
conda activate metal
pip install -u pip
pip install careless
pip install tensorflow-metal
You can confirm it worked by running careless devices:
(metal) PC103400:hewl_ssad kmdalton$ careless devices
Careless version 0.5.4
###############################################
# TensorFlow can access the following devices #
###############################################
- CPU: /physical_device:CPU:0
- GPU: /physical_device:GPU:0
You should see a GPU device listed if everything went to plan.
Thanks, I got it working too with some noodling. I also found the GPU run to be slower than the CPU (same Mac specs as you). I was hoping for some performance benefit… c’est la vie.
Yeah I’m honestly not sure whether to expect it to run faster on metal or not. I don’t know enough about Apple CPUs. Sorry it wasn’t a big performance win, but thanks for trying it out!
I’m back to looking at this after @rahewitt 's talk at the DIALS workshop. With Claude’s assistance I have mapped the ML parts of TensorFlow to PyTorch/MPS which is much better optimized for Apple Silicon/Metal than TensorFlow, which has many issues still. Initial tests look good, and I’m getting proper GPU acceleration. Will report back once I have done more tests.
Hi @aaronfinke, there is a complete pytorch port of careless on this branch: GitHub - rs-station/careless at pytorch-migration · GitHub. That might make a better starting point for you.
That is to say, I do plan to port careless to PyTorch, but it hasn’t hit main yet. It’d be great if we could coordinate efforts!
ok I’ll check out that branch, and fork careless so you can see what I’ve come up with. I’ll run some benchmarks too
With this fork I get ~2.2x speedup vs CPU on my Macbook pro M3 with the HEWL test dataset