Jobs Mob..... I'm not really a fan to be honest, but i will reserve my many opinions because i have a honest question.
Vram is always the biggest limiting factor in most AI workloads, i myself have specifically for this a RTX a6000 gpu.
Unfortunately it is prices as if there was a Apple sticker on it and it still tops out at 48gb.
So the new super duper always incredibly overpriced MMMMMega MMMMMMax apple stuff has the advantage of having a unified memory architecture, and can go up to 192gb Dram and with the 4 memory controllers it can do a decent amount of bandwidth
So anyone tried to make a very very big model? Or load a large language model for training? It sounds very ideal to have such a large memory pool, and with the a6000 gpu's costing upwards of 5000 euro's and bigger cards based around the H100 around the 20000, pricewise there is not really a difference.
My assumption of course when you load up to train on the overpriced apple gets stressed that it needs to do real compute, burns and slowdown to sludge and reveals the true nature of the The fruity cargo cult. That it's all show and no real compute.
But hey i could be completely wrong and that it holds up, and then the unified memory architecture seems really nice.
So anyone got one of those maxy apple devices and tried this out?