4-bit diffusion transformers on Apple silicon
Nunchaku Lite runs SVDQuant checkpoints on M5 Macs through PyTorch MPS and MLX, with Metal kernels for the GPU's Neural Accelerators
By Pham Hong Vinh
Nunchaku Lite SVDQuant 4-bit checkpoints on Apple silicon: new Metal kernels run the 4-bit layers as int8 matmuls on the M5 Neural Accelerators, 1.47x faster than mflux q4 on FLUX.1-dev at similar memory.
[Read More]