Polynomial Approximation for LLM Acceleration (Paper)

An academic paper investigating the use of short polynomial programs to approximate GPU transcendental functions such as sigmoid, tanh, and SiLU, offloading Special Function Unit workloads to Fused Multiply-Add units, and achieving up to 8% improvement in training throughput on NVIDIA GB200 hardware.