KS3 vs MLP: Faster Convergence on 4D Fractal Fitting, GPU Latency on Par

An independent researcher recently released a demo video on Reddit comparing their proposed K-Splanifolds version 3 (KS3) with a multilayer perceptron (MLP) on fitting a 4D Mandelbulb fractal. The results show that KS3 achieves significantly faster gradient descent convergence on random samples, while GPU inference latency is nearly on par with MLP. This result has potential implications for general regression tasks, especially in scenarios requiring fast training.

What Are K-Splanifolds and KS3

K-Splanifolds (KS for short) is a regression method based on a fast K-dimensional Hermite spline manifold algorithm. The author states that over the past year they have been investigating how to use this algorithm for general-purpose fast regression. KS3 is the third version of the algorithm, with core improvements in convergence efficiency and hardware adaptability. Unlike traditional MLPs, KS uses spline blocks to construct piecewise smooth manifolds, thereby reducing the number of training iterations required while maintaining expressiveness.

The author has released the code for all three versions and the KS1 paper on GitHub (link: https://github.com/curvedinf/k-splanifolds). Based on the description, KS is not designed to replace deep learning but rather to provide a more efficient alternative for regression problems.

Hermite splines are a piecewise polynomial interpolation method that matches not only function values but also first derivatives (and even higher-order derivatives) at each knot, ensuring smoothness at the joins. Compared with ordinary splines, Hermite splines offer more flexible control over curve shape and are commonly used in computer graphics and numerical analysis. In K-Splanifolds, the algorithm partitions the input space into multiple K-dimensional blocks, constructs a local manifold within each block using Hermite splines, and then smoothly joins them to form a global function. This piecewise structure gives the model high expressiveness in local regions while making gradient computation more efficient, because each sample activates only a few blocks, reducing redundant computation in backpropagation. This is one of the underlying reasons why KS3 can accelerate convergence.

4D Mandelbulb Fitting Experiment: Convergence Speed and Latency

In the demo video, the author uses gradient descent to fit a 4D Mandelbulb—a classic four-dimensional fractal structure—on random samples. The comparison includes MLP and earlier versions of KS. KS3 shows faster convergence while GPU latency is "almost identical" to MLP. This means that in the inference stage, KS3 does not sacrifice response speed for faster training.

Mandelbulb fitting is a task with high demands on function approximation capability. Achieving an advantage in this kind of geometric fitting indicates that KS3 is competitive in high-dimensional nonlinear regression.

The Mandelbulb is a three-dimensional generalization of the Mandelbrot set, generating complex fractal geometry through iterative spherical coordinate transformations. The 4D Mandelbulb extends this to four-dimensional space, and its function graph exhibits extremely high nonlinearity and self-similarity, placing stringent demands on the approximation capability of regression models. In machine learning, fitting a 4D Mandelbulb is often used as a benchmark task for testing model expressiveness and training efficiency, because it requires the model to capture fine structures in high-dimensional space. Unlike tasks such as image classification, the goal here is continuous function regression, and the loss surface may contain numerous local minima, so differences in convergence speed can intuitively reflect the efficiency of optimization algorithms. KS3's advantage on this task is partly attributed to the ability of spline blocks to quickly adapt to local features.

Cross-Task Performance: From Geometric Fitting to LLMs

The author notes that KS3 performs differently across types of regression problems. In tasks such as noise removal, geometric fitting, and financial forecasting, KS3 converges "considerably faster" than MLP; on large language model (LLM) related tasks, the two are "neck and neck." This observation comes from a single source and still requires more independent reproduction.

This difference may stem from the characteristics of the tasks themselves: LLMs involve extremely high dimensionality and complex semantics, where the advantages of spline methods may be diluted; whereas geometric fitting and low-dimensional regression can better leverage the efficiency of piecewise splines.

Multilayer perceptrons (MLPs) are a cornerstone of deep learning, composed of stacked fully connected layers, and theoretically can approximate continuous functions to arbitrary precision (universal approximation theorem). However, MLP training typically relies on many iterations and fine hyperparameter tuning, and the number of parameters grows rapidly with hidden layer width. As an alternative, KS3's design philosophy is to replace black-box fully connected layers with structured priors (piecewise spline manifolds), thereby reducing the number of iterations on specific tasks. However, LLM tasks involve trillions of tokens and deep semantic composition, and the locality of spline methods may struggle to capture long-range dependencies, which explains why the two are on par for LLMs. Understanding this difference helps determine which scenarios are suitable for introducing KS-type methods.

Open Source and Next Steps: Aiming for the nanogpt Speed Record

The author emphasizes that they are an independent researcher and has made all code public, hoping the community will participate in exploration. They also state that if someone is willing to donate 8x H100 compute, they will attempt to use KS3 to challenge the world record for nanogpt training speed—claiming that on proxy hardware they have already found a configuration that can beat the existing record.

If realized, this plan would serve as an important test of KS3 in large-scale training scenarios. However, this goal is currently still at the stage of seeking compute support, with no actual results yet.

Implications for AI Practitioners

The emergence of KS3 reminds us that there is still room for algorithmic optimization in regression tasks. MLPs have dominated most scenarios as universal approximators, but for specific problems (such as geometric fitting and financial forecasting), specialized algorithms may deliver order-of-magnitude improvements in convergence speed. For teams with limited resources, faster convergence means lower training costs.

Of course, the generality of KS3 still requires more independent validation. The code and paper provided by the author offer a basis for reproduction, but community feedback and cross-task benchmarks will be key to judging its practical value.

One-line Summary

Independent researcher releases the KS3 regression algorithm, which converges faster than MLP on 4D Mandelbulb fitting with comparable GPU latency.

Paragraph Summary

An independent researcher has open-sourced K-Splanifolds version 3 (KS3), a regression method based on K-dimensional Hermite spline manifolds. In a demo fitting a 4D Mandelbulb fractal, KS3's gradient descent convergence is significantly faster than a multilayer perceptron (MLP), while GPU inference latency is nearly identical. The author claims that KS3 converges notably faster in noise removal, geometric fitting, and financial forecasting tasks, but is on par with MLP on large language model tasks. The code and paper are open-sourced, and the author is seeking 8x H100 compute to challenge the nanogpt training speed record. KS3 offers a potentially efficient alternative to MLP for regression problems, but its generality still requires independent validation.