FlowGenesis

The FlowGenesis framework lets the network autonomously add or remove capacity during training through neuron splitting and pruning, without presetting a target architecture or conducting a separate architecture search.

FlowGenesis proposes "Representation Flux" as a local statistic to capture changes in a neuron's average activation on mini-batch data.

In FlowGenesis, high-flux neurons are split into two child neurons, and the splitting process is designed to approximately preserve the network's original function.

In FlowGenesis, low-flux and low-activity neurons are pruned.

The design of FlowGenesis requires neither defining a search space nor evaluating a large number of candidate architectures; the network performs structural adaptation during gradient descent.

Experiments on image and text classification tasks show that even when initial network sizes differ, after FlowGenesis adjustment the range of final neuron counts on the same dataset tends to converge, while predictive performance remains comparable.

FlowGenesis is extended to continual learning, where under task-incremental learning settings, representation-guided structural adaptation reduces the number of neurons and parameters allocated to subsequent tasks while maintaining accuracy comparable to fixed-width progressive neural networks.

FlowGenesis currently has limited experimental scale, covering only image and text classification, and the design in which splitting approximately preserves function may introduce additional computational overhead.