CapExp
A core building block of the PreCap activation function: a bounded exponential unit that switches to a linear extension once the input exceeds a threshold, equivalent to a shifted and scaled ELU.
Timeline (last 90 days)
PreCap is an activation function whose core building block, CapExp, is an exponential unit that switches to a linear extension once the input exceeds a threshold, equivalent to a shifted and scaled ELU.
PreCap adds a soft tanh clipping before CapExp, making the entire function's output smooth, monotonic and bounded on both sides.
Plain ELU performs comparably to SPreCap overall, suggesting that the gain from CapExp may not come entirely from its special design but is related to the overall characteristics of the ELU family.
All Facts (3)
PreCap is an activation function whose core building block, CapExp, is an exponential unit that switches to a linear extension once the input exceeds a threshold, equivalent to a shifted and scaled ELU.
50%UnverifiedPreCap adds a soft tanh clipping before CapExp, making the entire function's output smooth, monotonic and bounded on both sides.
50%UnverifiedPlain ELU performs comparably to SPreCap overall, suggesting that the gain from CapExp may not come entirely from its special design but is related to the overall characteristics of the ELU family.
50%