Differentiable Relaxation

A mathematical technique that transforms discrete combinatorial optimization problems into continuous optimization problems. By introducing learnable continuous importance scores, it allows gradients to flow back to the ranking criterion, and is commonly used in neural architecture search and pruning.