Linear Regression
A statistical and machine learning method that builds a predictive model by fitting a linear relationship between independent variables and a dependent variable, producing coefficients (weights) and an intercept that form an interpretable linear equation.
Core Facts
A linear probe is a simple linear classifier trained on model activations, used to detect whether specific behaviors or concepts exist inside the model.
A univariate linear regression model can be expressed as y = w * x + b, where w is the weight (slope) and b is the bias (intercept).
Timeline (last 90 days)
The leverage matrix originates from the concept of leverage in linear regression, used to measure the degree of influence of each training sample on the model's predictions.
The researchers derive a closed-form expression for the output-space gap in linear regression and extend it to nonlinear models via the generalized Gauss-Newton method and local linear approximation.
When the dependent variable is a continuous numerical value, linear regression is a reasonable starting point with strong interpretability.
The coefficients of linear regression are essentially the weights in the model formula and require no additional conversion.
Dragging an outlier in linear regression makes it visually apparent how the least squares method is skewed by anomalous values.
Multivariate linear regression can be expressed in matrix form as y = X · W + b.
A univariate linear regression model can be expressed as y = w * x + b, where w is the weight (slope) and b is the bias (intercept).
Normalization is not strictly necessary when an analytical solution (normal equation) exists for linear regression, but it is almost an essential step when using iterative optimization.
The gradient descent framework of linear regression can be transferred to more complex models such as logistic regression and neural networks.
The loss curve of linear regression takes the shape of a convex bowl; convexity guarantees that optimization has a unique global minimum, and gradient descent will not get trapped in local optima.
6 more timeline events
All Facts (16)
A linear probe is a simple linear classifier trained on model activations, used to detect whether specific behaviors or concepts exist inside the model.
75%VerifiedA univariate linear regression model can be expressed as y = w * x + b, where w is the weight (slope) and b is the bias (intercept).
65%UnverifiedThe leverage matrix originates from the concept of leverage in linear regression, used to measure the degree of influence of each training sample on the model's predictions.
50%UnverifiedThe researchers derive a closed-form expression for the output-space gap in linear regression and extend it to nonlinear models via the generalized Gauss-Newton method and local linear approximation.
50%UnverifiedWhen the dependent variable is a continuous numerical value, linear regression is a reasonable starting point with strong interpretability.
50%UnverifiedThe coefficients of linear regression are essentially the weights in the model formula and require no additional conversion.
50%UnverifiedDragging an outlier in linear regression makes it visually apparent how the least squares method is skewed by anomalous values.
50%UnverifiedMultivariate linear regression can be expressed in matrix form as y = X · W + b.
50%UnverifiedNormalization is not strictly necessary when an analytical solution (normal equation) exists for linear regression, but it is almost an essential step when using iterative optimization.
50%UnverifiedThe gradient descent framework of linear regression can be transferred to more complex models such as logistic regression and neural networks.
50%UnverifiedThe loss curve of linear regression takes the shape of a convex bowl; convexity guarantees that optimization has a unique global minimum, and gradient descent will not get trapped in local optima.
50%UnverifiedA project implementing linear regression from scratch using NumPy connects calculus, linear algebra, and programming, helping one understand the effect of gradient descent and learning rate on convergence.
50%UnverifiedUnder the homoscedasticity assumption, noise variance is constant, which is the standard assumption of classical linear regression; under heteroscedasticity, variance changes with input.
50%UnverifiedClassical linear regression adopts the homoscedasticity assumption.
50%Unverified线性回归的闭式解为 w = (X^T X)^{-1} X^T y，涉及矩阵转置、矩阵乘法和矩阵求逆
50%UnverifiedThe author compared four models—linear regression, ridge regression, random forest, and gradient boosting—and all four exhibited exactly the same bias.
50%