﻿ FSDA main page

## Technical introduction to Robust methods in regression

### Notation

Consider the usual regression model

$y_i = x_i^T\beta +u_i \qquad i=1, \ldots, n,$

where $u_i$ are random errors which have common variance equal to $\sigma^2$ and are independent from the covariates $x_i$.  $\beta,$ the location component of the linear model, is the parameter of interest. $\sigma$ is the so called scale component of the linear model.

Given an estimator of $\beta$, say $\hat \beta,$ the residuals are defined as

$$r_i(\hat \beta)=y_i- x_i^T \hat \beta$$

The traditional least squares estimate of $\beta$ is denoted with

$$\hat \beta = \hat \beta_{LS}=\hat \beta_{LS}(X,y)=(X'X)^{-1}X'y$$

We are concerned with the case where a certain proportion of the observations may not follow model above. Traditional robust estimators attempt to limit the influence of outliers by replacing in the estimation of $\beta$ the square of the residuals with a less rapidly increasing loss function or by a function ρ of the residuals themselves which is bounded.

### The $\rho$ function

In the literature of robust statistics the $\rho$ function denotes a function that

1. $\rho(x)$ is a non decreasing function of $|x|$
2. $\rho(0)=0$
3. $\rho(x)$ is increasing for $x>0$ such that $\rho(x) < \rho(\infty)$

Perhaps the most popular choice for the $\rho$ function in is Tukey’s biweight (bisquare) function

$\rho(x) = \begin{cases} \frac{x^2}{2}-\frac{x^4}{2c^2}+\frac{x^6}{6c^4} & \mbox{if} \quad |x| \leq c \\ \\ \frac{c^2}{6} & \mbox{if} \quad |x| > c, \\ \end{cases}$ where $c>0$ is a tuning constant which is linked to the breakdown point of the estimator of $\beta$.

Function $\rho$ for Tukey's biweight is implemented in routine TBrho.

### The $\psi$ function

In the literature of robust statistics a $\psi$ (psi) function denotes a function $\psi$ which is the derivative of a $\rho$ function $\psi = \rho'$ such that

1. $\psi$ is odd and $\psi(x) \geq 0$ for $x \geq 0$

Function $\psi$ for Tukey's biweight

$\psi(x) = \begin{cases} x- \frac{2x^3}{c^2}+\frac{x^5}{c^4} = x \Big[ 1- \Big( \frac{x}{c}\Big)^2 \Big]^2 & \mbox{if} \quad |x| \leq c \\ \\ 0 & \mbox{if} \quad |x| > c, \\ \end{cases}$

is implemented in routine TBpsi.