15.2.6 Algorithms (Fit Linear with X Error)

1 The Fitting Model
2 Fit Control
- 2.1 Computation Method
3 Quantities (York Method)
- 3.1 Fit Parameters
- 3.2 Fit Statistics
4 Quantities (FV Method)
5 Quantities (Deming Method)
- 5.1 Fit Parameters
- 5.2 Fit Statistics
6 Finding X/Y
7 Residual Plots
8 Reference

The Fitting Model

For given dataset $(X_i,Y_i), (\sigma_{x_i},\sigma_{y_i}), i=1,2,\ldots n$ , where X is the independent variable and Y is the dependent variable, and $(\sigma_{x_i},\sigma_{y_i})$ are Errors for X, Y, respectively. -- Fit Linear with X Error fits the data to a model of the following form:

$y=\beta _0+\beta _1x+\varepsilon$	(1)
$\left\{\begin{matrix} x_i=X_i+\sigma_{x_i}\\ y_i=Y_i+\sigma_{y_i} \end{matrix}\right.$	(2)

Fit Control

Computation Method

York Method

York Method is the computation method of D. York, described in Unified equations for the slope, intercept, and standard error of the best straight line
FV Method

FV Method is the computation method of Giovanni Fasano & Roberto Vio, described in Fittng a Straight Line with Errors on Both Coordinates.
Deming Method

Deming regression is the maximum likelihood estimation of an errors-in-variables model, the X/Y errors are assumed to be independent identically distributed.
Correlation Between X and Y Errors
Correlation Between X and Y Errors $r_i$ (For York method only)
Standard Deviation of X/Y
Standard Deviation of X/Y (For Deming method only)

Quantities (York Method)

When you perform a linear fit, you generate an analysis report sheet listing computed quantities. The Parameters table reports model slope and intercept (numbers in parentheses show how the quantities are derived):

Fit Parameters

Fitted Value and Standard Errors

Define $W_i$ which involves the weight (error) for both x and y;

$W_i = \frac{\omega_{x_i}\omega_{y_i}}{\omega_{x_i}+\beta_1^2\omega_{y_i}-2\beta_1 r_ia_i} =\frac{1}{\sigma_{y_i}^2+\beta_1^2\sigma_{x_i}^2 - 2\beta_1 r_i \sigma_{x_i} \sigma_{y_i}}$	(3)

Therein, $\omega_{x_i}=\frac{1}{\sigma_{x_i}^2}, \ \omega_{y_i}=\frac{1}{\sigma_{y_i}^2}$ are weights of $(X_i, Y_i)$ , $r_i$ is Correlation between X and Y Errors (i.e. $\sigma_{x_i}$ and $\sigma_{y_i}$ ), and $\alpha_i=\sqrt{\omega_{x_i} \omega_{y_i}}$ .

The slope of the fitted line for $(X_i, Y_i)$ with no weighting (errors) is the initial value for $\beta_1$ . They should be solved iteratively, until successive estimates of $\beta_1$ agree within desired tolerance.

The concise equations which estimate parameters $\hat{\beta_0}$ and $\hat{\beta_1}$ for the best-fit line with X_Y errors are:

$\hat{\beta_0}=\bar{Y}-\hat{\beta_1}\bar{X}$	(4)

$\hat{\beta_1}=\frac{\sum{W_i b_i V_i}}{\sum{W_i b_i U_i}}$	(5)

where $\bar{X} = \frac{ \sum{W_i X_i} }{ \sum{W_i} }, \ \bar{Y} = \frac{ \sum{W_i Y_i} }{ \sum{Y_i} }$ .

U and V are the deviation for X and Y:

$\left\{\begin{matrix} U=X-\bar{X}\\ V=Y-\bar{Y} \end{matrix}\right.$

and

$b_i=W_i \left[\frac{U_i}{\omega_{y_i}}+\frac{\hat{\beta_1}}{\omega_{x_i}}{V_i}-(\beta U_i+V_i)\frac{r_i}{\alpha_i} \right]$	(6)

The corresponding variation $\sigma^2$ and standard error $s$ for parameter is:

$\sigma_{\hat{\beta_0}}^2=\frac{1}{\sum{W_i}}+\bar{x}^2\sigma_{\hat{\beta_1}}^2$	(7)
$\sigma_{\hat{\beta_1}}^2=\frac{1}{\sum{W_i u_i^2}}$	(8)

where $\bar{x} = \frac{ \sum{W_i x_i} }{ \sum{W_i} }$ , $x_i$ is the expectation value of $X_i$ , and $u_i=x_i - \bar{x}$ .

The standard error for parameters is final given by:

$\varepsilon_{\hat{\beta_0}}=\sqrt{\sigma _{\hat{\beta_0}}^2}\sqrt{\frac{S}{n-2}}$	(9)

$\varepsilon_{\hat{\beta_1}}=\sqrt{\sigma _{\hat{\beta_1}}^2}\sqrt{\frac{S}{n-2}}$	(10)

where $S$ is:

$S=\sum W_i(Y_i - \beta_1 X_i- \beta_0)^2$	(11)

t-Value and Confidence Level

If the regression assumptions hold, we have:

$\frac{{\hat \beta _0}-\beta _0}{\varepsilon _{\hat \beta _0}}\sim t_{n^{}-1}$ and $\frac{{\hat \beta _1}-\beta _1}{\varepsilon _{\hat \beta _1}}\sim t_{n^{}-1}$	(12)

The t-test can be used to examine whether the fitting parameters are significantly different from zero, which means that we can test whether $\beta _0= 0\,\!$ (if true, this means that the fitted line passes through the origin) or $\beta _1= 0\,\!$ . The hypotheses of the t-tests are:

$H_0 : \beta _0= 0\,\!$ $H_0 : \beta _1= 0\,\!$

$H_\alpha : \beta _0 \neq 0\,\!$ $H_\alpha : \beta _1 \neq 0\,\!$

The t-values can be computed by:

$t_{\hat \beta _0}=\frac{{\hat \beta _0}-0}{\varepsilon _{\hat \beta _0}}$ and $t_{\hat \beta _1}=\frac{{\hat \beta _1}-0}{\varepsilon _{\hat \beta _1}}$	(13)

With the computed t-value, we can decide whether or not to reject the corresponding null hypothesis. Usually, for a given confidence level $\alpha\,\!$ , we can reject $H_0 \,\!$ when $|t|>t_{\frac \alpha 2}$ . Additionally, the p-value, or significance level, is reported with a t-test. We also reject the null hypothesis $H_0 \,\!$ if the p-value is less than $\alpha\,\!$ .

Prob>|t|

The probability that $H_0 \,\!$ in the t test above is true.

$prob=2(1-tcdf(\|t\|,df_{Error}))\,\!$	(14)

where tcdf(t, df) computes the lower tail probability for the Student's t distribution with df degree of freedom.

LCL and UCL

From the t-value, we can calculate the $(1-\alpha )\times 100\%$ Confidence Interval for each parameter by:

$\hat \beta _j-t_{(\frac \alpha 2,n^{}-k)}\varepsilon _{\hat \beta _j}\leq \hat \beta _j\leq \hat \beta _j+t_{(\frac \alpha 2,n^{}-k)}\varepsilon _{\hat \beta _j}$	(15)

where $UCL$ and $LCL$ is short for the Upper Confidence Interval and Lower Confidence Interval, respectively.

CI Half Width

The Confidence Interval Half Width is:

$CI=\frac{UCL-LCL}2$	(16)

where UCL and LCL is the Upper Confidence Interval and Lower Confidence Interval, respectively.

For more information ,see Reference 1 (below).

Fit Statistics

Degrees of Freedom

$df=n-2$	(17)

n is total number of points

Residual Sum of Squares

$RSS=\sum^n_{i=1} \frac{(\beta_0+\beta_1 x_i - y_i)^2}{\sigma^2_{y_i}+\beta_1^2\sigma^2_{x_i}}$	(18)

Reduced Chi-Sqr

$\sigma^2=\frac{RSS}{n-2}$	(19)

Pearson's r

In simple linear regression, the correlation coefficient between x and y, denoted by r, equals to:

$r=-R\,\!$ if $\beta _1\,\!$ is negative
$r=R\,\!$ if $\beta _1\,\!$ is positive	(20)

$R^2$ can be computed as:

$TSS=\sum_{i=1}^n(y_i-\bar{y})^2$
$R^2=\frac{SXY}{SXX*TSS}=1-\frac{RSS}{TSS}$	(21)

Root-MSE (SD)

Root mean square of the error, or residual standard deviation, which equals to:

$df_{Error}=n-2$
$RootMSE=\sqrt{\frac{RSS}{df_{Error}}}$	(22)

Covariance and Correlation Matrix

The Covariance matrix of linear regression is calculated by:

$\begin{pmatrix} Cov(\beta _0,\beta _0) & Cov(\beta _0,\beta _1)\\ Cov(\beta _1,\beta _0) & Cov(\beta _1,\beta _1) \end{pmatrix}=\sigma ^2\frac 1{SXX}\begin{pmatrix} \sum \frac{x_i^2}n & -\bar x \\-\bar x & 1 \end{pmatrix}$	(23)

The correlation between any two parameters is:

$\rho (\beta _i,\beta _j)=\frac{Cov(\beta _i,\beta _j)}{\sqrt{Cov(\beta _i,\beta _i)}\sqrt{Cov(\beta _j,\beta _j)}}$	(24)

Quantities (FV Method)

FV Method is the computation method of Giovanni Fasano & Roberto Vio, described in Fittng a Straight Line with Errors on Both Coordinates.

The weighting is defined as:

$W_i=\frac{1}{\beta_1^2\sigma_{x_{i}}^2+\sigma_{y_{i}}^2}$	(25)

The slope of the fitted line for $(X_i, Y_i)$ with no weighting (errors) is $\beta_1$ .

Let

$\bar{x}=\frac{\sum{W_i x_i}}{\sum W_i}$	(26)
$\bar{y}=\frac{\sum{W_i y_i}}{\sum W_i}$	(27)

by minimizing the sum $K^2=\sum{W_i (y_i-\beta_0-\beta_1 x_i)^2}$ , we can get the estimate value $\beta_0$ and $\beta_1$ by setting the partial derivatives to 0.

$\hat{\beta_0}=\bar{y}-\hat{\beta_1}\bar{x}$	(28)
$a\hat{\beta_1}^2+b\hat{\beta_1}-c=0$	(29)

where

$a=\sum{W_i^2\sigma_{x_i}^2(y_i-\bar{y_i})(x_i-\bar{x_i})}$	(30)
$b=\sum{W_i^2[\sigma_{y_i}^2(x_i-\bar{x_i})^2-\sigma_{x_i}^2(y_i-\bar{y_i})^2]}$	(31)
$c=\sum{W_i^2\sigma_{y_i}^2(y_i-\bar{y_i})(x_i-\bar{x_i})}$	(32)

$\hat{\beta_1}$ should be solved iteratively, until successive estimates of $\hat{\beta_1}$ agree within desired tolerance.

For each parameter standard error, please refer to Linear Regression Model

For more information ,see Reference 2 (below).