\documentclass{article}

\title{Notes on Introduction to Statistical Learning}
\author{Vishal Burman}
\date{\today}

\begin{document}
	\maketitle
	\section{Linear Regression}

	Mathematically we can write the linear relationship as:

	\begin{equation}
		Y \approx \beta_0 + \beta_1X
	\end{equation}\\
	Once we know the coefficients from training we can predict using:

	\begin{equation}
		\hat{y} = \hat{\beta_0} + \hat{\beta_1}x
	\end{equation}

	\subsection{Estimating the Coefficients}

	We need to estimate $\hat{\beta_0}$ and $\hat{\beta_1}$ in such a way that it is as close as possible to $\beta_0$ and $\beta_1$. The most common approach is using the least squares criterion.\\

	\textit{i}th residual is calculated as:
	\begin{equation}
		e_i =  y_i - \hat{y}_i
	\end{equation}\\
	We define the \textit{residual sum of squares} (RSS) as:

	\begin{equation}
		RSS = e_1^2 + e_2^2 + \cdot\cdot\cdot + e_n^2
	\end{equation}\\

	The least squares approach chooses $\hat{\beta_0}$ and $\hat{\beta_1}$ to minimize the RSS. The equation is as follows:

	\begin{equation}
		\hat{\beta_1} = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}
	\end{equation}

	\begin{equation}
		\hat{\beta_0} = \bar{y} - \hat{\beta}_1\bar{x}
	\end{equation}

	\subsection{Assessing the Accuracy of the Coefficients Estimates}
	How accurate is the sample mean $\hat{\mu}$ as an estimate of $\mu$? We answer this question by computing \textit{standard error} of $\hat{\mu}$. We have the well known formula:

	\begin{equation}
		Var(\hat{\mu}) = SE(\hat{\mu})^2 = \frac{\sigma^2}{n}
	\end{equation}

	\textit{Standard errors} associated with the predicted coefficients is given as:
	
	\begin{equation}
		SE(\beta_0)^2 = \sigma^2 \bigg[\frac{1}{n} + \frac{\bar{x}^2}{\sum_{i=1}^{n}(x_i - \bar{x})^2}\bigg]
	\end{equation}
	\begin{equation}
		SE(\beta_1)^2 = \frac{\sigma^2}{\sum_{i=1}^{n}(x_i - \bar{x})^2}
	\end{equation}

	Here $\sigma^2 = Var(\epsilon)$. In general $\sigma^2$ is not known but can be estimated from the data. The estimate is known as residual standard error and is given by the formula:
	
	\begin{equation}
		RSE = \sqrt{RSS/(n - 2)}
	\end{equation}

	Standard error can be used to calculate confidence interval. A 95\% confidence interval means the actual value lies in that interval with 95\% probability.
	
	For Linear Regression, a 95\% confidence interval for $\hat{\beta_1}$ and $\hat{\beta_0}$ is given as:
	
	\begin{equation}
		\hat{\beta_1} \pm 2 \cdot SE(\hat{\beta_1})
	\end{equation}

	\begin{equation}
		\hat\beta_0 \pm 2 \cdot SE(\hat{\beta_0})
	\end{equation}

	Standard errors can also be used to perform \textit{hypothesis tests} on the coefficients. The most common hypothesis test involves testing the \textit{null hypothesis} of:
	
	$H_0$ : There is no relationship between X and Y.
	\\
	
	versus the \textit{alternative hypothesis}
	
	$H_a$: There is some relationship between X and Y.
	\\
	
	If we take the equation:
	
	\begin{equation}
		Y = \beta_0 + \beta_1X
	\end{equation}

	Mathematically the \textit{null hypothesis} corresponds to testing:
	\begin{equation}
		H_0 : \beta_1 = 0
	\end{equation}

	versus
	\begin{equation}
		H_a : \beta_1 \neq 0
	\end{equation}

	
	To test the null hypothesis, we need to verify that $\hat{\beta_1}$ is sufficiently far from zero. For this we use $SE(\hat{\beta_1})$. If $SE(\hat{\beta_1})$ is small, then relatively small value
	
	\section{Multiple Linear Regression}
	This is the second section
\end{document}
