Study sheet: Statistics Foundations and Probability

📋 Course Outline

1. Statistical Concepts and Sampling
2. Variables, Scales, and Data Display
3. Descriptive Measures and Shape
4. Experiments and Probability Models
5. Random Variables and Normal Distribution
6. Standard Normal Distribution
7. Sampling Distributions
8. Point and Interval Estimation
9. Hypothesis Testing
10. Test Procedures and Applications
11. Normal and t Distribution Applications
12. Chi-Square Tests
13. ANOVA for Several Groups
14. Regression and Correlation Models
15. Regression Estimation and Prediction
16. Correlation Measures and Tests
17. Multiple Linear Regression
18. Extrapolation Limits

📖 1. Statistical Concepts and Sampling

🔑 Key Concepts & Definitions

• Statistics : a scientific method used to collect, present, analyze, and interpret data and to draw statistical conclusions
• Population : the set of all observation units whose characteristics are being studied
• Sample : a selected part of the population used for statistical analysis

📝 Essential Points

• Theoretical statistics develops and proves statistical theorems, methods, formulas, rules, and laws, whereas applied statistics uses them to solve real problems.

• A census surveys every element of the population, whereas a sample survey collects information from only part of the population.

• A random sample gives each population element a known nonzero probability of selection, whereas a non-random sample gives some elements no chance of selection.

💡 Memory Hook

Population → sample → variable → observation → data series

📖 2. Variables, Scales, and Data Display

🔑 Key Concepts & Definitions

• Variable : a characteristic studied across observation units that can take different values, whereas a constant has a fixed value

★ Must-know

📌 A quantitative variable can be expressed numerically, whereas a qualitative variable is classified into two or more categories without numerical values.

📌 A discrete variable has countable isolated values, whereas a continuous variable can take any value within one or more intervals.

• The four measurement scales are:
  • nominal
  • ordinal
  • interval
  • ratio

📌 A ratio scale has a meaningful zero indicating absence of the measured phenomenon, whereas an interval scale has a zero that does not indicate absence; weight uses a ratio scale, while temperature uses an interval scale.

📐 Formula — The relative frequency of a category equals its frequency divided by the sum of all frequencies: $f_r=\frac{f}{\sum f}$.

Further detail

📐 Formula — Sturges' rule estimates the number of grouped intervals as $C=1+3.3\log N$, where $N$ is the number of observations.

💡 Memory Hook

Nominal labels, ordinal ranks, interval differences, ratio proportions

📖 3. Descriptive Measures and Shape

🔑 Key Concepts & Definitions

• Arithmetic Mean : The arithmetic mean is the sum of all values divided by the number of values; for a population it is denoted by μ and for a sample by x̄.
• Median : The median is the middle value of data ranked in ascending order, or the arithmetic mean of the two middle values when the number of observations is even.
• Mode : The mode is the value occurring with the greatest frequency in a data series, and a series may have no mode or more than one mode.
• Standard Deviation : Standard deviation measures how closely data values are grouped around the arithmetic mean; smaller values indicate less dispersion and larger values indicate greater dispersion.

📝 Essential Points

📐 Formula — The coefficient of variation is the standard deviation divided by the arithmetic mean and multiplied by 100: $CV=\frac{\sigma}{\mu}\times100$ for a population and $CV=\frac{s}{\bar{x}}\times100$ for a sample.

📐 Formula — The interquartile range is the difference between the third and first quartiles: $IQR=Q_3-Q_1$.

💡 Memory Hook

Mean reacts to extremes; median resists them; mode marks frequency

📖 4. Experiments and Probability Models

🔑 Key Concepts & Definitions

• Sample Space : The sample space is the set of all possible outcomes of an experiment.
• Subjective Probability : assigned from an individual's opinion, experience, information, and belief when outcomes are neither equally likely nor repeatedly observable

📝 Essential Points

📐 Formula — When all elementary outcomes are equally likely, the probability of an elementary outcome is $P(E_i)=\frac{1}{\text{total number of outcomes}}$ and the probability of an event A is $P(A)=\frac{\text{number of favorable outcomes for }A}{\text{total number of outcomes}}$.

📌 Relative-frequency probability estimates an event probability as $P(A)=\frac{f}{n}$, where f is the number of realizations of A and n is the number of trials; the estimate becomes more stable as the sample grows.

• The four binomial conditions are:
  • n identical trials
  • two possible outcomes per trial
  • constant outcome probabilities
  • independent trials

💡 Memory Hook

Experiment → outcomes → sample space → probability model

📖 5. Random Variables and Normal Distribution

🔑 Key Concepts & Definitions

• Random Variable : a variable whose value is determined by the outcome of a random experiment

• Normal Distribution : The normal distribution is a symmetric, bell-shaped continuous probability distribution whose mean μ determines its center and standard deviation σ determines its dispersion.
• Standardized Normal Distribution : The standardized normal distribution is a normal distribution with mean 0 and standard deviation 1, and its standardized values are called z-values or z-scores.

📝 Essential Points

📌 A discrete random variable takes countably many values, whereas a continuous random variable can take any value in one or more intervals.

📐 Formula — A discrete probability distribution must satisfy $0\leq P(x)\leq1$ for every x and $\sum P(x)=1$.

📐 Formula — The expected value of a discrete random variable is $E(X)=\mu=\sum xP(x)$, and its standard deviation is $\sigma=\sqrt{\sum x^2P(x)-\mu^2}$.

💡 Memory Hook

Standardization converts distance from the mean into z-values

📖 6. Standard Normal Distribution

🔑 Key Concepts & Definitions

• Standardized normal distribution : a normal distribution with mean 0 and standard deviation 1, and its random variable is denoted by Z

📝 Essential Points

• The empirical rule gives these areas:
  • 68.26% within one standard deviation
  • 95.44% within two standard deviations
  • 99.74% within three standard deviations

📐 Formula — For a standardized normal variable, if F(z) is the area to the left of z, then $P(Z\leq a)=F(a)$, $P(Z\geq a)=1-F(a)$, and $P(a\leq Z\leq b)=F(b)-F(a)$.

📐 Formula — A normally distributed value x is standardized using $z=\frac{x-\mu}{\sigma}$.

💡 Memory Hook

68.26% → 95.44% → 99.74% within 1 → 2 → 3 standard deviations

📖 7. Sampling Distributions

🔑 Key Concepts & Definitions

• Sampling distribution : the probability distribution of a sample statistic, consisting of the different values that the statistic can take and their corresponding probabilities

★ Must-know

📌 The difference between a sample statistic and the corresponding population parameter is random error when the sample is random and no nonsampling error has occurred.

📐 Formula — For the sampling distribution of the sample mean, the expected value equals the population mean, $\mu_{\bar X}=\mu$, and, when the sample is at most 5% of the population, the standard error is $\sigma_{\bar X}=\frac{\sigma}{\sqrt n}$.

📌 By the central limit theorem, sample means from large samples are approximately normally distributed regardless of the population distribution; samples are generally considered large when $n\geq30$.

📐 Formula — For a sample proportion P, the expected value is $\mu_P=p$ and, when n/N ≤ 0.05, its standard error is $\sigma_P=\sqrt{\frac{pq}{n}}$, where q=1−p.

Further detail

📌 If the sampling fraction exceeds 5%, the standard error of the sample mean uses the finite population correction: $\sigma_{\bar X}=\frac{\sigma}{\sqrt n}\sqrt{\frac{N-n}{N-1}}$.

💡 Memory Hook

Random sampling → variable statistics → sampling distributions

📖 8. Point and Interval Estimation

🔑 Key Concepts & Definitions

• Statistical estimation : assigns numerical value or values to a population parameter using information obtained from a sample

★ Must-know

• 🔄 The estimation procedure consists of these steps:
  1. Selecting a sample
  2. Collecting the necessary information
  3. Calculating the sample statistic
  4. Assigning its value to the corresponding population parameter

📌 A point estimate is a single value calculated from a sample statistic, whereas an interval estimate gives a range believed to contain the corresponding population parameter.

📐 Formula — A confidence interval is constructed as $\text{point estimate}\pm\text{margin of error}$.

📐 Formula — When the population standard deviation is known, a confidence interval for the population mean is $\bar x\pm z\frac{\sigma}{\sqrt n}$.

📐 Formula — When the population standard deviation is unknown, a confidence interval for the population mean is $\bar x\pm t\frac{s}{\sqrt n}$, with n−1 degrees of freedom.

Further detail

• The most commonly used confidence levels are 90%, 95%, and 99%; the confidence level is written as 1−α, while α is the significance level.

💡 Memory Hook

Point estimate gives one value; interval estimate gives a range with confidence

📖 9. Hypothesis Testing

🔑 Key Concepts & Definitions

• Null hypothesis : a statement about a population parameter that is treated as true until sufficient evidence leads to its rejection

• Type I error : A Type I error occurs when the null hypothesis is true but is incorrectly rejected, and its probability is α.
• Type II error : A Type II error occurs when a false null hypothesis is not rejected, and its probability is β; the test power is 1−β.

★ Must-know

• The test directions are:
  • Two-tailed: rejection regions at both ends
  • Left-tailed: rejection region at the left end
  • Right-tailed: rejection region at the right end

Further detail

📌 With a fixed sample size, decreasing α increases β, so the probabilities of Type I and Type II errors cannot generally be reduced simultaneously.

💡 Memory Hook

Reject H₀ when evidence enters the rejection region; otherwise do not reject H₀

📖 10. Test Procedures and Applications

🔑 Key Concepts & Definitions

• P-value : the probability that the sample statistic deviates from the hypothesized parameter in the direction of H₁ by at least as much as the observed statistic

★ Must-know

• 🔄 The critical-value approach follows these stages:
  1. Formulating the null and alternative hypotheses
  2. Choosing the distribution
  3. Determining the rejection and nonrejection regions
  4. Calculating the test statistic
  5. Making the decision

📐 Formula — For testing a population mean with a normal distribution, the test statistic is $z=\frac{\bar x-\mu}{\sigma_{\bar X}}$, where $\sigma_{\bar X}=\frac{\sigma}{\sqrt n}$.

📌 Using the p-value approach, H₀ is rejected when the p-value is less than α and is not rejected when the p-value is greater than or equal to α.

Further detail

📐 Formula — When estimating a population mean with known σ, the required sample size for margin of error E is $n=\frac{z^2\sigma^2}{E^2}$; for a population proportion it is $n=\frac{z^2pq}{E^2}$.

💡 Memory Hook

Hypotheses → distribution → rejection region or p-value → decision

📖 11. Normal and t Distribution Applications

★ Must-know

• 🔄 The p-value procedure has four stages:
  1. Formulate the null and alternative hypotheses
  2. Choose the distribution
  3. Calculate the p-value
  4. Make the decision

📌 For estimating or testing a population mean with a known population standard deviation, the normal distribution is used when the sample is small and the population is normal, or when the sample is large with n ≥ 30.

📐 Formula — A confidence interval for a population mean in the normal-distribution cases is $\bar{x} \pm z\sigma_{\bar{x}}$, where $\sigma_{\bar{x}}=\frac{\sigma}{\sqrt{n}}$ and the margin of error is $E=z\sigma_{\bar{x}}$.

📌 The t distribution is used for a population mean when the population standard deviation is unknown, with a normal population required for a small sample and either a large sample or a normal population for the stated applications.

Further detail

• For a 95% confidence interval, the two tails each contain 0.025 of the normal-curve area and the critical value is z = 1.96.

📐 Formula — For two independent samples with known population standard deviations, the z statistic for the difference of means is $Z=\frac{\bar{X}_1-\bar{X}_2-(\mu_1-\mu_2)}{\sigma_{\bar{X}_1-\bar{X}_2}}$, where $\sigma_{\bar{X}_1-\bar{X}_2}=\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}$.

💡 Memory Hook

Known population deviation uses z; unknown population deviation uses t.

📖 12. Chi-Square Tests

🔑 Key Concepts & Definitions

• Goodness-of-fit test : tests whether observed frequencies follow a specified theoretical distribution

★ Must-know

📐 Formula — For a goodness-of-fit test, the expected frequency in a category is $E=np$ and the degrees of freedom are $df=k-1$.

📐 Formula — The goodness-of-fit chi-square statistic is $\chi^2=\sum\frac{(O-E)^2}{E}$, where O is the observed frequency and E is the expected frequency.

📐 Formula — For a contingency table with R rows and K columns, the independence-test degrees of freedom are $df=(R-1)(K-1)$ and the expected frequency in a cell is $E=\frac{(\text{row total})(\text{column total})}{\text{sample size}}$.

Further detail

• A multinomial experiment has four properties:
  • n identical trials
  • one of k possible outcomes with k > 2 per trial
  • independent trials
  • constant outcome probabilities

📌 The goodness-of-fit chi-square test is always right-tailed and requires every expected category frequency to be at least 5.

💡 Memory Hook

Observed–expected discrepancies → chi-square statistic → rejection decision.

📖 13. ANOVA for Several Groups

🔑 Key Concepts & Definitions

• One-way ANOVA : tests equality of the means of more than two populations using one factor or explanatory variable

★ Must-know

• One-way ANOVA requires:
  • normally distributed populations
  • equal population variances
  • random independent samples

📌 Between-sample variance VA estimates population variance from differences between sample means, whereas within-sample variance VR estimates it from differences among observations within samples.

📐 Formula — The ANOVA test statistic is $F=\frac{V_A}{V_R}$, with $V_A=\frac{S_A}{k-1}$ and $V_R=\frac{S_R}{n-k}$.

Further detail

📌 The one-way ANOVA F test is always right-tailed, with the rejection region at the right end of the F distribution.

💡 Memory Hook

Between-group variation is compared with within-group variation.

📖 14. Regression and Correlation Models

★ Must-know

📐 Formula — The deterministic simple linear regression model is $Y=\beta_0+\beta_1X$, where β0 is the intercept and β1 is the change in Y produced by a one-unit increase in X.

📐 Formula — The stochastic simple linear regression model is $Y=\beta_0+\beta_1X+\varepsilon$, where ε represents omitted variables and random variations.

• The regression assumptions are:

  • Linearity
  • Zero mean error
  • Homoskedasticity
  • No autocorrelation
  • X is not random
  • Normally distributed errors

• Spearman's coefficient ranges from −1 to 1; rS = 0 indicates no monotonic association, rS > 0 a direct monotonic association, and rS < 0 an inverse monotonic association.

Further detail

📐 Formula — Spearman's rank correlation coefficient is $r_S=1-\frac{6\sum d^2}{n(n^2-1)}$, where d is the difference between paired ranks.

📐 Formula — The sample linear correlation coefficient is $r=\frac{SP_{xy}}{\sqrt{SK_{xx}SK_{yy}}}$ and its test statistic is $t=\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$ with $df=n-2$.

💡 Memory Hook

Functional dependence gives one Y for each X; stochastic dependence allows several Y values.

📖 15. Regression Estimation and Prediction

★ Must-know

📌 Regression is used both to estimate the average value of Y for a given X and to predict an individual value of Y for a given X.

📐 Formula — The conditional mean in the regression population is $\mu_{Y|X}=\beta_0+\beta_1X$.

📐 Formula — A prediction interval for an individual Y value at X = Xp is $\hat{Y}_p\pm tS_{Y_p}$ with $S_{Y_p}=S\sqrt{1+\frac{1}{n}+\frac{(X_p-\bar{X})^2}{SK_{XX}}}$ and df = n − 2.

📌 Regression analysis has two main uses: estimating the mean value of Y for a given X and predicting an individual value of Y for a given X.

📐 Formula — The estimated mean response for a given value of X is obtained from the sample regression equation, and its confidence interval is $\hat{Y}_p \pm tS_{\hat{Y}_p}$ with $S_{\hat{Y}_p}=S\sqrt{\frac{1}{n}+\frac{(X_p-\bar{X})^2}{SK_{XX}}}$ and $df=n-2$.

Further detail

• Regression intervals use the t distribution because the unknown error standard deviation is replaced by its sample estimate, the regression standard error S.

💡 Memory Hook

Mean-response intervals estimate an average; prediction intervals estimate an individual outcome.

📖 16. Correlation Measures and Tests

🔑 Key Concepts & Definitions

• Pearson correlation coefficient : measures the degree of linear quantitative agreement between two variables and can take values from -1 to 1

★ Must-know

📌 A positive correlation means that Y tends to increase as X increases, whereas a negative correlation means that Y tends to decrease as X increases.

📐 Formula — Spearman's rank correlation coefficient is calculated after ranking X and Y separately, defining each rank difference as $d=u-v$, and using $r_S=1-\frac{6\sum d^2}{n(n^2-1)}$.

Further detail

📐 Formula — The sample Pearson correlation coefficient is calculated as $r=\frac{SP_{xy}}{\sqrt{SK_{xx}SK_{yy}}}$.

📐 Formula — For testing the population linear correlation, the test statistic is $t=\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$ with $df=n-2$.

📌 For Spearman's test, the null hypothesis is $H_0:\rho_S=0$, and it is rejected in a two-sided test when $r_S\leq-c$ or $r_S\geq+c$, in a right-sided test when $r_S\geq+c$, or in a left-sided test when $r_S\leq-c$.

💡 Memory Hook

Pearson measures linear association, whereas Spearman measures monotone association between ranks.

📖 17. Multiple Linear Regression

🔑 Key Concepts & Definitions

• Partial regression coefficient : represents the average change in Y caused by a one-unit increase in Xi while all other explanatory variables remain unchanged
• Multiple regression assumptions : The multiple regression assumptions are zero mean errors, mutually independent normally distributed errors with constant standard deviation, no linear dependence among explanatory variables, and no correlation between each explanatory variable and the error.

★ Must-know

📐 Formula — A multiple linear regression model with one dependent variable and k explanatory variables is $Y=\beta_0+\beta_1X_1+\beta_2X_2+\cdots+\beta_kX_k+\varepsilon$.

Further detail

📐 Formula — In multiple regression, the degrees of freedom for inference about a regression parameter are $df=n-k-1$, where n is the sample size and k is the number of explanatory variables.

• Multiple regression coefficients are estimated by minimizing the sum of squared residuals, $\min\sum(Y-\hat{Y})^2$.

💡 Memory Hook

Model → assumptions → least squares → estimation and prediction

📖 18. Extrapolation Limits

🔑 Key Concepts & Definitions

• Extrapolation : estimating or predicting Y for X values outside the range of X values represented in the sample
• Index number : the ratio of a phenomenon's level in the current period to its level in a base period, multiplied by 100
• Index number : a relative number equal to the current-period level divided by the base-period level and multiplied by 100
• Weighted group index : assigns greater weight to component series that have greater relative importance in the aggregate
• Time series : a chronologically ordered sequence of observations describing variations in a phenomenon over successive equal time intervals or moments
• Time-series components : Classical decomposition separates a time series into trend, cyclical variations, seasonal variations, and a residual component containing random and unforeseen effects.

📝 Essential Points

• Extrapolated conclusions should be interpreted with increasing caution as the target X value moves farther from the sample-data range.

• Individual indices describe changes in one phenomenon, whereas group indices describe changes in a group of related phenomena.

• Base indices use a fixed reference period, whereas chain indices compare each period with the preceding period.

• An index above 100 indicates a percentage increase equal to the amount by which it exceeds 100, an index below 100 indicates a decrease equal to the amount by which it is below 100, and an index of 100 indicates no change.

📐 Formula — A fixed-base index is calculated as $I_t=\frac{Y_t}{Y_0}\cdot100$, while a chain index is calculated as $L_t=\frac{Y_t}{Y_{t-1}}\cdot100$.

📌 A fixed-base index above 100 indicates a percentage increase equal to the excess over 100, an index below 100 indicates a decrease equal to the shortfall from 100, and an index equal to 100 indicates an unchanged level.

📌 Equal index numbers indicate equal relative changes, not equal absolute levels of the underlying phenomena.

📐 Formula — With the average-ratios method, a weighted group index is calculated as $I_t=\frac{\sum I_{it}\alpha_i}{\sum\alpha_i}$, commonly using the base-period value $\alpha_i=p_{i0}q_{i0}$ as the weight.

📌 The Laspeyres method uses base-period prices or quantities as weights, whereas the Paasche method uses current-period prices or quantities as weights.

📌 A moment series records the level of a phenomenon at particular moments and cannot meaningfully be summed, whereas an interval series records activity over intervals and can be cumulated.

💡 Memory Hook

Moving beyond observed X values → greater uncertainty and potentially invalid linear conclusions.

📊 Synthesis Tables

Probability Concepts

Concept	Condition	Method
Classical probability	Equally likely outcomes	Favorable outcomes divided by total outcomes
Relative-frequency probability	Repeated observations or existing data	Event frequency divided by number of trials
Subjective probability	No equal likelihood or repeatable experiment	Judgment, experience, information, and belief

Confidence Interval Choice

Condition	Distribution	Interval form
σ known	Normal (z)	x̄ ± zσ/√n
σ unknown	Student’s t	x̄ ± ts/√n
Small nonnormal or unknown population	Nonparametric method	No normal or t interval assumed