Lernzettel: Introduction to Probability and Uncertainty

📋 Course Outline

1. Probability as a measure and function
2. Knightian and radical uncertainty
3. Random variables
4. Probability distributions and CDFs
5. Expected value, variance, and standard deviation
6. Uniform distribution

📖 1. Probability as a measure and function

🔑 Key Concepts & Definitions

• Sample space S : The sample space is the set of all possible outcomes of a random experiment.
• Power set 2^S : The power set is the collection of all subsets of S, including the empty set and S itself.
• Probability function : A probability function assigns a number between 0 and 1 to each event (subset of the sample space).
• Event : An event is a subset of the sample space that collects outcomes sharing a common property.

📝 Essential Points

• Probability is unsatisfactory when described only as chance or likelihood because it is vague and lacks a solid mathematical foundation.
• Probability is “like a measure” because it assigns numerical sizes to events similarly to how physical measurement assigns sizes to objects.
• The probability function must take events as inputs rather than a formula for one specific case.
• The probability function’s domain is 2^S and its output is a real number in the interval [0,1].
• Including both ∅ and S ensures probabilities can be assigned to the two extreme events.
• Probabilities larger than 1 would not have a sensible interpretation as degrees of certainty.

💡 Memory Hook

Power set → every subset gets a probability number in [0,1].

📖 2. Knightian and radical uncertainty

🔑 Key Concepts & Definitions

• Knightian uncertainty : Knightian uncertainty is uncertainty where the precise probability of each outcome is not known.
• Radical uncertainty : Radical uncertainty is uncertainty where the set of possible outcomes is not completely known.
• Random experiment : A random experiment is a setting with repeatability, unpredictability, and well-defined outcomes with related probabilities.

📝 Essential Points

• Knightian uncertainty occurs when outcomes exist but their probabilities are not precisely specified.
• Radical uncertainty occurs when not all outcomes are known in advance.
• Traditional probability theory covers three characteristics of a random experiment: unpredictability, repeatability, and well-defined outcomes with probabilities.

💡 Memory Hook

Knightian: outcomes known, probabilities unknown; Radical: outcomes unknown too.

📖 3. Random variables

🔑 Key Concepts & Definitions

• Random variable X : A random variable is a function that maps outcomes of a random experiment to real numbers.
• Discrete random variable : A discrete random variable has a set of possible values that is finite or countably infinite.
• Countably finite set : A countably finite set is a finite set whose elements can be listed one by one.
• Countably infinite set : A countably infinite set is an infinite set whose elements can still be listed one by one.

📝 Essential Points

• A random variable is a numerical quantity determined by the outcome of a random experiment.
• The formal notation for a random variable is a mapping $X:S\to\mathbb{R}$ (or to a subset of real numbers).
• A random variable is discrete when its set of possible values is finite or countably infinite.
• The event describes what occurred (e.g., the die shows 1), while the random variable value is the number assigned by the function to that outcome.
• $X$ denotes the random variable itself as a function, while $x$ denotes one fixed value the random variable can take.

💡 Memory Hook

Event = outcome; Value = number given by X.

📖 4. Probability distributions and CDFs

🔑 Key Concepts & Definitions

• Probability distribution : A probability distribution assigns probabilities to the possible values of a random variable.
• Probability mass function : A probability mass function is the discrete distribution $f(x)=P(X=x)$ giving the probability at each value.
• Cumulative distribution function F : The cumulative distribution function is $F(x)=P(X\le x)$ for a random variable.

📝 Essential Points

• A probability distribution for a discrete random variable is a function that gives $P(X=x)$ for each possible value.
• It is called a mass function because probability is concentrated at discrete points rather than spread continuously.
• A proper probability distribution has non-negative probabilities that sum to 1.
• A CDF represents the probability that the random variable is less than or equal to a threshold value.
• For a discrete random variable, the CDF is a step function because probabilities accumulate at isolated values.
• For any $x$, $P(X>x)=1-F(x)$.

💡 Memory Hook

CDF stacks probabilities up to x: $F(x)=P(X\le x)$.

📖 5. Expected value, variance, and standard deviation

🔑 Key Concepts & Definitions

• Expected value E(X) : The expected value is the long-run average value of a random variable over repeated trials.
• Variance Var(X) : Variance measures how spread out the random variable values are around their expected value.
• Standard deviation : Standard deviation is the square root of the variance and has the same units as the random variable.

📝 Essential Points

• For a discrete random variable, $E(X)=\sum_x x\,P(X=x)$.
• Variance is the expected squared deviation from the mean, capturing dispersion around $E(X)$.
• Variance has units equal to the square of the random variable’s units (e.g., dollars squared).
• Standard deviation is often preferred because it is in the same units as the original variable.
• Two equivalent variance formulas are $Var(X)=E[(X-\mu)^2]$ and $Var(X)=E(X^2)-\mu^2$, where $\mu=E(X)$.
• For discrete $X$, $Var(X)=\sum (x-\mu)^2 P(X=x)$ and $Var(X)=\sum P(X=x) x-\mu^2$, with the $-\mu^2$ term outside the sum.

💡 Memory Hook

Variance: squared units; Standard deviation: returns to original units via square root.

📖 6. Uniform distribution

🔑 Key Concepts & Definitions

• Uniform distribution : A uniform distribution assigns the same probability to all values in its support.
• Equiprobable outcomes : Equiprobable outcomes are outcomes that share equal probability under a model.

📝 Essential Points

• A uniform distribution is characterized by all values being equiprobable.
• A die roll is a uniform random variable when each face has equal chance.
• Mapping coin outcomes to numbers (Heads=1, Tails=0) can produce a uniform random variable over {0,1}.

💡 Memory Hook

Uniform = all faces equal (equiprobable).

⚠️ Common Pitfalls & Confusions

1. Confusing an event with a random variable value leads to mixing up “what happened” (subset) and “the number assigned” by X.
2. Assuming probability can be computed for only one case instead of defined for every event in 2^S breaks the function-based definition.
3. Treating a discrete random variable as continuous is wrong when its set of possible values is finite or countably infinite.
4. Using $F(x)$ as $P(X<x)$ instead of $P(X\le x)$ gives incorrect tail probabilities.
5. For discrete variance, forgetting that $-\mu^2$ is outside the summation produces an incorrect formula.
6. Using variance units directly as if they matched the original variable’s units confuses dollars-squared with dollars.
7. Thinking “radical uncertainty” just means probabilities are unknown instead of outcomes being unknown also.

✅ Exam Checklist

1. Explain why the informal “chance/likelihood” notion of probability is mathematically insufficient.
2. Define probability as a function, including its input (event) and output ([0,1]) and its domain (2^S).
3. State why both the empty set and the full sample space must be included as events.
4. Distinguish Knightian uncertainty from radical uncertainty in terms of unknown probabilities vs unknown outcomes.
5. List the three characteristics of uncertainty covered by traditional probability theory for random experiments.
6. Define a random variable as a function and give the mapping notation $X:S\to\mathbb{R}$.
7. State when a random variable is discrete and what “countably finite” vs “countably infinite” means.
8. Differentiate the event from the value of the random variable using the die example idea.
9. Distinguish $X$ (the random variable/function) from $x$ (a fixed possible value).
10. Define a probability distribution for a discrete random variable and write $f(x)=P(X=x)$.
11. State the conditions a proper probability mass function/distribution must satisfy (non-negative and sums to 1).
12. Define the CDF notation $F(x)=P(X\le x)$ and state why it is a step function for discrete $X$.
13. Compute tail probability using the CDF via $P(X>x)=1-F(x)$.
14. Compute expected value for discrete $X$ using $E(X)=\sum_x xP(X=x)$.