Basic Probability • BP1

IOE Probability: Basic Definitions and Formulae

Important definitions, formulae and the Picture Method idea needed before solving probability and Bayes’ theorem problems.

Watch Video and Understand Theory

This video should explain the definitions and formulae before students start solving probability problems.

Very Important Definitions

Learn the meaning first. Probability becomes easy when the sample space, event and condition are clearly identified.

Random Experiment

A random experiment is an experiment whose particular outcome cannot be predicted in advance, although all possible outcomes are known.

Example: Tossing a coin, throwing a die, selecting one concrete cube for testing, or checking whether a network request succeeds.

Sample Space

The set of all possible outcomes of a random experiment is called the sample space. It is denoted by \(S\).

\[ S=\{H,T\}\quad \text{for one coin toss} \]

Example: For throwing a die, \(S=\{1,2,3,4,5,6\}\).

Event

An event is any subset of the sample space.

For a die throw, let \(A\) be the event of getting an even number. Then: \[ A=\{2,4,6\} \]

Mutually Exclusive Events

Two or more events are mutually exclusive if they cannot occur at the same time.

\[ A\cap B=\varnothing \]

Concrete example: Civil engineers test concrete cubes for strength. In one strength test, a cube cannot be both “passed” and “failed” at the same time. So “pass” and “fail” are mutually exclusive events.

Important: If two events with non-zero probabilities are mutually exclusive, they are not independent. Occurrence of one prevents the other.

Exhaustive Events

The word exhausted means completely used up. In probability, events are called exhaustive when they use the whole sample space completely.

That means at least one of the events must occur.

\[ E_1\cup E_2\cup E_3\cup\cdots\cup E_n=S \]

Example: In throwing a die, the event of getting an odd number and the event of getting an even number are exhaustive because together they cover all outcomes: \[ \{1,3,5\}\cup\{2,4,6\}=S \]

Mutually Exclusive and Exhaustive Events

If events are both mutually exclusive and exhaustive, they fragment the sample space.

Mutually exclusive means the fragments do not overlap. Exhaustive means the fragments cover the whole sample space.

\[ E_i\cap E_j=\varnothing\quad (i\ne j) \]

\[ E_1\cup E_2\cup E_3\cup\cdots\cup E_n=S \]

Understanding sample space partitions for mutually exclusive and exhaustive events

Picture Method: First divide the sample space into fragments \(E_1,E_2,E_3,\ldots\). In Bayes’ theorem, the required event will be placed across these fragments.

Independent Events

Two events are independent if the occurrence of one event does not affect the probability of the other event.

\[ P(A\cap B)=P(A)P(B) \]

\[ P(A|B)=P(A),\qquad P(B|A)=P(B) \]

An Electronics Engineer may treat failure of a resistor in one independent board and failure of an unrelated cable as independent events if one failure does not affect the other.

A Computer Engineer may treat “a student forgets the password” and “the classroom projector fails” as independent events if there is no technical connection between them.

Probability of an Event

If all outcomes are equally likely, then the probability of an event \(A\) is:

\[ P(A)=\frac{n(A)}{n(S)} \]

Here, \(n(A)\) is the number of favourable outcomes and \(n(S)\) is the total number of possible outcomes.

\[ 0\leq P(A)\leq1 \]

Complement of an Event

The complement of \(A\), denoted by \(A'\) or \(\bar{A}\), means that event \(A\) does not occur.

\[ P(A)+P(A')=1 \]

\[ P(A')=1-P(A) \]

If probability that a component is defective is \(0.04\), then probability that it is not defective is \(1-0.04=0.96\).

Addition Theorem

For any two events \(A\) and \(B\):

\[ P(A\cup B)=P(A)+P(B)-P(A\cap B) \]

If \(A\) and \(B\) are mutually exclusive, then \(P(A\cap B)=0\). Hence:

\[ P(A\cup B)=P(A)+P(B) \]

Conditional Probability

Conditional probability means probability of one event when another event is already known to have occurred.

\[ P(A|B)=\frac{P(A\cap B)}{P(B)},\qquad P(B)\ne0 \]

A Civil Engineer may ask: What is the probability that concrete strength is low, given that curing was improper?

A Computer Engineer may ask: What is the probability that login fails, given that the server is down?

An Electronics Engineer may ask: What is the probability that the circuit output is wrong, given that the sensor is faulty?

Multiplication Theorem

From conditional probability:

\[ P(A\cap B)=P(B)P(A|B) \]

Also:

\[ P(A\cap B)=P(A)P(B|A) \]

If \(A\) and \(B\) are independent, then:

\[ P(A\cap B)=P(A)P(B) \]

Total Probability Theorem

Let \(E_1,E_2,\ldots,E_n\) be mutually exclusive and exhaustive events with non-zero probabilities. If \(A\) is any event, then:

\[ P(A)=P(E_1)P(A|E_1)+P(E_2)P(A|E_2)+\cdots+P(E_n)P(A|E_n) \]

Picture Method: \(E_1,E_2,\ldots,E_n\) are the fragments of the sample space. Event \(A\) will cut across those fragments.

Bayes’ Theorem

Let \(E_1,E_2,\ldots,E_n\) be mutually exclusive and exhaustive events with non-zero probabilities. If \(A\) is an event with \(P(A)\ne0\), then:

\[ P(E_i|A)=\frac{P(E_i)P(A|E_i)}{\sum_{j=1}^{n}P(E_j)P(A|E_j)} \]

Bayes’ theorem is used to find the probability of a cause when the effect is already known.