Mastering Bayesian Mixture Models and System Reliability: A Spring 2025 Homework Guide

Introduction: Why Bayesian Mixture Models Matter in 2025

In the world of data science and engineering, Bayesian methods are increasingly used to incorporate prior knowledge into statistical inference. This tutorial focuses on two classic problems from ISYE 6420 Spring 2025 Homework 2: posterior inference with a mixture normal prior and reliability analysis of a k-out-of-n system with Weibull components. By the end, you'll understand how to compute posterior distributions and system probabilities using Bayes' theorem and binomial tails. These skills are essential for modern applications like AI model uncertainty, predictive maintenance in smart factories, and even fantasy sports analytics.

Problem 1: Posterior with a Mixture Normal Prior

Suppose we have data yi ~ N(μ, 1) for i = 1,...,n, and a prior that is a 50-50 mixture of two normals: μ ~ 0.5 N(-1, 1) + 0.5 N(1, 1). Given a single observation y = 1, we want the posterior distribution of μ. This scenario is reminiscent of modeling a player's true ability in esports, where prior belief might be split between two skill levels.

Step 1: Bayes Theorem

We have p(μ | y) ∝ p(y | μ) p(μ). The likelihood p(y | μ) is N(μ, 1) density. The prior is a mixture: p(μ) = 0.5 φ(μ; -1, 1) + 0.5 φ(μ; 1, 1). So the posterior is proportional to:

p(μ | y) ∝ φ(y; μ, 1) * [0.5 φ(μ; -1, 1) + 0.5 φ(μ; 1, 1)]

Step 2: Use the Product of Normal Densities Result

We apply the identity: φ(x; μ1, σ12) φ(x; μ2, σ22) = φ(x; (μ1/σ12 + μ2/σ22)/(1/σ12 + 1/σ22), 1/(1/σ12 + 1/σ22)) * φ(μ1-μ2; 0, σ12+σ22).

For the first component: φ(y; μ, 1) φ(μ; -1, 1) = φ(y; μpost1, σpost12) * φ(μ; μpost1, σpost12) * constant? Actually, we need to treat μ as variable. Let's derive carefully:

We have φ(y | μ) = φ(μ; y, 1) (since symmetric in mean). So product φ(μ; y, 1) φ(μ; -1, 1). Using identity with x = μ, μ1 = y, σ12 = 1, μ2 = -1, σ22 = 1. Then the product equals φ(μ; (y/1 + (-1)/1)/(1/1+1/1), 1/(1/1+1/1)) * φ(y - (-1); 0, 1+1) = φ(μ; (y-1)/2, 1/2) * φ(y+1; 0, 2). Similarly for second component: φ(μ; y, 1) φ(μ; 1, 1) = φ(μ; (y+1)/2, 1/2) * φ(y-1; 0, 2).

Thus posterior is proportional to:

p(μ | y) ∝ 0.5 * φ(μ; (y-1)/2, 1/2) * φ(y+1; 0, 2) + 0.5 * φ(μ; (y+1)/2, 1/2) * φ(y-1; 0, 2)

For y=1: φ(1+1;0,2)=φ(2;0,2) and φ(1-1;0,2)=φ(0;0,2). So constants differ. Let w1 = 0.5 * φ(2;0,2), w2 = 0.5 * φ(0;0,2). Then posterior is a mixture: w1*N((1-1)/2=0, 1/2) + w2*N((1+1)/2=1, 1/2) normalized. Compute φ(2;0,2) = (1/√(4π)) exp(-2^2/(4)) = (1/√(4π)) e^{-1} ≈ 0.0483. φ(0;0,2)=1/√(4π) ≈ 0.1995. So w1=0.02415, w2=0.09975. Normalized weights: w1' = 0.02415/(0.02415+0.09975)=0.195, w2'=0.805. So posterior is μ|y ~ 0.195 N(0, 0.5) + 0.805 N(1, 0.5).

Problem 2: k-out-of-n System Reliability with Weibull Components

An engineering system is operational if at least k out of n components work. Components have independent lifetimes with Weibull survival: P(T ≥ t) = exp(-λ tr). Given n=10, k=7, r=1.3, λ=1/20 per month. This is like a multiplayer game server where at least 7 out of 10 nodes must be online.

Part 1: Probability system is operational at time t

Let X be number of components working at time t. X ~ Binomial(n, p(t)) where p(t) = exp(-λ tr). System operational if X ≥ k. So P(operational at t) = P(X ≥ k) = 1 - P(X ≤ k-1). For t=6: p(6)=exp(-(1/20)*6^1.3). Compute 6^1.3 = exp(1.3 ln 6) = exp(1.3*1.7918)=exp(2.3293)=10.27. Then λ t^r = (1/20)*10.27=0.5135. So p=exp(-0.5135)=0.598. Then P(X ≥ 7) = 1 - binocdf(6,10,0.598) in MATLAB, or using Python: 1 - scipy.stats.binom.cdf(6,10,0.598). Let's compute approximate: P(X=7)=C(10,7)*0.598^7*0.402^3=120*0.598^7*0.402^3. 0.598^7≈0.030, 0.402^3=0.065, product=0.00195, times 120=0.234. Similarly X=8,9,10 sum to about 0.234+0.124+0.028=0.386. So approx 0.386.

Part 2: Given system operational at t=6, probability exactly 7 components work

We need P(X=7 | X≥7) = P(X=7)/P(X≥7). From above, P(X=7)≈0.234, P(X≥7)=0.386, so ratio=0.606.

Part 3: Given operational at t=6, probability operational at t=9

We need P(system operational at t=9 | operational at t=6). This requires joint distribution of X at two times. Since components are independent and lifetimes are Weibull, the conditional probability is not trivial but can be computed using the fact that given a component is working at t=6, its remaining lifetime is Weibull with same shape but shifted? Actually, Weibull is memoryless only for r=1. For r=1.3, we need to compute P(T>9 | T>6) = exp(-λ(9^r - 6^r)). So p_cond = exp(-λ(9^1.3 - 6^1.3)). Compute 9^1.3 = exp(1.3 ln 9)=exp(1.3*2.1972)=exp(2.8564)=17.40. So λ(17.40-10.27)=0.5135*7.13=3.66? Wait λ=1/20=0.05, so 0.05*7.13=0.3565. So p_cond=exp(-0.3565)=0.700. Then number working at t=9 given working at t=6 is Binomial(10, p_cond) but only for components that were working? Actually, we need the number working at t=9 among those that were working at t=6. Let Y be number working at t=9 among the X working at t=6. Then total working at t=9 is Y + (components that failed before t=6 but revived? No, can't revive). So given X at t=6, the number at t=9 is Binomial(X, p_cond). So P(operational at t=9 | operational at t=6) = sum_{x=7}^{10} P(X=x | X≥7) * P(Y ≥ 7 - (10-x)? Actually, we need at least 7 working at t=9. Let X be number working at t=6. Given X, number working at t=9 is Binomial(X, p_cond). So P(≥7 at t=9 | X) = P(Bin(X, p_cond) ≥ 7). But we also have components that failed before t=6 don't contribute. So we need to sum over x from 7 to 10: P(X=x | X≥7) * P(Bin(x, p_cond) ≥ 7). For x=7: need all 7 survive: p_cond^7. For x=8: need at least 7 survive: C(8,7)p_cond^7(1-p_cond)+p_cond^8. etc. Using p_cond=0.7, compute: x=7: 0.7^7=0.08235; x=8: C(8,7)*0.7^7*0.3 + 0.7^8 = 8*0.08235*0.3 + 0.05765 = 0.1976+0.0576=0.2552; x=9: C(9,7)*0.7^7*0.3^2 + C(9,8)*0.7^8*0.3 + 0.7^9 = 36*0.08235*0.09 + 9*0.05765*0.3 + 0.04035 = 0.267+0.1556+0.04035=0.4629; x=10: 1 - P(Bin(10,0.7)≤6) = 1 - binocdf(6,10,0.7) ≈ 1 - 0.3504 = 0.6496. Now we need P(X=x | X≥7) from part 1: P(X=7)=0.234, P(X=8)=0.124, P(X=9)=0.028, P(X=10)=? Actually, P(X=10)=p^10=0.598^10=0.006. Sum=0.234+0.124+0.028+0.006=0.392 (close to 0.386). So conditional probabilities: P(X=7|≥7)=0.234/0.392=0.597, P(X=8)=0.124/0.392=0.316, P(X=9)=0.028/0.392=0.071, P(X=10)=0.006/0.392=0.015. Then overall probability = 0.597*0.08235 + 0.316*0.2552 + 0.071*0.4629 + 0.015*0.6496 = 0.0492 + 0.0806 + 0.0329 + 0.0097 = 0.1724. So about 17.24%.

Conclusion

This tutorial walked through key Bayesian and reliability concepts from ISYE 6420 homework 2. Understanding mixture posteriors and system reliability is crucial for modern engineering and data science. Practice with these methods will prepare you for advanced topics like Bayesian neural networks and predictive maintenance in Industry 4.0.