[Solved] EECS491 Assignment1-Basic probability

Q1. Basic probabilityIn the proofs below you should use general probability distributions (as opposed to specific examples) and thebasic laws of probability. Be concise and clear. The proof should be in terms of mathematical facts of probabilitytheory.1.1. Prove1.2. ProveQ2. IndependenceAgain these proofs should use general probability distributions and the basic laws of probability. Note that the proofshould be in terms of mathematical facts. It should not be an argument that depends on real-world knowledge. Theexample should use common real-world knowledge and interpretation should convey the ideas of the proof.2.1 Prove that independence is not transitive, i.e. . Define a joint probability distributionfor which the previous expression holds and provide an example with an interpretation. (5 pts)2.2 Prove that conditional independence does not imply marginal independence, i.e. . Againprovide an example that illustrates the statement.Q3. Inspector Clouseau re-revisited3.1 Write a program to evaluate in Example 1.3 in Barber. Write your code and choose your datarepresentations so that it is easy to use it to solve the remaining questions. Show that it correctly computes thevalue in the example.3.2 Define a different distribution for . Your new distribution should result in the outcome thatis either or , i.e. reasonably strong evidence. Use the original values of and from theexample. Provide (invent) a reasonble justification for the value of each entry in .3.3 Derive the equation for .3.4 Calculate its value for both the original and the one you defined yourself. Is it possible to provide asummary of the main factors that contributed to the value? Why/Why not? Explain. (5 pts)Q4. Biased views4.1 Write a program that calculates the posterior distribution of the (probability of heads) from the Binomialdistribution given heads out of trials. Feel to use a package where the necessary distributions are defined asprimitives.4.2 Imagine three different views on the coin bias:I believe strongly that the coin is biased to either mostly heads or mostly tails.I believe strongly that the coin is unbiased.I dont know anything about the bias of the coin.Define and plot prior distributions that expresses each of these beliefs. Provide a brief explanation. (5 pts)4.3 Perform Bernoulli trials where one of these views is correct. Show how the posterior distribution of changesfor each view for =0, 1, 2, 5, 10, and 100. Each view should have its own plot, but with the curves of the posteriorafter different numbers of trials overlayed.4.4 Is it possible that each view will always arrive at an accurate estimate of ? How might you determine whichview is most consistent with the data after trials?Q5. Inference using the Poisson distributionSuppose you observe for 3 seconds and detect a series of events that occur at the following times (in seconds):0.53, 0.65, 0.91, 1.19, 1.30, 1.33, 1.90, 2.01, 2.48.5.1 Model the rate at which the events are produced using a Poisson distribution where is the number of eventsobserved per unit time (1 second). Show the likelihood equation and plot it for three different values of : less,about equal, and greater than what you estimate (intuitively) from the data.5.2 Derive the posterior distribution of assuming a Gamma prior (usually defined with parameters and ). Theposterior should have the form where is the total duration of the observation period and is thenumber of events observed within that period.5.3 Show that the Gamma distribution is a conjugate prior for the Poisson distribution, i.e. it is also a Gammadistribution, but defined by parameters and that are functions of the prior and likelihood parameters. (5 pts)5.4 Plot the posterior distribution for the data above at times = 0, 0.5, and 1.5. Overlay the curves on a single plot.Comment how it is possible for your beliefs to change even though no new events have been observed. (5 pts)Q6. Probability Distribution ExampleIn this problem you will illustrate a probability distribution in a settings of your choosing. It can be discrete orcontinuous. This is meant to be a simpler version of the letter seqeunce example shown in class (so dont use that).Your example should use two random variables that each have at least three distinct values (if it is discrete), i.e.dont use binary variables. The variables should not be independent, in other words, the setting you are modelingshould have structure, and ideally structure that is interesting and interpretable in some way. Your example shouldinclude the following:a decription of the settingan illustration of the joint probability and how it captures the structurean illustration of a conditional probabilityan illustration of marginal probabilityNote that illustration here means to explain with tables or figures that convey the ideas of the mathematicaloperations. The motivation behind this exercise is to help you develop a better understanding of how jointprobability distributions model probabilistic structure in a simplified setting, so try to choose something you arevery familiar with. If find this is getting too long, you can continue it as part of the exploration, but there you willalso need to add and inference problem.Exploration (40 pts)In these problems, you are meant to do creative exploration. Define and explore:E.1 A discrete inference problemE.2 A continuous inference problemThis is meant to be open-ended; you should not feel the need to write a book chapter; but neither should you justchange the numbers in one of the problems above. After doing the readings and problems above, you should pick aconcept you want to understand better or an simple modeling idea you want to try out. You can also start to exploreideas for your project. The general idea is for you to teach yourself (and potentially a classate) about a conceptfrom the assignments and readings or solidify your understanding of required technical background. For additionalguidance, see the grading rubric below.You can use the readings and other sources for inspiration, but here are a few ideas:An inference problem using categorical dataA disease for which there are two different testsA two-dimensional continuous inference problemThe idea of a conjugate priorExploration Grading RubricExploration problems will be graded according the elements in the table below. The scores in the column headersindicate the number of points possible for each rubric element (given in the rows). A score of zero for an element ispossible if it is missing entirely.Substandard (+1) Basic (+2) Good (+3) Excellent (+5)PedagogicalValueNo clear statement ofidea or concept beingexplored or explained;lack of motivatingquestions.Simple problem withadequate motivation;still could be a usefuladdition to anassignment.Good choice of problem witheffective illustrations ofconcept(s). Demonstrates adeeper level of understanding.Problem also illustrates orclarifies common conceptualdifficulties ormisconceptions.Novelty ofIdeasCopies existingproblem or makes onlya trivial modification;lack of citation(s) forsource of inspiration.Concepts are similar tothose covered in theassignment but withsome modifications ofan existing exericse.Ideas have clear pedagogicalmotivation; creates differenttype of problem or exercise toexplore related or foundationalconcepts more deeply.Applies a technique orexplores concept notcovered in the assignment ornot discussed at length inlecture.Clarity ofExplanationLittle or confusingexplanation; figureslack labels or usefulcaptions; noexplanation ofmotivations.Explanations arepresent, but unclear,unfocused, wordy orcontain too muchtechnical detail.Clear and concise explanationsof key ideas and motivations.Also clear and concise, butincludes illustrative figures;could be read andunderstood by students froma variety of backgrounds.Depth ofExplorationContent is obvious orclosely imitatesassignment problems.Uses existing problemfor different data.Applies a variation of atechnique to solve a problemwith an interesting motivation;explores a concept in a series ofrelated problems.Applies several concepts ortechniques; has clear focusof inquiry that is approachedfrom multiple directions.p(x, y|z) = p(x|z)p(y|x, z)p(x|y, z) = p(y|x, z)p(x|z)p(y|z)a b b c a cp(a, b, c)a b|c a bp(B|K)p(K|M, B) p(B|K)< 0.1 > 0.9 p(B) p(M)p(K|M, B)p(M|K)p(K|M, B)y nnn n p(|n, T, , ) T n TIn [ ]