Instructions
The idea of this homework is to answer the questions in a style similar to what you would
write during an exam. So short answers are desirable, containing sufficient, but not overly
elaborate, content. Of course, you do not have the same time constraints, so we do expect
neat and organised presentation of your answers.
You should use R to compute any numerical solutions, and also to typeset your answers.
However, you must render your output to PDF before uploading to Moodle. (If you don’t
have a version of LaTeX installed on your computer, first ‘knit’ the .Rmd to ‘Word’ and then
print to PDF. ONLY PDF SUBMISSIONS WILL BE ACCEPTED.)
Note that corrections made (16 and 20 October) have been highlighted in red.
This homework assignment is an individual assignment, worth 8% of the semester mark.
It is due on Moodle by **12NOON** on Saturday 26 October 2019.
Q1. Let p denote the probability that the coin will turn up as a Head when tossed. Given n
independent tosses of the same coin, what is the probability distribution associated with the
number of Head outcomes observed?
Q2. Suppose you have information that a coin in your possession is not a fair coin, and
further that either Pr(Head|p) = p is certain to be equal to either p = 0.33 or p = 0.66.
Assuming you believe this information, but are otherwise uncertain about which of the two
values is the correct probability, detail the corresponding prior probabilities associated
with the possible values of p.
Q3. You set about trying to determine the correct value of p by independently tossing the
coin n times, and you observe exactly x outcomes. Using your prior distribution from Q2,
detail the form of the resulting posterior distribution and explain how it is derived.
Q4. Compute the relevant posterior probabilities in each of the following cases corresponding
to the setting in Q3:
a. You toss the coin only one time, and get a Head outcome.
b. You independently toss the coin 25 times, and get 11 Head outcomes (“Heads”).
c. You independently toss the coin 100 times, and get 48 Heads.
1
Now let p be defined according to your answer to Q4c as follows:
p =
(
0.33 if Pr(p=0.33|n=100, x=48)
Pr(p=0.66|n=100, x=48) > 1 from Q4c,
0.66 otherwise
You decide you really have a good feeling about this coin, but after seeing the results from
Q4c, you are no longer convinced that p could only be equal to one of the two values
(i.e. p
2 {0.33, 0.66}) thusfar considered. Instead, you now believe that the true value of p
must lie somewhere in the continuum p
2
(0, 1), and so choose a Beta(, ) distribution to
represent this belief. In addition, you believe
C1. Pr(0.2
0.8
|
, ) = 0.5, and
C2. E[ p
|
, ] = p.
**Q5.** No response required!
Q6. Detail two mathematical equations involving the hyper-parameters and that
satisfy the two conditions C1 and C2 for p above. Then explain how to use these equations
to determine the hyper-parameters of your Beta(, ) prior, and report their values. (Hint:
You can use R’s uniroot function.)
Q7. You now set off to toss another 100 coins, and this time you observed 51 heads. Explain
the process required to update your the Beta prior obtained in Q6, to produce your posterior
distribution for p.
Q8. Explain what is the Bayes estimator for p under squared error loss, and report its value
given your result from Q7.
Q9. Report a 95% credible interval (i.e. a posterior probability interval) for p, using your
posterior distribution from Q7. (Note: you may exclude 2.5% probability from each tail
of the posterior when constructing your credible interval.) Provide an explanation of your
approach.
Q10. Now consider the problem from a frequentist perspective, and show that ˆp = x/n is
the MLE when X
Binomial(n, p).
Q11. Explain what is meant by a ‘credibility factor’, and show that the mean of the
posterior distribution you obtained in Q7, denoted by E[p
|
X, n, , ], satisfies the credibility
relationship
E[p
|
X, n, , ] = Z ˆpMLE
+ (1
−
Z) E[p
|
, ],
where Z =
n
++n.
2
Q12. Explain what is meant by the sampling distribution of ˆpMLE.
Q13. Show that the estimate of the Fisher information matrix, I(p), given by
1
n
J(ˆp) =
−
d2 lnL
n(p)
n dp2
p=ˆpMLE
where
L
n(p) denotes the likelihood function, satisfies 1
nJ(ˆp) = 1
ˆpMLE(1−ˆpMLE) . Use this,
together with the CLT for MLEs, given by
p
n
ˆ
n
−
D
!
N(0, I()−1), as n
! 1
and provide a justification for the approximation given by ˆpMLE
approx
N(p, ˆpMLE (1−ˆpMLE)
n
).
Q14. Use the approximation from Q13 to report a 95% CLT-based confidence interval for p
that is based on the MLE, ˆp, given you have observed 51 heads out of 100 coin tosses.
Q15. Carefully describe how to construct a 95% bootstrap confidence interval for p, based
on the results of the 100 independent Bernoulli(p) trials that resulted in observing 51 heads
out of 100 coin tosses. (Note, you do not have to implement this, just explain it!)
Q16. Discuss the relative advantages and disadvantages of reporting
i) a CLT-based 95% confidence interval for p,
ii) a bootstrap-based 95% confidence interval for p, and
iii) a 95% credible interval for p.
3