HOMEWORK.

Assignment 1: due Thursday, April 4, 2019, in class.
Please hand in one page answering the following questions.
A. In one paragraph, describe your background and interest in taking this course. Indicate any specific topics you are interested in.
B. In one paragraph,
(a) if you are from a department other than Mathematics, describe a problem where you would like to be able to use a stochastic process model; the n describe in words what X(t), the state of the process at time t would represent. Specify the time index set and the state space for your process. Try to formulate questions that you would like to be able to answer for this process. Try to draw a typical sample path.
(b) for Mathematics students, or those who do not have a problem in mind, describe a phenomenon in the real world where you think a stochastic pro cess model would be useful and then answer the same questions as in (a) above.
C. Describe the state space for a stochastic process that describes the random evolution in time of a finite DNA segment, where mutations, insertions and deletions of sites may occur. (You may assume that there are 4 possible values at each site, A, C, G, T.)

Assignment 2: due in class, Tuesday, April 16, 2019.
Problems last half of 1.1.3 (the part that starts on page 9), 1.1.4, 1.1.6, 1.2.1, 1.3.2, 1.3.3, 1.3.4 from the text by Norris.
Hint for problem 1.3.4: you may use the fact (due to Euler) that the sum from n=1 to infinity of 1/ n^2 is pi^2/6.
For solutions to Homework 2, click here. (You will need the login information provided in class on the first day (for the class notes) to access this site.)

Assignment 3: due Tuesday, April 30, 2019, 5pm, in the TA's dropbox in the basement of AP&M (her drobbox is on the top of Group 2; her name is Yingjia Fu).
Problems 1.5.1, 1.5.4 (note that the third part of this problem is quite challenging), 1.7.1 (here invariant distribution means stationary distribution), 1.8.2(b)(for this problem you will need to also solve 1.1.7b). In addition, for the Markov chain in 1.8.2(b), find the long run probability that the Markov chain is in each of the states 1,2,3. Also solve the following: Show that the Markov chain in Exercise 1.3.4 is transient and deduce that P(X(n) tends to infinity as n tends to infinity)=1 given X(0)=i for any i>=1. Exercise 1.9.1 (b), (e)(assume S is finite), 1.10.1.
Additional problem: Consider the Markov chain X with transition matrix given by P in EXAMPLE 1.2.2 on Page 11 of Norris. Prove that T=inf{n>10: X(n) = 6} is a stopping time for the Markov chain when started from the state labelled 1.
For solutions to homework 3, click here.

Assignment 4: due Tuesday, May 14, 2019 in class. Consider a Hidden Markov Model representation of a coin tossing experiment. Assume a two-state model (corresponding to two coins) with probability of output (of H or T) given by: P(H|Coin 1) = 0.5, P(T|Coin 1) = 0.5 and P(H|Coin 2)= 0.2, P(T|Coin 2)=0.8. Assume that the transition probability matrix for the Markov chain governing the coin state is: P(11) = 0.7, P(12)=0.3, P(21)=0.4, P(22) =0.6, where P(ij) denotes the probability that given that the coin is currently in state i, it is in state j at the next time step. Assume that initially the coin is equally likely to be coin 1 or 2.
(a) Suppose that you observe the sequence HHTTHTHT. What is a/the most likely coin state sequence to generate this output?
(b) What is the probability that, given the observation sequence, it was generated entirely by coin 1?
For solutions to homework 4, click here.

Homework 5, due Friday May 31, 5pm in the TA's homework box in the basement of AP&M. (The TA is Yingjia Fu).
For the homework assignment, click here.

Homework 6 (OPTIONAL) due 5pm, Tuesday, June 11, 2019 in the TA's homework box in the basements of AP&M. (The TA is Yingjia Fu). For the homework assignment, click here.