---
author: "Your name here"
title: "Math 365 Exam 3"
output: pdf_document
---

Please answer each problem as clearly and completely as you can.  Do not discuss these problems with other students, or anyone else but me. You may use your textbook, lecture notes, class materials (including those posted on the website), and homework, but do not use other books, the internet, or any materials other than those directly associated with the course.  Please do feel free to ask me questions, either via email or coming by my office.  Show all work to demonstrate that you understand your answer. You may use R for any computations; submit either your R script or pdf knit from your R markdown.

This exam is due 4pm Thursday May 10. Late submissions will be penalized by 10 points per day. You may submit by emailing me your R markdown pdf or as a combination of paper and R work.

## Problem 1 (20pt)

Suppose that a continuous-time Markov chain with state space $S=\{1,2,3,4\}$ has generator matrix
```{r}
Q<-matrix(0,4,4)
Q[1,1:3]<-c(-3,2,1)
Q[2,1:3]<-c(1,-2,1)
Q[3,1:4]<-c(1,1,-3,1)
Q[4,2:4]<-c(1,0,-1)
Q
```
(a) Find the equilibrium distribution $\pi$.

(b) If $X_0=2$, what is the expected amount of time until it changes state for the first time?

(c) If $X_0=1$, what is the expected amount of time until the chain first hits state 3?

(d) If $X_0=4$, what is the expected amount of time until the chain first hits state 3?

## Problem 2 (25pt) 

Let $B_t$ denote a standard Brownian motion. Calculate the probability of each of the following events:

(a) $B_2>B_1$

(b) $B_2>B_1>B_3$

(c) $B_t=0$ for some $t$ with $2\le t\le 3$

(d) $B_t<4$ for all $t$ with $0<t<3$

(e) $B_t>0$ for all $t>10$

## Problem 3 (15pt)  

Suppose that $N_t$ is a Poisson process with arrival rate parameter $\lambda=1$. Determine the conditional expectations $\mathbb{E}(N_2 | N_1)$ (how many total arrivals during [0,2] we expect if we know how many arrived during [0,1]) and $\mathbb{E}(N_1 | N_2)$ (how many arrivals we expect occurred during [0,1] if we know how many total arrived during [0,2]).

## Problem 4 (20pt) 

(a) Assume that $Z_0, Z_1, \dots$ is a branching process whose offspring distribution has mean $\mu$. Demonstrate that $M_n= Z_n/\mu^n$ is a martingale by proving that $E(M_{n+1}\,|\,M_n)=M_n$.

(b) Let $N_t$ be a Poisson process with arrival rate parameter $\lambda$, and define $Y_t=N_t-\lambda t$ for $t\ge0$. Prove that $Y_t$ is a martingale with respect to $N_t$ (according to the definition on page 358; prove both conditions).

## Problem 5 (20pt)  

Consider a two-server queueing system where customers arrive according to a Poisson process with rate $\lambda$ and  service times are exponential with rate $\mu$. Moreover, suppose that arrivals finding both servers busy immediately depart (don't enter the system), whereas those finding at least one free server immediately enter service and then depart when their service is completed.

(a) If both servers are presently busy, what is the expected time until the next customer enters the system?

(b) If the system starts with no customers, find the expected time until both servers are busy. Hint: think about the first two events and what situations can occur.

## Bonus Problem (5pt extra credit)

Consider this simple game: Flip a fair coin twice. You win if you get two heads and lose otherwise. The probability of winning is 1/4. Design a game involving only flips of fair coins such that you have a 1/3 probability of winning.