A machine in excellent condition earns $100 profit per week, a machine in good condition earns $70 per week, and a machine in bad condition earns $20 per week. At the beginning of any week, a machine...

A machine in excellent condition earns $100 profit per week, a machine in good condition earns $70 per week, and a machine in bad condition earns $20 per week. At the beginning of any week, a machine may be sent out for repairs at a cost of $90. A machine that is sent out for repairs returns in excellent condition at the beginning of the next week. If a machine is not repaired, the condition of the machine evolves in accordance with the Markov chain shown in Table.

The company wants to maximize its expected discounted profit over an infinite horizon (β = .9).

a Use policy iteration to determine an optimal stationary policy.

b Use linear programming to detemine an optimal stationary policy.

c Perform two iterations of value iteration.