3. Consider a home energy storage (battery) system that can store up to 2 units of energy. At every time step, there is a demand for energy in the home which is drawn from 0,1,2 units with equal...

Extracted text: 3. Consider a home energy storage (battery) system that can store up to 2 units of energy. At every time step, there is a demand for energy in the home which is drawn from 0,1,2 units with equal probability independent of the demands in the previous time steps. At every point in time, you have to satisfy the demand either by discharging the needed energy from the battery or purchasing power from the grid (or a combination of the two). You could also choose to purchase power from the grid to charge your battery. The grid energy price is either H(igh) or L(ow) according to a Markov chain (Price moves from H to L with probability p, and from L to H with probability q). (a) Model the decision making as an infinite horizon MDP where the objective is to minimize the discounted cost of energy purchased over an infinite horizon. (b) Write down a policy a of your choosing. Perform two steps of the operator T, for your policy, followed by one step of T..