A robot is going to clean an office with layout as Figure 1. However, this robot has no prior knowledge of office layout. The robot equips sensors to perfectly sense the status of 4 immediate...

A robot is going to clean an office with layout as Figure 1. However, this robot has no prior knowledge of office layout. The robot equips sensors to perfectly sense the status of 4 immediate neighboring grids (left, right, up, down) and move accordingly. The black grids are prohibited to enter (i.e. wall). The robot has memory to store the grids being visited and being sensed, with relative position to the entry point to form its own reference (i.e. map). Suppose all colored grids are not accessible. Please develop the RL algorithm and simple planning algorithm based on the memory to visit all white grids and leave from the entrance (indicated by green and red triangles) of the office. How many steps does the robot take based on your programming? Please submit a MP4 video to show how your robot moves.


1. A robot is going to clean an o_ce with layout as Figure 1. However, this robot has no prior knowledge of o_ce layout. The robot equips sensors to perfectly sense the status of 4 immediate neighboring grids (left, right, up, down) and move accordingly. The black grids are prohibited to enter (i.e. wall). The robot has memory to store the grids being visited and being sensed, with relative position to the entry point to form its own reference (i.e. map). Suppose all colored grids are not accessible. Please develop the RL algorithm and simple planning algorithm based on the memory to visit all white grids and leave from the entrance (indicated by green and red triangles) of the o_ce. How many steps does the robot take based on your programming? Please submit a MP4 video to show how your robot moves.