OK, quiz time, let's say we have these rewards, zero point one zero point one zero point one zero 1
point one zero point one. 2
And this is, let's say reward, let's say two. 3
And this award is one. 4
And this award is, let's say minus one. 5
And this award is also zero point one. 6
Let's say a particular agent is following a policy and let's say the environment is also, let's say 7
deterministic. 8
And the agent is following a sequence of moves like this. 9
That's a red agent. 10
Let me call that agent, red agent. 11
So in this state, the agent goes back and like this or rather than rather than writing this as 12
a policy, let me just write that as a loop. 13
Then the agent actually is exploring this particular sequence of states. 14
So it is moving to that, then to that, then to that and then to that. 15
And then that and that. 16
And then it moves back here, then here, then here, then here, then here and here and then moves 17
back. 18
So which means it is observing a reward of point one point, one point, one point, one point one, 19
and keep on observing this infinitely many times, which means if we add all these rewards, then whatever 20
sequence, whatever the sequence is, that kind be that that might be called as a reward of or the total 21
reward of this particular sequence that we are moving it. 22
Consider another sequence, for example, consider another sequence. 23
That is this. 24
This sequence, if you see the sequence, that is zero point one, zero point one. 25
Zero point one. 26
dot dot dot then a two, then again, zero point one plus zero point one plus zero point one, then dot 27
dot dot then plus a two, then so on. 28
And this is moving this way. 29
Now the question is for the perspective of agent, which is going to collect the maximum total reward 30
for the future, which sequence of states, if the agent visits, if the agent is allowed to go to this 31
sequence or that sequence, which one is better? 32
Hope to see you with the answer.