Why does it work so well in nchain env

[soonjune]

I ran your code on nchain and it seems to work too well. Using DQN and BootstrappedDQN agents, we found the optimal reward of 10.0 too quick (900 episodes for DQN and within 50 episodes for BootstrappedDQN). This is too fast compared to the original paper, Deep Exploration via BootstrappedDQN which suggests around 500 episodes to reach max reward for 100 states. (https://arxiv.org/pdf/1602.04621.pdf)

[soonjune]

Also, when I change the reward setting to left=0.05 and right=0.5 BootstrappedDQN moves right which is akward.

[Chen-Ginger]

I ran your code on nchain and it seems to work too well. Using DQN and BootstrappedDQN agents, we found the optimal reward of 10.0 too quick (900 episodes for DQN and within 50 episodes for BootstrappedDQN). This is too fast compared to the original paper, Deep Exploration via BootstrappedDQN which suggests around 500 episodes to reach max reward for 100 states. (https://arxiv.org/pdf/1602.04621.pdf)

I had the same problem. I ran DQN with epsilon in 100-Chain environment and found the agent could get max reward in several hundred episoids, which is really strange. Did you figure out why this happened?