How Q-learning learns action values from experience: bootstrapping, the TD target and TD error, off-policy vs SARSA, convergence, and Deep Q-Networks.