Notation

Notation

Reinforcement Learning

This section provides commenly used mathematical notation in reinforcement learning. The notation is based on the book "Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto.

Capital letters are used for random variables, lowercase letters for their values.

Markov Decision Processes

Notation	Explanation
$s, s'$	states
$a$	an action
$r$	a reward
$\mathcal{S}$	set of all nonterminal states
$\mathcal{S}^+$	set of all states plus a terminal state
$\mathcal{A}(s)$	set of actions available in state $s$
$\mathcal{R}$	set of possible rewards
$t$	discrete time step
$A_t$	action at time $t$
$S_t$	state at time $t$
$R_t$	reward at time $t$
$\pi$	policy
$\pi(s)$	action taken in state $s$
$$\pi(a	s)$$
$G_t$	return following time $t$
$h$	horizon
$G_{t:t+n}$
$$p(s',r	s,a)$$
$$p(s'	s,a)$$
$r(s,a)$
$r(s,a,s')$
$v_{\pi}(s)$	value of state $s$ under policy $\pi$ (expected return)
$v_{*}(s)$	value of state $s$ under the optimal policy
$q_{\pi}(s,a)$
$q_{*}(s,a)$
$V$	array estimates of state-value function $v_{\pi}$ or $v_{*}$