Markov Decision Process Solver
Markov Decision Process Solver computes the optimal policy and value function for a 3-state, 2-action MDP using value iteration. Enter transition probabilities for each action and state, rewards, and the discount factor \\( \gamma \\).
MDP Solution via Value Iteration
The solver uses value iteration to find the optimal policy and value function for a Markov Decision Process. The value function \\( V(s) \\) satisfies the Bellman equation:
\\[
V(s) = \max_a \left[ R(s, a) + \gamma \sum_{s’} P(s’|s, a) V(s’) \right]
\\]
Where:
- \\( R(s, a) \\): Reward for state \\( s \\) and action \\( a \\).
- \\( P(s’|s, a) \\): Transition probability to state \\( s’ \\) given state \\( s \\) and action \\( a \\).
- \\( \gamma \\): Discount factor (0 ≤ \\( \gamma \\) ≤ 1).
Value Iteration: Iteratively update \\( V(s) \\) until convergence, then extract the optimal policy \\( \pi(s) \\).