Pebellslearn by building
0%
RoadmapsMatrixCoursesSandboxSign in
Types of Machine Learning  ›  Capstone Project

Reinforcement Learning

Reinforcement Learning 16 minIn reinforcement learning an agent acts in an environment, receives rewards, and learns a policy that maximises total reward over time, balancing exploration against exploitation.

Reinforcement Learning

The agency wants software that adjusts a listing's asking rent each week. Nobody can hand it a table of "correct" prices; the only feedback arrives later, when a tenant signs or the flat sits empty. Learning from delayed rewards, by trial and error, is reinforcement learning (RL).

The rule

An agent acts, the world replies with reward; learn actions that maximise total reward.

The parts, for the pricing problem:

  • Agent: the pricing program.
  • Environment: the rental market it acts in.
  • State: what the agent knows now: weeks listed, current asking rent, enquiries so far.
  • Action: raise by ₹1 thousand, keep, or lower by ₹1 thousand.
  • Reward: a number after each step, such as the rent earned when a tenant signs, minus a penalty for every empty week.
  • Policy: the agent's rule for choosing an action in each state. This is what RL learns.

Unlike supervised learning, the agent is never told the right action. It only sees a reward after acting, and rewards may come many steps later. Future rewards are usually discounted by a factor γ\gamma between 0 and 1, because a reward sooner is worth more. With γ=0.9\gamma = 0.9 and rewards 0, 0, 22 over three weeks, the return is

0+0.9×0+0.92×22=0.81×22=17.820 + 0.9 \times 0 + 0.9^2 \times 22 = 0.81 \times 22 = 17.82

The central tension is exploration versus exploitation. Exploiting means choosing the action the agent currently rates best. Exploring means trying something else, in case it is better. An agent that only exploits never discovers that a lower price might fill the flat two weeks sooner. A simple balance is ε-greedy: with probability ε (say 0.1) pick a random action, otherwise pick the best-known one.

Methods such as Q-learning estimate the value of each action in each state from experience, and the policy then picks the highest-valued action. The same ideas power game-playing agents and robot controllers.

The shortcut

Find the agent, the actions and the reward. If you cannot name a reward that arrives after acting, it is not RL.

One episode of the pricing agentStep 1 of 4

  1. Week 1

    State: new listing at 24. The agent keeps the price. No tenant: reward 0.

Drills

Answer fast — trust the rule.

quick drill

Which of these is a reinforcement learning problem?

quick drill

With discount 0.9, the rewards over three steps are 0, 0 and 22. What is the discounted return?

quick drill

An ε-greedy agent uses ε = 0.1. What share of its actions are random?

quick drill

What is the key difference from supervised learning?

Remember

  • An agent acts, the world replies with reward; learn actions that maximise total reward.
  • Shortcut: name the agent, the actions and the reward.
  • Trap: an agent that never explores, and so never finds a better action.

Practice — level up

0/1 solved

Write your answer, then compare it with a rubric and a model answer.

ExplainDesign a pricing agent

Describe a reinforcement learning agent that adjusts a listing's asking rent each week. Name its state, actions and reward, compute the discounted return of rewards 0, 0 and 22 with a discount of 0.9, and explain why it must sometimes explore.