Error in mc_prediction.py

[RacoFernandez]

Hi!

First of all, thanks for your code. I'm using it in a reinforement learning course and it's really useful.

In mc_prediction.py I'm getting this error when I try to implement it:

My implementation:

def sample_policy(observation):
    """
    Una política que se planta si el jugador tiene un puntaje > 20 y pide otra carta caso contrario.
    """
    score, dealer_score, usable_ace = observation
    return 0 if score >= 20 else 1

V_10k = mc_prediction(sample_policy, env, num_episodes=10000)
plotting.plot_value_function(V_10k, title="10,000 Episodios")

V_500k = mc_prediction(sample_policy, env, num_episodes=500000)
plotting.plot_value_function(V_500k, title="500,000 Episodios")

The error message I get:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-30-42df6d70edc6> in <module>
----> 1 V_10k = mc_prediction(sample_policy, env, num_episodes=10000)
      2 plotting.plot_value_function(V_10k, title="10,000 Episodios")
      3 
      4 V_500k = mc_prediction(sample_policy, env, num_episodes=500000)
      5 plotting.plot_value_function(V_500k, title="500,000 Episodios")

<ipython-input-28-40e4c9202b71> in mc_prediction(policy, env, num_episodes, discount_factor)
     35             # choose the action based on policy
     36             probs = policy(current_state)
---> 37             action = np.random.choice(len(np.arange(probs)), p=probs)
     38 
     39             next_state, reward, done, _ = env.step(action)

mtrand.pyx in mtrand.RandomState.choice()

TypeError: object of type 'int' has no len()

Maybe I'm doing something wrong!

Thanks in advance,

Ramiro.