How I enable to get trained policy of Isaac-Repose-Cube-Allegro-v0 task?

[cold-young]

Hi there,

I have been trying to train the Isaac-Repose-Cube-Allegro-v0 task using the skrl PPO algorithm.
(num_envs 2048, <1 hour)
Even though the reinforcement learning reward graph has converged, I cannot get an expert policy.
I think I need well defined shaped rewards or other learning approaches like imitation learning.
By the way, I found a good demonstration of in-hand manipulation with shadow hand. on this page
To acquire expert policy, what i do ?

Thanks

[RandomOakForest]

Thanks for posting this. By expert policy, do you mean the final trained policy? You may find them in this directory:

./IsaacLab/logs/<learning_library_name>/<task_name>

[cold-young]

Thank you for your answer.

I mean, I cannot get a well-trained policy using the "manager_based" Isaac-Repose-Cube-Allegro-v0 task.

I was able to acquire expert policy using "Isaac-Repose-Cube-Allegro-Direct-v0" task.

Although the "manager_based" task is useful to adjust many parameters (in reward and states), I cannot get expert policy. (Maybe I should change many parameters in the Isaac-Repose-Cube-Allegro-v0 task).
How do I do for getting well-trained policy using "Isaac-Repose-Cube-Allegro-v0" task.
Default setting is not work to train

[StrainFlow]

I think the first step is for us to reproduce what you're seeing. Can you give me reproduction steps for your two policies so I can see if I get the same results? Are you using the samples found here directly:

https://isaac-sim.github.io/IsaacLab/main/source/overview/environments.html

Are these also the shadow hand examples you're looking for?

[cold-young]

Hi,

I used exactly this example without any changes.
"./isaaclab.sh -p source/standalone/workflows/skrl/train.py --task Isaac-Repose-Cube-Allegro-v0 --num_envs 4096 --headless" and
"./isaaclab.sh -p source/standalone/workflows/skrl/train.py --task Isaac-Repose-Cube-Allegro-Direct-v0 --num_envs 4096 --headless"

I think "Isaac-Repose-Cube-Allegro-Direct-v0" has well-defined parameters (rewards), but "Isaac-Repose-Cube-Allegro-v0" doesn't.
And I found the radomisation of "Isaac-Repose-Cube-Allegro-v0" is more varied including mass, texture.

Thanks for your help.

[jeffyu880]

Hi @cold-young Were you able to determine how to get the Allegro-v0 task to give a good policy? I have run into the same issue, where the policy does not perform well with the default training parameters. I increased the timesteps and the mean reward is not any better. As seen in the image below