Skip to content

first version of training loop for dyna #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Misterhoonster wants to merge 32 commits into
base: main
Choose a base branch
from
Open

first version of training loop for dyna #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
81c6ad0
created dyna skeleton + setup for training loop
Misterhoonster Apr 6, 2025
4e8ffab
training loop v1
Misterhoonster Apr 8, 2025
2a49273
removed runnerstate
Misterhoonster Apr 12, 2025
227f070
change var names to runner_state
Misterhoonster Apr 12, 2025
1b94a63
created dyna agent
Misterhoonster Apr 12, 2025
ff2ff9c
renamed TimeStep.env_state to Timestep.state
Misterhoonster Apr 13, 2025
abe5b4d
updated actor_step fn to work with new DynaAgent, added TimestepWrapper
Misterhoonster Apr 13, 2025
bdab0e6
autoreset false on crafter
Misterhoonster Apr 14, 2025
820b218
added simpolicy, finished total_loss fn
Misterhoonster Apr 23, 2025
ac9382e
finished loss class v1
Misterhoonster Apr 25, 2025
1f04915
removed rolling windows
Misterhoonster May 4, 2025
e0549b8
added encoder class
Misterhoonster May 4, 2025
8344fde
integrated encoder
Misterhoonster May 4, 2025
d2a547c
fixed simpolicy for actor and sim
Misterhoonster May 5, 2025
076b58c
added rolling windows back with window_size=1
Misterhoonster May 5, 2025
4d9f455
logging v1
Misterhoonster May 6, 2025
aa87b2a
fixed optimizer + added incremental updates for target network
Misterhoonster May 6, 2025
ce77e16
fixed naming and typing issues
Misterhoonster May 6, 2025
c45b9cc
fix env.step syntax
Misterhoonster May 7, 2025
144e778
added q_heads and MLP class to DynaAgent
Misterhoonster May 7, 2025
1eb3749
cleaned up logger; added gradient logging
Misterhoonster May 7, 2025
a83917f
set up wandb
Misterhoonster May 7, 2025
73e0ff4
removed jaxneurorl import; added fns to file
Misterhoonster May 7, 2025
6be7f65
first runnable version
Misterhoonster May 7, 2025
ffa896a
update dynaagent initialize carry
Misterhoonster May 8, 2025
25eaca0
bugs fixed up to and including simulate_n_trajectories
Misterhoonster May 8, 2025
6f3f80a
bug fixes up to _learn_step
Misterhoonster May 8, 2025
aa4fab2
full code runs without errors
Misterhoonster May 8, 2025
370de3e
added reqs
Misterhoonster May 8, 2025
bcd90f9
tree_map to tree.map
Misterhoonster May 9, 2025
7efd25d
fix timesteps counter
Misterhoonster May 9, 2025
e921bec
changed reqs to cuda
Misterhoonster May 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,9 @@ dmypy.json
# Cython debug symbols
cython_debug/

# wandb logs
wandb/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
Expand Down
Loading