Skip to content

WIP: feature(pu): unizero and muzero multitask ddp pipeline #350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 176 commits into from
Closed

WIP: feature(pu): unizero and muzero multitask ddp pipeline #350

wants to merge 176 commits into from

Conversation

puyuan1996
Copy link
Collaborator

@puyuan1996 puyuan1996 commented Apr 25, 2025 •
edited
Loading

  • Please note that this is a temporary PR intended solely for reference.
  • This branch dev-uz-lop-finetune is the original dev version.

puyuan1996 and others added 30 commits July 5, 2024 16:32
@puyuan1996
feature(pu): add UniZero multitask related pipeline
dd2c95c
@dyyoungg
polish(pu): polish unizero_multitask config
8769a5c
@dyyoungg
fix(pu): fix empty_keys_values in init_infer
c342ce1
@dyyoungg
feature(pu): add softmoe head option in unizero_multitask
6eb772a
@dyyoungg
fix(pu): fix unizero reset in muzero_collector
71f55b4
@dyyoungg
polish(pu): polish unizero-multitask config
445fd70
@dyyoungg
fix(pu): fix replay ratio
4954581
@dyyoungg
feature(pu): add moe option of feedforward in transformer backbone
44304bf
@dyyoungg
feature(pu): add value_priority in unizero_multitask
d6be21a
@dyyoungg
polish(pu): polish value_priority in unizero_multitask
fde51cc
@dyyoungg
sync code
b460d2f
@dyyoungg
fix(pu): fix moe in feedforward layer of transformer and polish configs
5117459
@dyyoungg
feature(pu): add mistralai moe in transformer feedforward and head of…
2495d60 
… unizero
@PaParaZz1
polish(pu): polish quantize_state_hash and deepcopy
95886bd
@PaParaZz1
fix(pu): fix np.array dtype bug in buffer
0e49a30
@PaParaZz1
polish(pu): use 0 deepcopy in kv_cache operation in collect/eval phas…
00147f4 
…e of unizero
@PaParaZz1
polish(pu): use custom deepcopy for kv_cache
b40c71b
@PaParaZz1
polish(pu): use value_array rather than value_list in compute_target_…
2cc81be 
…value
@PaParaZz1
polish(pu): optimize compute_target_policy_non_re
bc5332f
@PaParaZz1
polish(pu): optimize kv_caching update()
a6c6a8e
@PaParaZz1
polish(pu): kv_cache_dict no to_cpu
b5dcdcc
@PaParaZz1
polish(pu): optimize custom kv_cache copy
5b0cbd4
@PaParaZz1
polish(pu): kv_cache_dict no to_cpu
0035829
@PaParaZz1
feature(pu): add unizero ddp config
043727b
@dyyoungg
fix(pu): fix unizero ddp
d568008
@dyyoungg
sync code
d349137
@PaParaZz1
polish(pu): use de kv_cacheepcopy only in recur_infer load
3a344aa
@dyyoungg
Merge branch 'dev-efficiency' of https://github.com/opendilab/LightZero…
40053f7 
… into dev-efficiency
@dyyoungg
sync code
61a1139
@dyyoungg
polish(pu): polish suz dmc config
0e545c7
PaParaZz1 and others added 28 commits January 23, 2025 19:03
@PaParaZz1
sync code
557e8f9
feature(pu): add register_token_shared option
8fe1a6d
polish(pu): add moco multigpu support
97988e2
polish(pu): polish moco multigpu option
1860f11
polish(pu): adapt to discrete action space env like atari
04c4a68
polish(pu): adapt to discrete action space env like atari
e9ad078
tmp: polish(pu): add only_use_moco_stats option
a04b896
feature(pu): add unizero_multitask atari concat_task_embed support
d53a402
fix(pu): fix atari uz mt when use concat_task_embed in reanalyze phase
772ba00
tmp
17eea95
tmp:fix(pu): adapt to single-task setting
f8a8957
fix(pu): fix cal_dormant_ratio
3a25c08
feature(pu): add analysis_dormant_ratio_weight_rank option in single-…
ed9d7c1 
…task setting
fix(pu): fix encoder dormant_ratio
b7c9295
fix(pu): fix encoder dormant_ratio log
3bb1e57
feature(pu): add analysis_dormant_ratio_weight_rank option in unizero…
22a2428 
…_mt atari
feature(pu): add lora option in transformer
1c115c9
@PaParaZz1
fix(pu): fix backward when only_use_moco_stats is True
fc3b2e4
@PaParaZz1
feature(pu): add share_head and final_norm_option
83a9f35
@PaParaZz1
tmp
f0c57bb
@PaParaZz1
tmp
8c36a6f
polish(pu): polish uz-mt config
05dd910
polish(pu): polish uz-mt finetune config
fb7520e
polish(pu): polish dmc suz_mt config
8dc52ea
polish(pu): polish dmc mt config
dc58845
fix(pu): fix unizero_moco_8gpu_18task
4c5b5ab
fix(pu): fix atari uz mt moco
8e76cbc
feature(pu): add use_adaptive_scale option in encoder
26e8a6d
@puyuan1996
Copy link
Collaborator Author

We have a polished PR: #353

@puyuan1996 puyuan1996 closed this May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@puyuan1996 @dyyoungg @PaParaZz1 @jiayilee65