Skip to content

Make scaffolding Controller more generic #3408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
WeiHaocheng opened this issue Apr 9, 2025 · 0 comments
Open

Make scaffolding Controller more generic #3408

WeiHaocheng opened this issue Apr 9, 2025 · 0 comments
Assignees

Comments

@WeiHaocheng
Copy link
Collaborator

WeiHaocheng commented Apr 9, 2025

We need to do some optimizations to make the Controller more generic, which will help with better modularity.
These changes will not affect the existing API.

  • We hope the Controller such as MajorityVoteController can be constructed by any Generation Controller. For a example, the Generation Controller may be a dynasor cot Controller which will yield for different and uncertainly turns between different sample. We develop "parallel_process_helper" to support it. The code looks like:

`

    controllers = [self.generation_controller.clone() for _ in range(sample_num)]

    tasks_list = [[copy.deepcopy(tasks[0])] for _ in range(sample_num)]

    kwargs_list = [kwargs for _ in range(sample_num)]

    yield from parallel_process_helper(controllers, tasks_list, kwargs_list)`
  • We make the reward method in BestOfNController more generic as we observe that reward methods become various in recently paper and works.

  • We remove most of type check for Task as users may override them.

@WeiHaocheng WeiHaocheng self-assigned this Apr 9, 2025
WeiHaocheng added a commit to WeiHaocheng/TensorRT-LLM that referenced this issue Apr 9, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
WeiHaocheng added a commit to WeiHaocheng/TensorRT-LLM that referenced this issue Apr 10, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
WeiHaocheng added a commit to WeiHaocheng/TensorRT-LLM that referenced this issue Apr 10, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
WeiHaocheng added a commit to WeiHaocheng/TensorRT-LLM that referenced this issue Apr 11, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
WeiHaocheng added a commit to WeiHaocheng/TensorRT-LLM that referenced this issue Apr 12, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
WeiHaocheng added a commit that referenced this issue Apr 12, 2025
Signed-off-by: fredw (generated by with_the_same_user script) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant