Skip to content

Commit fac17bb

Browse files
authored
typo(vllm): gpu_memory_utilisation typo (#553)
* Update vllm_model.py * Update use-vllm-as-backend.mdx * Update quicktour.mdx
1 parent 78b68ab commit fac17bb

File tree

3 files changed

+4
-4
lines changed

3 files changed

+4
-4
lines changed

docs/source/quicktour.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ accelerate).
148148
### VLLM
149149

150150
- **pretrained** (str): HuggingFace Hub model ID name or the path to a pre-trained model to load.
151-
- **gpu_memory_utilisation** (float): The fraction of GPU memory to use.
151+
- **gpu_memory_utilization** (float): The fraction of GPU memory to use.
152152
- **batch_size** (int): The batch size for model training.
153153
- **revision** (str): The revision of the model.
154154
- **dtype** (str, None): The data type to use for the model.

docs/source/use-vllm-as-backend.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,4 +57,4 @@ model: # Model specific parameters
5757
5858
> [!WARNING]
5959
> In the case of OOM issues, you might need to reduce the context size of the
60-
> model as well as reduce the `gpu_memory_utilisation` parameter.
60+
> model as well as reduce the `gpu_memory_utilization` parameter.

src/lighteval/models/vllm/vllm_model.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@
7676
@dataclass
7777
class VLLMModelConfig:
7878
pretrained: str
79-
gpu_memory_utilisation: float = 0.9 # lower this if you are running out of memory
79+
gpu_memory_utilization: float = 0.9 # lower this if you are running out of memory
8080
revision: str = "main" # revision of the model
8181
dtype: str | None = None
8282
tensor_parallel_size: int = 1 # how many GPUs to use for tensor parallelism
@@ -174,7 +174,7 @@ def _create_auto_model(self, config: VLLMModelConfig, env_config: EnvConfig) ->
174174
"""
175175
self.model_args = {
176176
"model": config.pretrained,
177-
"gpu_memory_utilization": float(config.gpu_memory_utilisation),
177+
"gpu_memory_utilization": float(config.gpu_memory_utilization),
178178
"revision": config.revision + (f"/{config.subfolder}" if config.subfolder is not None else ""),
179179
"dtype": config.dtype,
180180
"trust_remote_code": config.trust_remote_code,

0 commit comments

Comments
 (0)