feat: use cudaMalloc to allocate kvCache

[chuangz0]

use cudaMalloc to allocate kvCache , so kvCache pool can be register by transfer enigne

[chuangz0]

/bot run --add-multi-gpu-test

[tensorrt-cicd]

PR_Github #1218 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #1218 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #913 completed with status: 'FAILURE'

[chuangz0]

/bot run --add-multi-gpu-test --disable-fail-fast

[tensorrt-cicd]

PR_Github #1226 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #1226 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #919 completed with status: 'FAILURE'

[chuangz0]

/bot run --add-multi-gpu-test --disable-fail-fast

[tensorrt-cicd]

PR_Github #1247 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #1247 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #939 completed with status: 'FAILURE'

[chuangz0]

/bot run --add-multi-gpu-test --disable-fail-fast

[tensorrt-cicd]

PR_Github #1290 [ run ] triggered by Bot

[tensorrt-cicd]

PR_Github #1290 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #972 completed with status: 'SUCCESS'

[Shixiaowei02]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #1389 [ reuse-pipeline ] triggered by Bot

[tensorrt-cicd]

PR_Github #1389 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #1290 for commit 70f5745

[schetlur-nv]

@chuangz0 this change seems to be pretty general, and is not limited to disaggregated serving. Did we assess that using cudaMalloc does not affect other use-cases?