Dfcache support to import persistent cache task to dfdaemon in Node by UDS

[gaius-qi]

Description

This feature only needs to be implemented on the Rust Client.

Due to the import protocol between dfcache and dfdaemon, it's possible to import a file from a fixed path on the host node directly into the dfdaemon process, refer to https://github.com/dragonflyoss/api/blob/main/proto/dfdaemon.proto#L174.

// UploadPersistentCacheTaskRequest represents request of UploadPersistentCacheTask.
message UploadPersistentCacheTaskRequest {
  // content_for_calculating_task_id is the content used to calculate the task id.
  // If content_for_calculating_task_id is set, use its value to calculate the task ID.
  // Otherwise, calculate the task ID based on the file content, tag and application by crc32 algorithm`.
  optional string content_for_calculating_task_id = 1;
  // Upload file path of persistent cache task.
  string path = 2;
  // Replica count of the persistent persistent cache task.
  uint64 persistent_replica_count = 3;
  // Tag is used to distinguish different persistent cache tasks.
  optional string tag = 4;
  // Application of the persistent cache task.
  optional string application = 5;
  // Piece length of the persistent cache task, the value needs to be greater than or equal to 4194304(4MiB).
  optional uint64 piece_length = 6;
  // TTL of the persistent cache task.
  google.protobuf.Duration ttl = 7;
  // Download timeout.
  optional google.protobuf.Duration timeout = 8;
}

However, in a Kubernetes environment, Dragonfly's dfdaemon is typically deployed as a DaemonSet, meaning there is one dfdaemon instance running on each Node.

Therefore, when a user inside a Pod utilizes dfcache to import a file, it necessitates mounting the Unix Domain Socket (UDS) of the dfdaemon running on that specific Node into the user's Pod.

Subsequently, dfcache communicates via this mounted UDS to transfer the file's content to dfdaemon. Upon receiving the content, dfdaemon writes the file to its configured storage location.

Link

Client Repo: https://github.com/dragonflyoss/client