Cosi v1alpha2 changes

[BlaineEXE]

• One-line PR description: begin v1alpha2 doc - initial work is fixing old errors without adding large features

• Issue link: Object Storage Support (COSI) #1979

• Other comments:

Update the COSI KEP's base design to v1alpha2. The primary purpose of
this work is to reshape the doc and spec to fix inconsistencies and
vestigial items, and fix the most glaring issues in the v1alpha1 spec.

COSI maintainers are planning follow-up items to add larger individual
features after this.

Notably, bucketInfo.json has been changed to individual secret fields
with COSI_<KEY>: <VALUE> format, as the JSON blob was flagged as a
problem by several v1alpha1 users.

Additionally, rework the existing APIs/spec to give driver sidecars fewer responsibilities and take on more coordination responsibility in the main COSI controller. This mirrors the implementation of volume snapshotter uses and should help keep version mismatch issues between sidecar/controller less frequent. It also means the sidecar -- and thus vendor drivers -- require fewer RBAC permissions.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[xing-yang]

/remove-lifecycle stale

[BlaineEXE]

It's probably better to define high-level behavior expectations here, rather than pigeonhole ourselves into a specific implementation solution.

[BlaineEXE]

This section contains sections that describe design details that we have discarded, including many changes between v1alpha1 and v1alpha2. I have taken care to try to make this a good effective overview and discussion of any complex points.

[BlaineEXE]

I listened to @saad-ali 's response in the last sig-storage meeting to whether this is needed. One of the important things I heard was that the intent is that drivers can "advertise" what they support.

Reading more into Saad's reply, I can see how this field might be important for portability in some ways. Importantly, if I were to copy a BucketClaim spec to a new/different K8s cluster with different COSI driver, the presence of Protocols here should help me feel certain that COSI will return an error if the ported copy is incompatible with the new k8s cluster.

Something that wasn't present in the v1alpha1 spec (or implementation) was a way for drivers to "advertise" anything. Drivers were expected to error out if they couldn't support all protocols, but I don't consider this "advertising" things. And certainly, any driver could decide to ignore the list when provisioning if they wanted to.

Thus, for the v1alpha2 spec, I added a gRPC return value for DriverGetInfo for drivers to advertise their supported protocols. I also added the note here that we expect COSI itself (I think the Sidecar) to validate that what the user is requesting in Parameters is advertised as supported by the driver.

If my analysis and understanding of Saad's feedback is correct, I think this is good to keep (along with the ability for to drivers advertise support).

---

As an alternative to the input check, we originally planned v1alpha2 to only have the BucketClaim.status.supportedProtocols list, which would inform users what protocols the Bucket supports, but only after the BucketClaim/Bucket are done being provisioned. It wouldn't guarantee an error in ported BucketClaim provisioning if the 'new' cluster didn't support the same protocols.

This alternative would easier to implement because it's less complex. However, it wouldn't allow them to state their intentions before provisioning happens.

CC: @xing-yang @shanduur

[shanduur]

After reading the comment and the doc, I think that spec field is the correct approach. It allows better validation and failing early, if desired protocols does not match protocols advertised by driver.

[BlaineEXE]

Because we are trying to keep the Sidecar responsibilities (and RBAC) as small as possible (following the design of Volume Snapshotter), we needed significant changes to how BucketAccesses are provisioned.

TL;DR: the controller gets info from BucketAccessClass and BucketClaim and copies it to the BucketAccess.status, after which the sidecar can do its thing. Sidecar no longer does Class/Claim inspection.

[BlaineEXE]

If Protocols is important for BucketClaims/Buckets it stands to reason (to me) that Protocol is important for BucketAccess also.

[BlaineEXE]

This section has a quick overview of important changes from v1alpha1 to v1alpha2.

[shanduur]

Few nitpicks, but honestly this is great! Good job.

[shanduur]

Do we need to specify what actually is being protected? Why not just use objectstorage.k8s.io/protection?

[BlaineEXE]

I suggested reusing the same finalizer name a couple weeks ago. It seemed like Xing was against it, or perhaps confused about what I was trying to express. I would prefer the approach you are suggesting, so I will make the change. This detail isn't the most important, so it can be reverted if there is still opposition

[moonlight16]

In this previous version, I liked the idea of COSI driver being able to grant or deny access requests. This concept seemed to have disappeared in v1alpha2. If some rules were defined in the vendor-specific section of BucketAccessClass, it could allow COSI drivers to make access decisions—granting or denying requests based on that policy. This would give vendors flexibility to enforce rules like user-based permissions, time-based access, or namespace-level constraints. Or to allow users to make specific requests in the BucketAccess resource, such as requesting read-only access or specifying expiration times—inputs that could be evaluated by the COSI driver when deciding whether to grant or deny access.

[BlaineEXE]

The details here haven't been removed from the v1alpha2 design. They have been clarified and expounded upon a bit, and the organization of the doc has changed.

We did remove a vague set of details around COSI supporting IAM, which wasn't fully baked, but I don't think that's what you're talking about.

To add clarity, both DriverGrantBucketAccess() and DriverRevokeBucketAccess() are the gRPC calls used by COSI to request the driver grant/revoke access. Those were present in v1alhpa1 and still present here.

[moonlight16]

So your saying COSI drivers will still have the ability to grant or deny access requests? I didn't really see that language anywhere like it was in v1alpha1.

[BlaineEXE]

Grant and "Revoke" are core operations the drivers must support in order to allow BucketAccesses to be provisioned and deleted.

"Deny" implies access control to me. Access control wasn't part of the v1alpha1 spec, but we do have access control plans for the v1alpha2 spec in the near future: kubernetes-retired/container-object-storage-interface-api#107

[moonlight16]

i'm finished reviewing.

[shanduur]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BlaineEXE, moonlight16, shanduur
Once this PR has been reviewed and has the lgtm label, please assign saad-ali for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-storage/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment