[NSXServiceAccount] Watch mgmt-proxy service and reconcile all NSXSA

[Atish-iaf]

Reconcile all existing NSXSA incase of following events of K8s service corresponding to mgmt-proxy.

• new service is created.
• existing service is deleted.
• existing service is updated with LB VIP.

While reconciling realized NSXSA, update proxyEndpoints if proxyEndpoints have changed.

Steps for testing -

1. Build nsx-operator image and save it in a tar file.
   Run make photon in nsx-operator repo.
   docker image save github.com/vmware-tanzu/nsx-operator:latest -o my-nsx-operator.tar
2. Get WCP testbed
3. Copy nsx-operator image my-nsx-operator.tar file to SVCP VMs using scp
4. Inside each SVCP VMs, run ctr -n=k8s.io images import <path to my-nsx-operator.tar> to load nsx-operator image.
5. Update nsx-ncp deployment.
   kubectl set image deploy/nsx-ncp nsx-operator=github.com/vmware-tanzu/nsx-operator:latest -n vmware-system-nsx
6. Create a guest cluster, nsxsa gets created with empty proxyEndpoints.
7. register and install nsx-mgmt-proxy supervisor service.
8. kubectl describe nsxsa, proxy endpoints get updated.
9. Change proxy LB IP from VC UI by filling data-value loadBalancerIP: <someIP>, check in VC UI, it gets updated, then kubectl describe nsxsa and see proxyEndpoints get updated with new LB IP.

[edwardbadboy]

Thanks for the PR. Let me review it.

[Atish-iaf]

Hi @edwardbadboy @gran-vmv @liu4480
Please help to review this PR.
Thanks

[gran-vmv]

Overall LGTM. A small comment.

[edwardbadboy]

Looks good. Only minor comments.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 27.71084% with 60 lines in your changes missing coverage. Please review.

Project coverage is 74.28%. Comparing base (2bd7c0e) to head (55c2f00).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
.../nsxserviceaccount/nsxserviceaccount_controller.go	23.28%	52 Missing and 4 partials ⚠️
pkg/nsx/services/nsxserviceaccount/cluster.go	60.00%	3 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (27.71%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1019      +/-   ##
==========================================
- Coverage   74.52%   74.28%   -0.24%     
==========================================
  Files         118      118              
  Lines       16330    16412      +82     
==========================================
+ Hits        12170    12192      +22     
- Misses       3391     3446      +55     
- Partials      769      774       +5     

Flag	Coverage Δ
unit-tests	74.28% <27.71%> (-0.24%)	⬇️

Files with missing lines	Coverage Δ
pkg/nsx/services/nsxserviceaccount/cluster.go	79.16% <60.00%> (-0.46%)	⬇️
.../nsxserviceaccount/nsxserviceaccount_controller.go	68.39% <23.28%> (-20.91%)	⬇️

[Atish-iaf]

/e2e

[Atish-iaf]

/e2e

[liu4480]

LGTM

[edwardbadboy]

Hello @Atish-iaf ,

Can you test this PR and describe how you test it in the PR description? Thanks!

[Atish-iaf]

Hello @Atish-iaf ,

Can you test this PR and describe how you test it in the PR description? Thanks!

Hi @edwardbadboy

1. no proxy->nsxsa realized with empty proxyEndpoints->then proxy created->nsxsa got updated with proxy endpoints.
2. change proxy LB IP via VC UI by filling data-values->realized nsxsa proxyEndpoints got updated with new proxy LB IP.
3. delete proxy, nsxsa doesn't get updated, it still has old proxyEndpoints. ProxyEndpoints should be empty now ?
   Again create proxy, nsxsa get updated and replaces old proxyEndpoints.

Update
3. delete proxy, nsxsa proxyEndpoints become empty(it takes a little longer time to get updated after proxy delete event), Again create proxy, nsxsa gets updated with proxyEndpoints.

[liu4480]

Hello @Atish-iaf ,
Can you test this PR and describe how you test it in the PR description? Thanks!

Hi @edwardbadboy

1. no proxy->nsxsa realized with empty proxyEndpoints->then proxy created->nsxsa got updated with proxy endpoints.

2. change proxy LB IP via VC UI by filling data-values->realized nsxsa proxyEndpoints got updated with new proxy LB IP.

3. delete proxy, nsxsa doesn't get updated, it still has old proxyEndpoints. ProxyEndpoints should be empty now ?
   Again create proxy, nsxsa get updated and replaces old proxyEndpoints.

did you test this patch together with antrea-interworking change

[Atish-iaf]

Hello @Atish-iaf ,
Can you test this PR and describe how you test it in the PR description? Thanks!

Hi @edwardbadboy

1. no proxy->nsxsa realized with empty proxyEndpoints->then proxy created->nsxsa got updated with proxy endpoints.

2. change proxy LB IP via VC UI by filling data-values->realized nsxsa proxyEndpoints got updated with new proxy LB IP.

3. delete proxy, nsxsa doesn't get updated, it still has old proxyEndpoints. ProxyEndpoints should be empty now ?
   Again create proxy, nsxsa get updated and replaces old proxyEndpoints.

did you test this patch together with antrea-interworking change

Yes, register Pod detects the change but fails to update bootstrap-config map and register pod restarts.
I have updated it on interworking changes PR.

[edwardbadboy]

Thanks for the test I think this PR should be good to merge.