Be able to calculate sizes of PutMetricData requests client-side

[ianvernon]

Describe the feature

Right now, there is no clear way to know whether a request that is generated client-side will exceed 40 KiB request size limit for PutMetricData requests. Exposing this to clients will allow them to craft requests to PutMetricData that maximize the amount of datapoints per-request without exceeding the per-request limit.

Use Case

PutMetricData has many restrictions on its use, specifically:

• No more than 150 requests per second per-region for a given account
• No more than 150 datapoints per metric datum in a given request
• No more than 20 metric datum per request
• Requests must not exceed 40 KiB in size

The rate limit motivates clients to pack as many values as possible (3,000) per request, but to do so in a way that does not exceed the 40 kilobyte / request limit. It would be nice if there were a way to expose what the size of a request is client-side so that we don't hit errors like:

RequestEntityTooLarge: Request size 45305 exceeded 40960 bytes
status code: 413,

Proposed Solution

A function should be exposed for a cloudwatch.PutMetricDataInput which calculates the size of the request so that clients can determine whether or not they need to split up a single cloudwatch.PutMetricDataInput into multiple, smaller cloudwatch.PutMetricDataInputs. Additionally, it would be useful to have the size calculated with or without gzip compression enabled.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

v1.44.33

Environment details (Version of Go (go version)? OS name and version, etc.)

go version go1.17.6 linux/amd64

[vudh1]

Hi @ianvernon thanks for bringing this up. We have discussed and this feature would be a good addition. However, the team has no bandwidth to work on this feature at the moment. We can work on a timeline on feature requests later on, but you are welcome to help us implement the feature.

[ianvernon]

Until there is an official solution, I found that doing something like the following has been helpful for calculating request size:

func putMetricDataRequestLength(request) (int, error) {
    awsRequest, _ := cloudWatchAPI.PutMetricDataRequest(request)
    if err := awsRequest.Build(); err != nil {
      return 0, err
    }

    reqBytes, err := ioutil.ReadAll(awsRequest.Body)
    if err != nil {
        return 0, errr
    }
    return len(reqBytes), nil
}

I can't be sure that this gives an accurate request length, but I've stopped getting RequestEntityTooLarge errors by checking request sizes with this functionality. I've also used gzip compression to help reduce request size as well.

[github-actions]

We have noticed this issue has not received attention in 1 year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.

[ianvernon]

I still think this would be useful.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.