Getting hit by rate limits against maven central, even though very conservative settings

[janderssonse]

How are you running Renovate?

A Mend.io-hosted app

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

No response

Please tell us more about your question or problem

I started getting ratelimit failures against maven central, even though i have very conservative settings.
Am I doing something wrong in rate limiting? What is the expected max values, any hints would be great. Thanks.

Part of the config, with main settings and rate limiting, and the whole can be found here: https://github.com/diggsweden/cose-lib/blob/main/renovate.json

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "osvVulnerabilityAlerts": true,
  "dependencyDashboardOSVVulnerabilitySummary": "all",
  "extends": [
    "config:best-practices",
    "workarounds:all",
    "security:openssf-scorecard",
    ":configMigration",
    ":dependencyDashboard",
    ":gitSignOff",
    ":maintainLockFilesWeekly",
    ":automergeMinor",
    ":semanticCommits",
    "npm:unpublishSafe",
    ":rebaseStalePrs",
    ":semanticCommitTypeAll(chore)",
    "mergeConfidence:all-badges"
  ],
  "commitMessageLowerCase": "auto",
  "minimumReleaseAge": "4 days",
  "labels": [
    "dependencies"
  ],
  "vulnerabilityAlerts": {
    "labels": [
      "security",
      "dependencies"
    ]
  },
  "hostRules": [
    {
      "hostType": "maven",
      "matchHost": "https://repo.maven.apache.org/maven2/",
      "concurrentRequestLimit": 1,
      "maxRequestsPerSecond": 2
    }
  ],

Logs (if relevant)

Logs

"response": {
      "statusCode": 429,
      "statusMessage": "Your IP: 35.169.49.18 has hit the rate limit with Maven Central. Too many requests. Find out more in https://www.sonatype.com/blog/maven-central-and-the-tragedy-of-the-commons",
      "body": "Your IP: 35.169.49.18 has hit the rate limit with Maven Central. Too many requests. Find out how to address this at https://www.sonatype.com/blog/maven-central-and-the-tragedy-of-the-commons",

[rarkins]

Is that IP address exclusively for your Renovate account, or belongs to your company and may be shared by other systems (developers, CI/CD, etc)? I don't recall seeing Maven rate limiting before - Mend's massive Renovate app doesn't get limited, and nor do I recall any other Renovate user.

[sergej-koscejev]

I'm hitting this as well with several (all?) open-source GitHub repositories that I am maintaining, using Mend's Renovate instance (not self-hosted). I believe the IPs that are rate-limited belong to Mend (e.g. 35.169.49.18, resolving to ec2-35-169-49-18.compute-1.amazonaws.com.).

[pedromfmachado]

I'm also facing the same problem. Introduction of rate limiting on Maven Central was announced a few months ago, although it worked normally until a few days back. Maybe there was a change in Mend's IPs that triggered the rate limiting or something similar?

[rarkins]

My mistake, I misread the above. That's definitely new, and we need to take an urgent look

[rarkins]

I have reached out to the Maven Central team at Sonatype to ask for a temporary reprieve/lifting of rate limits for our fixed IPs. Although the Renovate service undoubtedly generates as lot of requests, it also caches them for all users so they'd see a 100-1000x increase if users were forced to churn and effectively "DDoS" instead.

[janderssonse]

Thanks, and the problem seems to be gone now.

[rarkins]

Yes, they've been very quick to help out and lift the ban for now, to give time for some longer term optimizations.