Use parts of matchStrings in autoReplaceStringTemplate

[ThomasSteinbach]

What would you like help with?

I would like help with my configuration

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab; Renovate '37-full'

Please tell us more about your question or problem

I have created a custom manager:

{
    "customManagers": [
        {
            "customType": "regex",
            "fileMatch": [
                "^.*\\.ya?ml$"
            ],
            "matchStrings": [
                "originalImage=(?<depName>.*?)\\s.*: *(?<currentValue>[vV]?\\d+[\\.\\-\\w+]*)(@(?<currentDigest>sha256:[a-f0-9]+))?"
            ],
            "datasourceTemplate": "docker"
        }
    ]
}

With this manager I can define the actual dependency in a comment before the line with the value/version. Here an example:

# originalImage=the-actual/image
version: 1.2.3

# originalImage=another/image
image: my-virtual/image:1.2.3

In the previous example Renovate successfully detects:

• the-actual/image:1.2.3
• another/image:1.2.3

To enable Renovate to pin digests, I have read that I have to tell Renovate how to replace the original match. This is by defining autoReplaceStringTemplate.

But how can I re-use required parts of the "matchStrings" in my "autoReplaceStringTemplate"? This is required, because I don't know the whole match aforehand.

I tried to use a custom named capture group named "customPrelude", which didn't work:

{
    "customManagers": [
        {
            "customType": "regex",
            "fileMatch": [
                "^.*\\.ya?ml$"
            ],
            "matchStrings": [
                "originalImage=(?<depName>.*?)\\s(?<customPrelude>.*: *)(?<currentValue>[vV]?\\d+[\\.\\-\\w+]*)(@(?<currentDigest>sha256:[a-f0-9]+))?"
            ],
            "autoReplaceStringTemplate": "originalImage={{{depName}}}\n{{{customPrelude}}}:{{{newValue}}}{{#if newDigest}}@{{{newDigest}}}{{/if}}",
            "datasourceTemplate": "docker"
        }
    ]
}

So is this even possible?
If not, is there a different approach to enable Renovate to pin digests?

Logs (if relevant)

No response

[rarkins]

Try using both depName and packageName

[ThomasSteinbach]

[Update 2024-05-17 15:50] This post is obsolete as with the final solution posted below.

---

Eureka! 🥳

I've found a working solution.

Knowing the previously described behavior of Renovate, I've created two custom managers. They are both equal, despite the blank between the last colon before the version and the version itself.
👉 https://github.com/ThomasSteinbach/renovate-29068-example/blob/working-state/renovate.json

I also made the colon and blank (: ) and not part of the depName. Knowing that those definetly exist (in the first manager), I could separately add them to the autoReplaceStringTemplate. Thus I have no trouble, that Renovate will trim the blank away.

Of course it would be better, if

• Renovate would use the same depName in the autoReplaceStringTemplate as discovered by the matchStrings regex.
• Renovate would allow custom handlebars in matchStrings that one could use in autoReplaceStringTemplate.

Nevertheless thank you @rarkins for looking into this. Your quick answers encouraged me to investigate further.

[rarkins]

Thanks for sharing your solution!

Allowing non-trimmed depNames is a hard decision to make. There's probably a few people who have regex which has mistakenly extracted a depName with unintended additional whitespace and changing this could break them.

On the second matter, I don't fully understand your suggestion but if you could provide an example or perhaps even code for us to look at, we can definitely consider it.

[ThomasSteinbach]

[Update 2024-05-17 15:50] This post is obsolete as with the final solution posted below.

---

I am no JS/TS coder, so I try to describe my second suggestion in more detail:

The custom manager from my example can handle Docker dependencies, where the Docker registry / repository (packageName) is defined as meta information (comment) on one line and the Docker tag (currentValue) on the nextline, with arbitrary text in between. Here again two examples:

# originalImage=mydockerregistry.net/myimage
image: 1234567890.dkr.ecr.eu-central-1.amazonaws.com/myimage:1.2.3-20200101@sha256:abcdef

image: 1234567890.dkr.ecr.eu-central-1.amazonaws.com/myimage
# originalImage=mydockerregistry.net/myimage
version: 1.2.3-20200101@sha256:abcdef

In both examples image: and version: are randomly chosen and could be completely different.

That means, I don't know what is between packageName and currentVersion. But I need to know this string, to re-use it in my autoReplaceStringTemplate.

With your suggestion from above (#29068 (comment)) I used depName for this unknown part. However that is a misuse of depName because this string could be totally garbage. Or at least from the second example, depName would just be version: , which is not the intended use of depName.

For this it would be great, If you could define custom named capture groups in matchStrings, you could then re-use in your autoReplaceStringTemplate.

Here a super simple example:

"matchStrings": [
  "originalImage=(?<depName>.*?)\\s(?<customPrelude>.*:\\s*)(?<currentValue>\\d+\\.\\d\\.\\d)(@(?<currentDigest>sha256:[a-f0-9]+))?"
  ],
"autoReplaceStringTemplate": "originalImage={{{depName}}}\n{{{customPrelude}}}:{{{newValue}}}{{#if newDigest}}@{{{newDigest}}}{{/if}}",

In this example customPrelude could be a custom named capture group introduced in matchStrings, which I can easily use in autoReplaceStringTemplate.

[ThomasSteinbach]

I've found a way better solution than before.

With this solution I ship around the "blank problem" and also don't need any custom named capture groupd.

For this solution I needed to understand, that Renovate only replaces the last match, when using recursive match strategy.
So in the last match I can focus on detecting the version and possible digest.
I didn't need to bother with whats between the version and the package name.

I have update my example with the latest tag: https://github.com/ThomasSteinbach/renovate-29068-example/blob/best-solution/renovate.json

The essence is, that in the last regex I only match the version. Then there is no need for a complex autoReplaceStringTemplate.

"matchStrings": [
  "originalImage=(?<depName>.*?)\\n[^\\n]+",
  "(?<currentValue>\\d+\\.\\d+\\.\\d+[\\w\\-\\.]*)(@(?<currentDigest>sha256:[a-f0-9]+))?"
],
"autoReplaceStringTemplate": "{{{newValue}}}{{#if newDigest}}@{{{newDigest}}}{{/if}}",