stdlib: add support regexp match and replace for strings

[mikedanese]

Would be easy to implement as a builtin.

https://github.com/google/re2

[sparkprime]

Hard in Jsonnet, easy as a builtin, but the trick is to make sure every language that someone might want to implement Jsonnet in (and therefore have to provide an implementation for each of the builtins) has a native regular expression library with exactly the same regex syntax and semantics.

[mikedanese]

PCRE seems to be implemented in many languages. Perhaps it would be better to implement this as a native extension (#108) to the language and not part of the core.

[sparkprime]

Does it support unicode typically?

[nand0p]

+1

[benley]

It appears that PCRE does typically support unicode: http://man7.org/linux/man-pages/man3/pcreunicode.3.html

[sparkprime]

Would be great if the 3 of you could offer some real use cases for this functionality so I can figure out how to prioritize it.

[nand0p]

For our use case, we need to strip all non-alphanumeric chars from a string variable. i am thinking this can be done currently by splitting string to array of chars, checking each char, and then rejoining.... but that seems very ugly.

This may be able to be done more easily with a new function like std.toAlpha(x), but i would think full-on regex capabilities would be a more complete solution.

[sparkprime]

It's not too bad but I can see why you'd rather write it with 0-9A-Za-z type ranges and on one line.

local is_alpha(x) =
    std.setMember(x,"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz");
local to_alpha(str) = std.join("", std.filter(is_alpha, std.stringChars(str)));
to_alpha("He.llo World123")

[emmanuel]

I have a case where I'd like to be able to replace all instances of - with _. That seems even more cumbersome than the filter-to-alphanumeric use-case discussed above. Regex is overkill for my use-case, but either regex-based or tr-style replace functionality would be helpful.

On a nearer-term note, can the stdlib/built-ins be composed to produce a 'replace each instance of character x with an instance of character y' behavior? I'm not coming up with it, though I'm quite new to Jsonnet.

[sbarzowski]

Here's an example implementation of proposed 'replace each instance of character x with an instance of character y'

local replaceChars(str, mapping) =
    std.join("", std.map(function(c) if c in mapping then mapping[c] else c, std.stringChars(str)));
replaceChars("abcd", {"b": "!", "d": "?"})

produces:
"a!c?"

The implementation above also support deleting characters or replacing them with multiple characters:
replaceChars("abcd", {"b": "xx", "d": ""})
produces:
"axxc"

It may be useful generally enough to add it to stdlib. @sparkprime what do you think?

[emmanuel]

@sbarzowski thank you, that is fantastic. Not only a clean interface to accomplish what I'm looking to get done, but also a good bit of insight about how to approach programming jsonnet. I'd be in favor of adding this to stdlib, but I'm not on the hook for maintenance, so perhaps merely adding to the documentation would be sufficient to help future seekers like myself.

Whatever the decision about adding to stdlib or docs, thank you for the help @sbarzowski.

[sparkprime]

tr-like functionality is definitely a good candidate for stdlib.

[sparkprime]

Since this has come up again - do we have compatible implementations of PCRE in Go and C++ that will work with unicode?

[sbarzowski]

Well, there is this thing: https://github.com/glenn-brown/golang-pkg-pcre. This is an interface to libpcre. It seems to hardcode assumptions about where libpcre is installed, though... I couldn't find anything else. Probably using libpcre directly with cgo would be a better option.