math/rand: seed global generator randomly

[rsc]

As a step toward a larger overhaul of math/rand (for example, #21835 or #26263), @robpike and I propose that math/rand seed the global generator randomly and deprecating Seed, rather than behaving as if Seed(1) had been called. A more aggressive step (which we are not proposing at the moment) would be to make the global Seed function a no-op.

Seeding the global generator randomly at startup eliminates the possibility of programs that don't call Seed depending on the algorithm behind the global functions. If the package randomly seeded the global generator, that would let us change the global generator's source algorithm as an implementation detail: nothing could tell that the algorithm was different.

Auto-seeding would also fix any programs that are missing calls to Seed and think they are getting non-deterministic values but are not. (This is a common mistake.)

Auto-seeding would break programs that have no calls to Seed and expect a deterministic sequence. The most common time this might happen would be in golden tests.

There are lots of other possible changes that could be made to math/rand, and we are intentionally leaving them all to the side for the purposes of this proposal. This proposal is only about auto-seeding, precisely because it targets the most common backwards compatibility concern that comes up in any other changes to math/rand. Addressing that one concern in isolation greatly simplifies any other changes we might make later.

I hope that this proposal discussion can focus on the impact of making this change, specifically programs that would break that we need to be concerned about. If it turns out that we can't make this change, then maybe we will propose a math/rand/v2 instead. But we'd like to understand whether it can be done in the original math/rand first.

Again please limit comments to the impact of auto-seeding and not on other possible changes to math/rand. Thank you!

[dominikh]

While I am sympathetic to the idea, I believe this would be a violation of the backwards compatibility guarantee in a very direct manner. The package documentation clearly states that

Top-level functions, such as Float64 and Int, use a default shared Source that produces a deterministic sequence of values each time a program is run. Use the Seed function to initialize the default Source if different behavior is required for each run.

That is, the fact that the global RNG is deterministic isn't an implicit detail that people may be relying on; it's documented fact.

I don't believe that the downsides of accidentally using an unseeded RNG outweigh the cost of breaking compatibility, especially because this can't be considered a bug fix nor a security fix.

[zhangyunhao116]

I guess the final implementation may be similar to https://github.com/zhangyunhao116/fastrand ? The internal implementation of this library has been widely used in my company, and we have proposed similar ideas internally, but we have not implemented it yet(replace math/rand with this library).

According to our experience, this change will not bring any negative effects in most cases(maybe ~99%?). In rare cases, this may break some services or tests(e.g., call Seed(1) and then generate a random slice, use it as a test case or send it to other places).

But I think it's not worthing to break the compatibility for golang/go, as @dominikh said before. I think math/rand/v2 or math/fastrand is better.

[robpike]

It's a bit of a stretch, but it can be argued that this is in fact a security issue. Programs that don't seed their random numbers will behave predictably, which can sometimes be a risk if the numbers are used to generate unique, supposedly unguessable results.

It was a design error, copied from Unix, to have rand be reproducible like this. It should be changed. The question is whether it can be changed without a version bump on the library.

[atdiar]

I would add this as a math/rand2 variant. I personally have used the behaviour in the past to generate stable id references for different runs of a same program.

When I don't need that behavior, I usually go for devurandom.

Unless I'm mistaken somewhere?

Edit: ha but I usually provide a seed. Please ignore sorry.

[kortschak]

Programs that don't seed their random numbers will behave predictably, which can sometimes be a risk if the numbers are used to generate unique, supposedly unguessable results.

I'd say that if people are using math/rand [out of the box] for security-sensitive uses there are other problems.

[dallbee]

Go1 compatibility promise aside (and I do think this would be a compatibility break),

Seeding math/rand doesn't make it any less predictable. Instead, it just becomes easier to use as a footgun in security crtical contexts.

Today, it's common for junior devs to ask me why math/rand produces the same values every time they use it. This is a great learning point - they become aware of the difference between a PRNG and a CSPRNG.

Autoseeding would make it easy to think math/rand is suitable for all uses.

[seebs]

This would definitely break some existing programs, even if those programs are bad. I think, though, dallbee's point is the one I'm most concerned by. I am worried about anything that makes it easier for people to accidentally get apparently-not-horrible behavior from the default rand, without actually getting good enough behavior that it's reasonably safe. I'd rather have the behavior be sufficiently clearly bad that people are encouraged to change it.

[srowles]

It's just an anecdote, but I tinker with procedural content generation toys from time to time, these are often UI based or 1-shot programs that provide an output. I often use the global random in those pieces of code and I initialise the seed based on some input.

Default seeding the global generator would not break my code, but making the global Seed() call a no-op would definitely break the code today I have today.

I also agree with the other comments that this should not be changed as it breaks documented behaviour (https://pkg.go.dev/math/rand#Seed)

I don't see that it's a security issue because nobody should be using math/rand for security related random and in fact the doc's for math/rand even explicitly cover this:

"For random numbers suitable for security-sensitive work, see the crypto/rand package."

[rsc]

Re security:

There's no positive security argument here. Code that needs true randomness should use crypto/rand.

But there's also no negative security argument here. People accidentally reaching for math/rand when they mean crypto/rand does not seem like justification for making math/rand as bad as possible.

[rsc]

Re general breakage and backwards compatibility:

@dominikh pointed out

Top-level functions, such as Float64 and Int, use a default shared Source that produces a deterministic sequence of values each time a program is run. Use the Seed function to initialize the default Source if different behavior is required for each run.

He reads that as a promise, but I read it as a warning (don't forget to call Seed).

Either way, using the global source and expecting a deterministic sequence is incredibly fragile. All it takes is one new import that happens to call Seed or Int (or any of the other getters) and your code breaks. Suppose instead of auto-seeding we made a change where some commonly used package (say, strings) called rand.Int a few times at init time. Is changing the number of times math/rand is called by a package's init function a breaking change for that package? That can't be right.

Auto-seeding is like you have a package linked into the binary that reads a non-deterministic amount of randomness from math/rand at init time.

To be clear, at the moment I am assuming that if you call global Seed(1) then determinism is restored. Programs that want determinism can and probably should ask for it explicitly.

[rsc]

Re specific breakage:

@zhangyunhao116 reported their experience:

According to our experience, this change will not bring any negative effects in most cases(maybe ~99%?). In rare cases, this may break some services or tests(e.g., call Seed(1) and then generate a random slice, use it as a test case or send it to other places).

In contrast, @seebs said:

This would definitely break some existing programs, even if those programs are bad.

I expect it would break some (again, latently fragile) golden tests. Those are annoying but easy to fix. What about non-tests? Can anyone think of a case where a non-test would break? That would concern me more.

[dr2chase]

Regarding security and warning new programmers of foot-guns, if that is a problem, the safer solution is to make the default RNG cryptographically secure, and when it shows up in profiles, then we have the discussion about whether this application needs the secure rand or not. Premature optimization, etc.

I favor this change. Current behavior is a foot-gun, adds to the learning curve for new Go programmers, adds one line of "don't forget" code to good programs, and it compromises security, where one of the rules (at least, my rules) for security is "you never know what's going to end up exposed to strangers on the internet".

I am less swayed by the need for compatibility here because the winning example is "tests". Those will break, they'll be easy to fix, it's unlikely that production code depends on this, and my actually-educated guess is that any such uses are unintentional and this change will fix (through removed security holes) more code than it breaks.

I am not sure what to do with Seed. Short-term, we'll need "Seed(1)" to fix any tests that this breaks, but then what's the plan for changing the RNG down the road, if that becomes necessary, or or if people are swayed by my
Great Idea in the first paragraph? Special case Seed(1) to select the old RNG?

[srowles]

Regarding security and warning new programmers of foot-guns, if that is a problem, the safer solution is to make the default RNG cryptographically secure

Pseudo random has use-cases beyond security use cases, using it to produce "random" but deterministic output. This is useful for a wide range of scenarios.

Pseudo random shouldn't be removed from the core library just because some use cases are better with cryptographic random, not everything needs or wants real random behaviour or the overhead of secure random when pseudo is good enough.