-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Reduce default stack size from 5Mb to 1Mb #14177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
38865bc
to
6fa466a
Compare
I like that the default pthread stack size now matches the default stack size of the main thread. Seems like a good idea to have them be the same by default. |
ChangeLog.md
Outdated
than on native platforms (since its only used for address taken values) it | ||
seems like 1Mb might still be on the high side here. For reference, llvm | ||
(`wasm-ld`) use 64kb as the default stack size. `DEFAULT_PTHREAD_STACK_SIZE` | ||
was also reduces from 2Mb to 1Mb to match primary stack. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be even better to match LLVM? It seems there is no reason to have differences in the wasm ecosystem here.
That is, maybe we should go to 64K here? Or maybe LLVM should go to 1MB? (I don't think there's any reason for 5MB, I agree with the logic there - I think it was originally because that's the linux stack size.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about matching it to musl by reducing it to 128K?
https://wiki.musl-libc.org/functional-differences-from-glibc.html#Thread-stack-size
http://git.musl-libc.org/cgit/musl/commit/?id=c0058ab465e950c2c3302d2b62e21cc0b494224b.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the linux stack is 8Mb.. so I'm not sure where 5Mb came from.. maybe asm.js needed a lot more of it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree it would make sense to have emscripten and llvm agree on the default.. but I fear that we could break more folks if we go down to 64k. I'm trying to remember now why we chose 1 page (64K) in wasm-ld..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, good question, yeah... Linux is 8. So not sure where 5 came from. Maybe a typo 😉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the default of 64Kb in wasm-ld goes way back to the initial commit: https://reviews.llvm.org/D34851
@sunfishcode would you object to doubling the default stack stack in the linker to 128k to match musl's default for new thread. The goal would be to allow emscripten to use the same default as wasm-ld.
29efc0b
to
5d079e8
Compare
Based on feedback from the mailing list I'd like to move forward with this change to go from 5Mb to 1Mb. We may wish to further reduce this to 64Kb at some point but it seems much more likely to trip up some developers if/when we make that switch. I don't see the cost of possibility updating this limit twice as very significant. Nothing about this change prevents a future reduction. |
There is a cost to doing this twice, in that each time there will be some users hit by it. In particular, in debug builds the shadow stack may be used quite a lot - that's the main thing I worry about. Checking our current errors, if I build a simple program with too little stack, then in
Perhaps it's worth improving that error message before making this change? But even that message is not guaranteed to happen. Maybe we can document the symptoms of a too-small stack in our docs at least? Another option would be to stack checks mode 2 in |
The set of users hit but this first change will by definition not be hit by the second (because they will have set TOTAL_STACK to whatever they need). So the two sets will be distinct and the total number of affected users will be unchanged. So the cost seems pretty small. Indeed it could be informative to us to make this change first.. see how many folks report they were affected and use that as a data point when deciding if we should move forward with further reductions. So I think maybe we could say doing it in two phases might actually have a benefit rather than a cost. As you suggest I will look into the improving the error reporting before moving forward with this. |
Checking today with #include <memory.h>
int main()
{
char str[1024*1024];
memset(str, 0, 1024*1024);
}
I get
Same error message occurs with #include <memory.h>
#include <alloca.h>
int main()
{
char *str = (char*)alloca(1024*1024);
memset(str, 0, 1024*1024);
} so looks like this has been improved at some point? Although with However with the improved message in stack check=2 mode, I would be happy to reduce stack size all the way to 64KB. I think it is good to be very conservative by default, since many applications do not need much stack, and many developers may not know about the stack size setting, leading them to suboptimal results by accident. |
If people need more, one way to improve optics would be to implement stack bump profiling hooks, and then update That way developers would be able to report how much stack they are actually using. That kind of visualization feature already exists there from the old asm.js days, but it no longer functions and always reports zero since we don't get that info from the wasm backend. |
See ChangeLog.md for rationale.
5d079e8
to
d82bfd4
Compare
Previously we defaulted to 1 in this case, but we want to improve the error reporting around stack overflow in preparation for lowering the default stack size. See #14177
Previously we defaulted to 1 in this case, but we want to improve the error reporting around stack overflow in preparation for lowering the default stack size. See #14177 I ran the benchmark test suite under node and could not measure any significant effect:
Previously we defaulted to 1 in this case, but we want to improve the error reporting around stack overflow in preparation for lowering the default stack size. See #14177 I ran the benchmark test suite under node and could not measure any significant effect:
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows STACK_OVERFLOW_CHECK=1 to detect stack overflow conditions better without relying on the STACK_OVERFLOW_CHECK=2 binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the STACK_OVERFLOW_CHECK=2 binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Also, when not using santizers (which currently rely on using the start of memory). This allows `STACK_OVERFLOW_CHECK=1` to detect stack overflow conditions better without relying on the `STACK_OVERFLOW_CHECK=2` binaryen pass. This works because when stack is placed first in memory stack overflow results in SP dropping below zero which is a runtime error. We can then distinguish such runtime errors by looking at the SP value at the time of the crash. This change is part of a sequence of the effort to reduce the default stack size. The hope here is that we will be able to accurately catch overflows in debug builds even without the `STACK_OVERFLOW_CHECK=2` binaryen pass, thus minimizing the impact of reducing the stack size. See #14177
Closing in favor of #18191 |
See ChangeLog.md for rationale.