-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Ensure ThreadPool is closed in setup_helpers #3548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The ParallelCompile setup helper using a ThreadPool to enable its parallelism. It does not properly close the pool when it is done with it. This can lead to a "Exception ignored in: <function Pool.__del__..." message with traceback being printed at shutdown. Use pool.terminate() instead of context manager for Python 2.7 compatibility
@henryiii I just updated this PR because CI failed against Python 2.7. Turns out that the pool does not support being used as a context manager in that version. I've changed it to use a finally block that calls |
I've not seen the "ICC latest" error before. Seems unrelated to this PR. Maybe coincidentally the compiler was just updated? |
I'd either add a note or maybe even use an if statement - we are very close to dropping Python 2 support (Jan 1) and long term I'd like the nicer syntax. There should be something making it easy to remember to clean up. "Python 2", "sys.verison_info" are a couple of things I'd normally grep when cleaning up. |
@henryiii If it's only two weeks or so until Python 2 support is dropped, I'd lean towards reverting this back to the first version I had and having you not merge it until then. |
I'd like to get as many fixes as possible in before dropping Py 2. So how about we merge this one, then you could make a PR to the nicer version that we merge in 2 weeks? :) |
@henryiii Sounds good. I've added a comment that mentions "Python 2". There are already couple others in this same file (related to Distutils using old-style class in Python 2) that can maybe be cleaned up at the same time as this one. This should be ready to merge. |
The note is fine. :) I'll be expecting the old-style class stuff, but won't be likely to remember this context manager simplification without the note. |
Please do not delete the changelog entry portion of the template. It fills our changelog. (fixed manually). |
Thanks! |
The ParallelCompile setup helper using a ThreadPool to enable its parallelism. It does not properly close the pool when it is done with it.
This can lead to a
Exception ignored in: <function Pool.__del__...
message with traceback being printed at shutdown.Suggested changelog entry: