[Roadmap] XGBoost 1.0.0 Roadmap

[CodingCat]

@dmlc/xgboost-committer please add your items here by editing this post. Let's ensure that

• each item has to be associated with a ticket

• major design/refactoring are associated with a RFC before committing the code

• blocking issue must be marked as blocking

• breaking change must be marked as breaking

for other contributors who have no permission to edit the post, please comment here about what you think should be in 1.0.0

I have created three new types labels, 1.0.0, Blocking, Breaking

• Improve installation experience on Mac OSX (Better XGBoost installation on Mac OSX? #4477)
• Remove old GPU objectives.
• Remove gpu_exact updater (deprecated) Deprecate gpu_exact, bump required cuda version in docs #4527
• Remove multi threaded multi gpu support (deprecated) [RFC] Remove support for single process multi-GPU #4531
• External memory for gpu and associated dmatrix refactoring [RFC] External memory support for GPU #4357 [RFC] Possible DMatrix refactor #4354
• Spark Checkpoint Performance Improvement ([jvm-packages] Checkpointing performance issue in XGBoost4J-Spark #3946)
• [BLOCKING] the sync mechanism in hist method in master branch is broken due to the inconsistent shape of tree in different workers ([HOTFIX] distributed training with hist method  #4716, [BLOCKING] Per-node sync slows down distributed training with 'hist' #4679)
• Per-node sync slows down distributed training with 'hist' ([BLOCKING] Per-node sync slows down distributed training with 'hist' #4679)
• Regression tests including binary IO compatibility, output stability, performance regressions.

[thesuperzapper]

Not a committer, but can we please target PySpark API for 1.0?
Issue: #3370
Current PR: #4656

[CodingCat]

for other contributors who have no permission to edit the post, please comment here about what you think should be in 1.0.0

[thesuperzapper]

Also, should we target moving exclusively to the Scala based Rabit tracker (for Spark) in 1.0?

[trams]

I am also not a committer but me and the company I work in is very interested in fixing the performance issue with checkpointing (or at least mitigate it) #3946

[trivialfis]

@trams @thesuperzapper I think this is an overview for everyone to have a feeling for what's coming next. It would be difficult to list everything coming since XGBoost is a community driven project. Just open a PR when it's ready.

Not a committer, but can we please target PySpark API for 1.0?

@thesuperzapper Let's track the progress. I certainly hope that I can start testing it. :-)

[thesuperzapper]

There is also the secondary consideration, that we might not be ready for 1.0, and the API guarantees that come with that, for example, we could instead do 0.10.0 next?

[trivialfis]

@thesuperzapper 1.0 is not gonna be a final version. It's just we are trying to do semantic versioning.

[RAMitchell]

Added some gpu related items.

[chenqin]

would like to get native xgb fix included.
#4753

[trivialfis]

JSON is removed from the list. See #4683 (comment)