Skip to content

Release 0.11.0 #476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 31, 2021
Merged

Release 0.11.0 #476

merged 9 commits into from
Jul 31, 2021

Conversation

yannikschaelte
Copy link
Member

@yannikschaelte yannikschaelte commented Jul 30, 2021

Diverse:

Semi-automatic summary statistics and robust sample weighting (#429)

Breaking changes:

  • API of the (Adaptive)PNormDistance was altered substantially to allow
    cutom definition of update indices.
  • Internal weighting of samples (should not affect users).

Semi-automatic summary statistics:

  • Implement (Adaptive)PNormDistance with the ability to learn summary
    statistics from simulations.
  • Add sumstat submodule for generic mappings (id, trafos), and especially a
    PredictorSumstat summary statistic that can make use of Predictor objects.
  • Add subsetting routines that allow restricting predictor model training
    samples.
  • Add predictor submodule with generic Predictor class and concrete
    implementations including linear regression, Lasso, Gaussian Process,
    Neural Network.
  • Add InfoWeightedPNormDistance that allows using predictor models to weight
    data not only by scale, but also by information content.

Outlier-robust adaptive distances:

  • Update documentation towards robust distances.
  • Add section in the corresponding notebook.
  • Implement PCMAD outlier correction scheme.

Changes to internal sample weighting:

  • Do not normalize weights of in-memory particles by model; this allows to
    more easily use the sampling weights and the list of particles for
    adaptive components (e.g. distance functions)
  • Normalization of population to 1 is applied on sample level in the
    sampler wrapper function
  • In the database, normalization is still by sample to not break old db
    support; would be nicer to also there only normalize by total sum
    -- requires a db update though.

Changes to internal object instruction from samples:

  • Pass sample instead of weighted_sum_stats to distance function.
    This is because thus the distance can choose on its own what it wants
    -- all or only accepted particles; distances; weights; parameters;
    summary statistics.

Visualization:

  • Function to plot adaptive distance weights from log file.

yannikschaelte and others added 5 commits May 11, 2021 11:18
* add la nb

* allow subsetting labels

* cont

* run la nb; fix walltime plots

* cont

* fix up
* tmp fix redis clean-up

* update to pyarrow changes
* init

* limit look-ahead sample number in delayed mode

* update releasenotes -> 0.10.15

* refactor: weight normalization applied to all particles; pass sample to distance function

* fix typo in changelog

* fix tmp changes

* fixup

* pycharm annoys me

* fixit

* fix population test

* fix zero division error

* fix docs

* whatever

* remove file

* init

* tmp

* tmp

* refactor adaptive distances: sumstat + vectorize

* add working version of sumstat and predictor modules

* whatever

* add nbs

* handle trivial statistics better

* normalize info weighting correctly

* refactor anew info weighting + normalization + gp and layer handles

* fix flake8

* add lasso sumstat

* set indices to keep correctly

* add option to not normalize per parameter in info weight

* cont

* implement late model use

* remove slad

* tidy up

* update nbs; fix various things

* add predictor test

* add model selection test

* additional tests

* update readme; add raise tests

* add sumstat test

* add test for dict2arr

* test info weighting

* test sample construction

* test fit index construction

* test inf norm; test scales errors

* fixup

* implement subsetting

* fix imports

* test augmentation

* add missing base class dependency

* move worker signup up

* add logger

* always normalize linreg inputs; postpone default fit indices

* do not clear up redis server

* fix typo

* reset default scale function from rmsd to std for stability in most cases

* cont

* cont

* update

* add tests

* whatever

* Allow fitting at simulation-based events (#462)

* Allow fitting at simulation-based events

* update nb

* cont

* fix test

* fix test

* Add distance weight plot (#463)

* fix wrong deviation threshold 0.5 -> 0.33

* Small fixes (#466)

* Add distance weight plot

* add colors

* enable passing keys

* integer coordinates

* implement option to use only accepted particles for scale calculation in adaptive distances (#467)

* implement only accepted particles for scale calculation

* add test

* fix indent

* add max mlp method

* log fitting time

* add train-test-split model selection method

* better info weight calculation

* add pre_before_fit and from_events

* fix

* change default to weights

* normalize in subsetter

* add n_sample option to data plot

* fix stuff

* allow kwargs in distance weights plot

* add pcmad convenience

* apply la normalization to all particles

* fix defaults

* final edits

Co-authored-by: Yannik Schälte <[email protected]>
Co-authored-by: Yannik Schälte <[email protected]>
@codecov-commenter
Copy link

codecov-commenter commented Jul 30, 2021

Codecov Report

❗ No coverage uploaded for pull request base (main@bb90794). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #476   +/-   ##
=======================================
  Coverage        ?   88.11%           
=======================================
  Files           ?      113           
  Lines           ?     7104           
  Branches        ?        0           
=======================================
  Hits            ?     6260           
  Misses          ?      844           
  Partials        ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb90794...5942c10. Read the comment docs.

@yannikschaelte yannikschaelte changed the title Release 0.10.17 Release 0.11.0 Jul 30, 2021
@yannikschaelte yannikschaelte self-assigned this Jul 31, 2021
@yannikschaelte yannikschaelte merged commit b7c08a9 into main Jul 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants