Latency

[antoine-levitt]

Could be good to run https://discourse.julialang.org/t/new-tools-for-reducing-compiler-latency/52882, although I would imagine that our latency cost is because of our deps, not because of us.

[mfherbst]

Actually I think SnoopCompile will help quite a bit. Could also be super helpful for my new design in asedftk. I'm definitely going to take a look at that at some point.

[antoine-levitt]

At least it'll tell us which packages slow us down and then we can bug them.

[antoine-levitt]

I just tried the new 1.6 beta, a lot of things in general feel faster but time julia silicon.jl is actually slower, so there might be some scope for latency optimization in DFTK.

[antoine-levitt]

Something is seriously wrong somewhere. Just taking out lobpcg_hyper_impl.jl with a simple test script

A = randn(10, 10); A = A+A'
X = randn(10, 2)
ret = LOBPCG(A, randn(10, 2))
println(ret.n_matvec)

has a second @time include of 4s, due to inference. This is on julia 1.6.1

[antoine-levitt]

Playing with JuliaLang/julia#41612:

julia> using DFTK
    125.7 ms  FillArrays
    965.6 ms  ArrayLayouts
    853.7 ms  StaticArrays
    859.2 ms  DiffResults
      4.0 ms  Compat
      6.0 ms  Ratios
      1.0 ms  Requires
      0.9 ms  Adapt
      7.9 ms  WoodburyMatrices
     11.6 ms  AxisAlgorithms
     57.2 ms  ChainRulesCore
     55.1 ms  OffsetArrays
    198.2 ms  Interpolations
     86.4 ms  DataStructures
      0.7 ms  NaNMath
      0.3 ms  UnPack
      1.4 ms  DataAPI
      0.4 ms  StatsAPI
      0.9 ms  SortingAlgorithms
     11.3 ms  Missings
     46.0 ms  StatsBase
      1.8 ms  PositiveFactorizations
      1.8 ms  Parameters
      4.3 ms  DocStringExtensions
      0.6 ms  CompilerSupportLibraries_jll
      3.1 ms  LogExpFunctions
      0.6 ms  Preferences
      2.8 ms  JLLWrappers
     12.6 ms  OpenSpecFun_jll
     33.3 ms  SpecialFunctions
     54.1 ms  MacroTools
     57.5 ms  CommonSubexpressions
      1.1 ms  DiffRules
    193.2 ms  ForwardDiff
      0.3 ms  IfElse
     48.9 ms  Static
    198.5 ms  ArrayInterface
     14.6 ms  FiniteDiff
      7.8 ms  NLSolversBase
    478.1 ms  LineSearches
    564.8 ms  Optim
      0.5 ms  ExprTools
     26.1 ms  RecipesBase
      2.2 ms  Mocking
     67.1 ms  TimeZones
    102.5 ms  Intervals
    311.7 ms  MutableArithmetics
   1202.4 ms  Polynomials
    461.6 ms  Libxc_jll
    466.3 ms  Libxc
      6.7 ms  Distances
      0.3 ms  VersionParsing
     13.2 ms  Conda
   1476.3 ms  PyCall
     10.2 ms  AbstractFFTs
      4.3 ms  MPICH_jll
     52.1 ms  MPI
      0.6 ms  Reexport
    297.0 ms  LinearMaps
      0.5 ms  CommonSolve
      1.8 ms  ConstructionBase
     47.6 ms  TimerOutputs
    195.7 ms  Unitful
   1121.0 ms  PeriodicTable
     36.0 ms  IterTools
      3.1 ms  ProgressMeter
     20.5 ms  IterativeSolvers
      4.9 ms  NLsolve
    333.1 ms  UnitfulAtomic
      1.3 ms  FFTW_jll
    274.1 ms  FFTW
    193.0 ms  BlockArrays
      1.4 ms  spglib_jll
      9.5 ms  Roots
   8915.1 ms  DFTK

[mfherbst]

Hmm once #483 is in we can probably make PyCall optional. That already is quite a large chunk. As for PeriodicTable that is also mostly for convenience, but I wonder why it takes so long to load. It is a pretty small package.

[mfherbst]

But 9 seconds is really not great. We should definitely work on that.

[antoine-levitt]

The load time is not even that much of an issue, the time to first SCF feels longer in my experience. The LOBPCG stuff is apparently responsible for some of it (though I have no idea why)

[antoine-levitt]

https://discourse.julialang.org/t/ann-new-package-snoopprecompile/84778/4 sounds like exactly what we need

[mfherbst]

No it does not help. I tried it yesterday. We have too many invalidations. We need to fix those first.

[antoine-levitt]

Pretty sure that's not our issue. They always talk about that and it's one source of latency, but I don't think that's the one we suffer from. At least I couldn't find anything problematic when I took at look last time

[antoine-levitt]

List of things to try:

• see if removing nlsolve helped (thanks @epolack )
• once Julia PR 47184 is in, add precompilation (ie run an example computation on precompilation)
• transition to https://pkgdocs.julialang.org/dev/creating-packages/#Conditional-loading-of-code-in-packages-(Extensions) instead of Requires for 1.9
• fix the LOBPCG slowness described above

[antoine-levitt]

The julia beta released today has the latency improvements, which look pretty spectacular. Not at a computer for a while, but very curious to see what it looks like for us