helm-get-candidates performance issue?

[LiuGavin]

I was using magit in emacs, and whenever it a 'diff' on a relative-large(maybe hundreds diffs), it performed badly and the CPU is at 100% for a couple of minutes.. below is my "profiler-report" when it's slow.-->

- ...                                                          163858  96%
 - helm-get-candidates                                         162478  95%
  - let*                                                       162478  95%
   - condition-case                                            162478  95%
    - if                                                       162478  95%
     - helm-interpret-value                                    162478  95%
      - cond                                                   162478  95%
       - helm-apply-functions-from-source                      162478  95%
        - let                                                  162478  95%
         - let*                                                162478  95%
          - while                                              162478  95%
           - setq                                              162478  95%
            - apply                                            162478  95%
             - #<lambda 0x1ace6d31292ddae6>                    162478  95%
              - let*                                           162478  95%
               - complete-with-action                          162476  95%
                - #<lambda 0xfc457663eef3ce8>                  162476  95%
                 - let*                                        162476  95%
                  - completion-all-completions                 162373  95%
                   - completion--nth-completion                162373  95%
                    - completion--some                         162373  95%
                     - #<compiled 0x423eb7d>                   162373  95%
                      - helm-completion-all-completions             162136  95%
                       - let*                                  162136  95%
                        - helm-completion--multi-all-completions             162135  95%
                         - let*                                162135  95%
                          - helm-completion--multi-all-completions-1             162135  95%
                           - let*                              162135  95%
                            - if                               162135  95%
                             - all-completions                 161961  95%
                              - #<lambda 0x11a75dd4d76e4fd8>             161868  95%
                               - let                           161751  95%
                                - if                           161714  95%
                                 - and                         161637  95%
                                  - helm-mm-match              161465  94%
                                   - let*                      161421  94%
                                    - progn                    161407  94%
                                     - let                     161395  94%
                                      - funcall                161379  94%
                                       - helm-mm-3-match             161361  94%
                                        - let*                 161353  94%
                                         - if                  161224  94%
                                          - car-safe             161224  94%
                                           - prog1             161223  94%
                                            - helm-execute-selection-action-1             161218  94%
                                             - let             161218  94%
                                              + catch             154910  91%
                                              + if               6304   3%

Thanks in advance.

[thierryvolpiatto]

LiuGavin ***@***.***> writes:

I was using magit in emacs, and whenever it a 'diff' on a relative-large(maybe hundreds diffs), it performed badly and the CPU is at 100% for a couple of minutes.. below is my "profiler-report" when it's slow.--> - complete-with-action 162476 95% - #<lambda 0xfc457663eef3ce8> 162476 95% - let* 162476 95% - completion-all-completions 162373 95% - completion--nth-completion 162373 95% - completion--some 162373 95% - #<compiled 0x423eb7d> 162373 95% - helm-completion-all-completions 162136 95% - let* 162136 95% - helm-completion--multi-all-completions 162135 95% - let* 162135 95% - helm-completion--multi-all-completions-1 162135 95%

If this is called I guess you are using 'emacs as `helm-completion-style` value, right? Helm is using 'helm as value to avoid using native emacs completion API which is slow. So what is slow here is not `helm-get-candidates` but the emacs completion default system (basically all what is in minibuffer.el). I suggest using helm default value for `helm-completion-style` which should be much faster.

…

-- Thierry

[LiuGavin]

Thanks for the quick response. I am new to Helm, and not sure how to set 'helm-complemtion-style' properly in emacs.

 '(helm-completion-style 'emacs)
 '(completion-styles '(flex))         // ---> I added this line here, but it does not seem to help? 
 '(helm-mode t)

what should I set instead?

[thierryvolpiatto]

LiuGavin ***@***.***> writes:

Thanks for the quick response. I am new to Helm, and not sure how to set 'helm-complemtion-style' properly in emacs. '(helm-completion-style 'emacs) '(completion-styles '(flex)) // ---> I added this line here, but it does not seem to help? '(helm-mode t) what should I set instead?

First of all, it looks you have your custom stuff in your init.el or .emacs.el file, right? If so separate your custom stuff from your personal config in init.el. (setq custom-file "~/.emacs.d/.emacs-custom.el") Then move all what you have at end of your init.el file in .emacs-custom.el (what starts by "(custom-set-variables...") Now you can either use customize interface to customize your user vars like helm-completion-style or remove this entry in .emacs-custom.el file, then helm will use the default value which is 'helm. For helm-mode you can remove the entry in .emacs-custom.el and add to init.el file: (helm-mode 1) Once helm-completion-style is set to 'helm completion-styles will have no more effect in helm. Don't forget to load the .emacs-custom.el file from your init.el file: (load custom-file) As helper, my config file for helm is here: https://github.com/thierryvolpiatto/emacs-config/blob/main/init-helm.el And init file for emacs here: https://github.com/thierryvolpiatto/emacs-config/blob/main/init.el

…

-- Thierry

[thierryvolpiatto]

Also about setting variables in emacs:

Normally you can use setq e.g. (setq foo nil) but some variables built with defcustom may have a :set function, in this case you have to use customize-set-variable instead of setq which takes care of calling the :set function among other things.

[LiuGavin]

I try to use your config file and see some improvement, but still slow ... below is the updated 'profiler-report'

- ...                                                            1616  91%
 - helm                                                          1411  80%
  - let                                                          1411  80%
   - if                                                          1411  80%
    - if                                                         1411  80%
     - apply                                                     1411  80%
      - helm-internal                                            1411  80%
       - let                                                     1411  80%
        - unwind-protect                                         1411  80%
         - condition-case                                        1411  80%
          - let                                                  1411  80%
           - if                                                  1234  70%
            - helm-display-buffer                                1234  70%
             - let                                               1234  70%
              - prog1                                            1234  70%
               - helm-execute-selection-action-1                 1231  70%
                - let                                            1231  70%
                 - catch                                         1207  68%
                  - let                                          1207  68%
                   - let*                                        1207  68%
                    - while                                      1207  68%
                     - setq                                      1207  68%
                      - cons                                     1207  68%
                       - helm-compute-matches                    1207  68%
                        - save-current-buffer                    1207  68%
                         - let                                   1207  68%
                          - helm-process-filtered-candidate-transformer               1207  68%
                           - if                                  1147  65%
                            - helm--initialize-one-by-one-candidates               1147  65%
                             - helm-take-first-elements               1147  65%
                              - helm-get-cached-candidates               1146  65%
                               - let*                            1146  65%
                                - let                            1146  65%
                                 - if                            1146  65%
                                  - let                          1146  65%
                                   - helm-get-candidates               1146  65%
                                    - let*                       1146  65%
                                     - condition-case               1145  65%
                                      - if                       1145  65%
                                       - helm-interpret-value               1145  65%
                                        - cond                   1145  65%
                                         - helm-apply-functions-from-source               1145  65%
                                          - let                  1145  65%
                                           - let*                1145  65%
                                            - while               1145  65%
                                             - setq               1145  65%
                                              - apply               1145  65%
                                               + #<lambda -0x14fe7f5858d7859c>               1145  65%
                              - if                                  1   0%
                                 >                                  1   0%
                           + let                                   60   3%
                 + if                                              24   1%
               + funcall                                            3   0%
           + unwind-protect                                       176  10%
           + helm-initialize                                        1   0%
   Automatic GC                                                   199  11%
 + #<lambda -0x7e4261a6c36b733>                                     5   0%

Do you mind to do a 'diff' in your environment to see what's the performance there? I attached two text files here. I put b2.csv 'staged' in magit first then rename c2.csv to b2.csv, hightlight it then emacs running 100%CPU at this point for me.
b2.csv
c2.csv

[thierryvolpiatto]

LiuGavin ***@***.***> writes:

Do you mind to do a 'diff' in your environment to see what's the performance there? I attached two text files here. I put b2.csv 'staged' in magit first then rename c2.csv to b2.csv, hightlight it then emacs running 100%CPU at this point for me. b2.csv c2.csv

How diff would involve helm? Can you be more specific on what you are doing exactly? A recipe would be nice. Also how did you install helm? Do you use a minimal install or are you using a preconfigured emacs (spacemacs, doom or whatever)? Thanks.

…

-- Thierry

[LiuGavin]

I installed the 'magit' from here: https://magit.vc/manual/magit/Installing-from-Melpa.html

   To use Melpa-Stable: 

(require 'package)
(add-to-list 'package-archives
             '("melpa-stable" . "http://stable.melpa.org/packages/") t)

(I already have 'git' installed in my linux).
once magit installed , try to create a new git project(or any existing git project would be fine), copy b2.csv to this directory, in magit press 'ctrl x+g' to bring up the git repo, highlight b2.csv and press "s" to stages it(try to use it as a baseline). copy c2.csv to this directory, rename it to 'b2.csv'(mimic the case some new changes made to b2.csv), press 'g' in magit to refresh the git repo, you should see "Unstaged" b2.csv and "Staged" b2.csv , hightlight the "Unstaged" b2.csv file in magit, (where I believe it's doing the 'diff' here, because once it's finished it will show the diff result between 'unstaged' and 'staged' version of the b2.csv.) and here is where I did the 'profiler report' and see the 100%CPU.

   Unstaged changes (1)
   modified     b2.csv             // ---> highlight this one now and you will notice CPU in 100%

   Staged changes (1)
   modified     b2.csv

I installed helm while ago, I believed I did it from https://github.com/emacs-helm/helm/wiki#install with the same "Melpa" way. here is my emacs config related to 'helm'

(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(completion-styles '(flex))
 '(dired-async-mode t)
 '(helm-mode t)
 '(package-selected-packages
   '(flex-isearch lsp-mode yasnippet lsp-treemacs helm-lsp projectile hydra flycheck company avy which-key helm-xref dap-mode))
 '(tab-stop-list '(4 8 12 16)))

(setq
 helm-gtags-ignore-case t
 helm-gtags-auto-update t
 helm-gtags-use-input-at-cursor t
 helm-gtags-pulse-at-cursor t
 helm-gtags-prefix-key "\C-cg"
 helm-gtags-suggested-key-mapping t
 )

(require 'helm-gtags)
;; Enable helm-gtags-mode
(add-hook 'dired-mode-hook 'helm-gtags-mode)
(add-hook 'eshell-mode-hook 'helm-gtags-mode)
(add-hook 'c-mode-hook 'helm-gtags-mode)
(add-hook 'c++-mode-hook 'helm-gtags-mode)
(add-hook 'asm-mode-hook 'helm-gtags-mode)

(define-key helm-gtags-mode-map (kbd "C-c g a") 'helm-gtags-tags-in-this-function)
(define-key helm-gtags-mode-map (kbd "C-c g b") 'helm-gtags-find-rtag)
(define-key helm-gtags-mode-map (kbd "C-j") 'helm-gtags-select)
(define-key helm-gtags-mode-map (kbd "M-.") 'helm-gtags-dwim)
(define-key helm-gtags-mode-map (kbd "M-,") 'helm-gtags-pop-stack)
(define-key helm-gtags-mode-map (kbd "C-c <") 'helm-gtags-previous-history)
(define-key helm-gtags-mode-map (kbd "C-c >") 'helm-gtags-next-history)

(autoload 'dired-async-mode "dired-async.el" nil t)
(dired-async-mode 1)

Appreciated.

[thierryvolpiatto]

LiuGavin ***@***.***> writes:

I installed the 'magit' from here: https://magit.vc/manual/magit/Installing-from-Melpa.html To use Melpa-Stable: (require 'package) (add-to-list 'package-archives '("melpa-stable" . "http://stable.melpa.org/packages/") t) (I already have 'git' installed in my linux). once magit installed , try to create a new git project(or any existing git project would be fine), copy b2.csv to this directory, in magit press 'ctrl x+g' to bring up the git repo, highlight b2.csv and press "s" to stages it(try to use it as a baseline). copy c2.csv to this directory, rename it to 'b2.csv'(mimic the case some new changes made to b2.csv), press 'g' in magit to refresh the git repo, you should see "Unstaged" b2.csv and "Staged" b2.csv , hightlight the "Unstaged" b2.csv file in magit, (where I believe it's doing the 'diff' here, because once it's finished it will show the diff result between 'unstaged' and 'staged' version of the b2.csv.) and here is where I did the 'profiler report' and see the 100%CPU.

I installed magit and followed your recipe, the delay you see is due to magit preparing the diff, not helm. You can verify by turning off helm-mode and run the same command from magit, you will see the delay before the diff popup is the same.

…

-- Thierry

[LiuGavin]

Thanks for the checking. Indeed it's the magit's diff causing performance...I didn't know that and the "profiler-report" made me thought it was helm. Now I have to look at what could make the magit's diff performs better.