Modularize library

[uruuru]

It would be nice to have have several modules instead of one large file: core, alg.layered, alg.stress, etc. However, I don't see any way to realize this with the current GWT compilation solution.

Currently the library is about 1.3 MB in size (minified); about 600 kB for core and layered, and 100 kB for the remaining algorithms.

[tonatos]

same issue!))

[uruuru]

An update:

There's the j2cl project which seems promising to me. After all the GWT transpiler itself is more or less discontinued (future GWT will be somewhat based on j2cl) and doesn't properly support newer Java versions.

Thus, elkjs has to transition to a different transpiler anyway. However, getting everything to work (if possible) with j2cl seems to be quite some effort.

[Nic30]

I am using elkjs + d3.js and the minified bundle is 2.1MB in size. If I use elk in widget for jupyter notebook I am gettitng around 8MB of minified javascript on nearly empty page.
elkjs also significantly slows down webpack build.
Another problem is webworker-threads it's getting deprecated and is unreliable.

[uruuru]

Unfortunately I don't see a modularized version of the algorithms itself any time soon. In addition to the necessity that the transpiler supports it, I see several further difficulties.

I'll look into the webworker-threads point though - I wasn't aware of the deprecation yet. It's an optional dependency, though.

In which way does it slow down the webpack build? Without being a webpack expert, can't you exclude elkjs from webpack itself (in particular, if you use a webworker)? I believe we do it like that in https://github.com/kieler/elkgraph-web

[Nic30]

can't you exclude elkjs from webpack itself

In normal build yes. However I do not know how to do it in bundle for jupyter widget (and I do not think it is possible)

https://github.com/Nic30/jupyter_widget_hwt/blob/master/js/webpack.config.js#L39

I guess that it takes 90s just because it is too big. https://travis-ci.org/github/Nic30/jupyter_widget_hwt/builds/708874010
It is not a big problem unless it is in docker on Binder which is my case. https://mybinder.org/v2/gh/Nic30/jupyter_widget_hwt.git/master?filepath=examples%2Fexample_simple.ipynb

[BoykoAlex]

Any chance instead of modularizing we could have elkjs-layered for example? Same as klayjs with one layered algorithm only but based on ELK... The size of this becomes 550Kb and rather straight-forward to accomplish it seems... What do you think?

[uruuru]

I guess I could do that.

So far I was still hoping to find a proper solution instead of simply publishing duplicate builds. It still requires some effort though to properly automate the process.

EDIT: I just double-checked the resulting file size. Did you achieve the 550kB with a build yourself or are you referring to my original comment, which seems inaccurate? For me removing all algorithms despite layered yields 1.3MB (instead of 1.4). This wouldn't justify the effort in my eyes (for layered at least).

[BoykoAlex]

Hmm... i thought i got 550Kb but i was mistaken it is 1.3Mb :-(
I did build it myself by excluding other algorithms in the build.gradle...

Makes me wonder how klayjs is 500Kb. Is it the EMF, Xtext, Guava dependencies that make elk.bundled.js so heavy?

[uruuru]

Regarding klayjs:

• guava should already have been used there as well
• xtext shouldn't add any significant size in this case (it's only required during compile time to generate code from the melk files)

EMF is the largest factor, I presume. Also, ELK layered itself continuously increased in size.

I thought about removing EMF from ELK itself for several reasons but that's probably at least a month's worth of effort.

[uruuru]

As to the original intent of this issue:

By now, several bytecode to js/webassembly transpilers have emerged: bytecoder, TeaVM, JWebAssembly, to name what I'm aware of so far.

I could imagine them being a viable alternative for the future, especially, since they allow compilation to web assembly as well.

[redexp]

I wrote few scripts, which removes functions declarations from elk-worker.js which are not used in my project. I'v got 2.1 MB of source file (comparing to 4.5 MB of original file) and 800 KB of minified file (I think it could be better if spend more time on minifier options).

To make it possible I went through next steps:

1. Run first script which adds to each function body line like $used[123] = true; (123 is id of this function) and creates new file like elk-worker-dev.js
2. Run this file in real project
3. Get keys from object $used and pass them to second script which will remove all functions which not in list and creates new file like elk-worker-clear.js
4. Run this file in real project and see if there any errors. If it is then go to step 2 and try to use your project more meticulously

I can share those two scripts if you interested

[joryphillips]

@redexp Can you share your scripts? I would like to give that a try. Thank you!

[redexp]

@redexp Can you share your scripts? I would like to give that a try. Thank you!

https://gist.github.com/redexp/7d4e7b7731c85e4ce8f29c26cb234fad

first npm install -D recast esprima-next

then run build-dev.js it will create elk-worker-dev.js with extra code which will every second check is array of used functions from elk is changed, if it is then it will log it to console. This array you should write to used.json and run build-clear.js. Tested on elkjs 0.8.2