Usage of multiple cores during report generation

[Lorilatschki]

Hi,

we have recognized that the generator doesn't take advantage of all available cores. We have very powerful build agents where ReportGenerator is executed, but it seems that it doesn't scale.

32 Cores 1,5 hours
88 Cores almost the same 1,5 hours

Is it possible, to extend ReportGenerator by a new command argument specifing the amount of cores ReportGenerator should take care of during generation?

[danielpalme]

Hi,

I will consider this for the next release.
It won't work for every report type. But for HTML it should be possible.

Your execution time seems extremely long.
How many classes and lines of code does your code base have?

[Lorilatschki]

Almost one million lines of code...it's a huge project with many assemblies from different parties. The following statistics are just those we cover during test execution.

Assemblies: 19
Classes: 4557
Files: 4407
Coverable lines: 203307
Total lines: 835926

[danielpalme]

Could you please try this version:
https://www.nuget.org/packages/ReportGenerator/4.0.0-alpha2

I'm now using several cores to generate the class reports
Let me know if this works for you or if you have any issues.

How many hours/minutes does it take to generate the report now?

[Lorilatschki]

Hi Daniel, I will try out tomorrow. Do i have to use a special argument or do you implemented mutlicore usage by default?

[danielpalme]

Nothing special to do. Just replace the existing version.

[Lorilatschki]

Thanks i‘ll keep you informed.

[Lorilatschki]

Hi Daniel, i got a SystemOutOfMemory exception during the first run. Exception of type 'System.OutOfMemoryException' was thrown. Did you close all file handles immediately?  

[danielpalme]

Thanks for testing.
ReportGenerator first parses the XML file(s) generated by OpenCover.

The individual class files are only parsed when the report file for that class is generated.

In your case it seems that the parsing of the XML files generated by OpenCover fails.
How big are these file(s)?

[Lorilatschki]

About 80MB each file.

[danielpalme]

Strange that you don't have issues with the current version ReportGenerator, I did not change anything regarding the parsing of the XMLs.

How many XML files are passed to ReportGenerator?

Can you share one of those large reports by mail?
You can of course obfuscate your namespace and class names if you like?

If not, I have to create a large file myself.

[Lorilatschki]

No we never had such an exception before. It‘s about 500 files and each of it has something like 80mb. Could you please create such a big file on your own because of security restrictions i am not allowed to share it easily.

[danielpalme]

OK, will create one myself.

[Lorilatschki]

Ok if you need further informations let me know.