XPS documents memory leak (System.Windows.ContextLayoutManager+LayoutQueue+Request and XpsDocument.GetFixedDocumentSequence())

[wstaelens]

originally posted at runtime but is for wpf: dotnet/runtime#66756

Description

Situation: we have a windows service running in .net 6 with an admin panel in asp.net core 6 (+ blazor + pri nt drivers etc...)

(complex solution, but for the memory leak mainly does this:)

PDF files dumped in hotfolder 
--> opened by our code and converted to XPS 
--> modifications to XPS file (e.g. adding barcode) 
--> converting back to PDF the single (or merged) modified XPS documents.
--> dumping the resulting PDF files to a folder

When processing 5 documents, we see an increase of memory with ±1.5MB memory because System.Windows.ContextLayoutManager+LayoutQueue+Request and this is because of being called by XpsDocument.GetFixedDocumentSequence() are never released or GC'd.

Eventually when our service runs for a time, it just runs out of memory.

Reproduction Steps

unable to create a minimal repro-solution, our software should be run and configured to reproduce (if needed somebody can get in contact with me in PM).

I guess these code in stackoverflow topics will demonstrate a similar issue:
https://stackoverflow.com/questions/218681/opening-xps-document-in-net-causes-a-memory-leak
https://stackoverflow.com/questions/51463348/c-sharp-reading-xps-causing-memory-leak
https://stackoverflow.com/questions/12703796/creating-fixed-document-causes-memory-leak
https://social.msdn.microsoft.com/Forums/en-US/becc0d42-908a-435c-a4ff-175843b83ad8/memory-leak-while-opening-the-xps-document-in-documentviewer
https://social.msdn.microsoft.com/Forums/vstudio/en-US/b7fec24a-138b-4d3b-bdb3-cd4f0785e4ac/populating-collection-with-rendertargetbitmap-via-thread-causes-memory-leak

Expected behavior

Processing XPS documents should release memory.

Actual behavior

When processing XPS documents the memory increases, after processing 5.000 - 10.000 documents, our service crashes due to OutOfMemory. (depending on the hardware and type of documents this number can be higher/lower...)

Regression?

We've seen this issue in every version of .net core, seen it last in .net 5 and after upgrading to .net 6 it is still present.

This memory leak and the issues have been out there already for a while (also in .NET Framework).
see:

https://stackoverflow.com/questions/218681/opening-xps-document-in-net-causes-a-memory-leak
https://stackoverflow.com/questions/51463348/c-sharp-reading-xps-causing-memory-leak
https://stackoverflow.com/questions/12703796/creating-fixed-document-causes-memory-leak
https://social.msdn.microsoft.com/Forums/en-US/becc0d42-908a-435c-a4ff-175843b83ad8/memory-leak-while-opening-the-xps-document-in-documentviewer
https://social.msdn.microsoft.com/Forums/vstudio/en-US/b7fec24a-138b-4d3b-bdb3-cd4f0785e4ac/populating-collection-with-rendertargetbitmap-via-thread-causes-memory-leak

Known Workarounds

No working workaround for us... :sad

Configuration

VS 2019 Pro 16.11.11
VS 2022 17.2.0 Preview 2.0
Issue occurred in every version we've been using .net 5, .net 6

Windows 10 21H1
Windows 11 21H2

Other information

suggested fixes as preloading PresentationCore and/or PresentationFramework don't work (https://web.archive.org/web/20110404040352/http://support.microsoft.com/kb/942443)

Calling updatelayout as suggested in stackoverflow also doesn't seem to work:
https://stackoverflow.com/questions/218681/opening-xps-document-in-net-causes-a-memory-leak
https://stackoverflow.com/questions/51463348/c-sharp-reading-xps-causing-memory-leak
https://stackoverflow.com/questions/12703796/creating-fixed-document-causes-memory-leak
https://stackoverflow.com/questions/50560580/memory-leak-at-xpsdocument-getfixeddocumentsequence
https://social.msdn.microsoft.com/Forums/en-US/becc0d42-908a-435c-a4ff-175843b83ad8/memory-leak-while-opening-the-xps-document-in-documentviewer
https://social.msdn.microsoft.com/Forums/vstudio/en-US/b7fec24a-138b-4d3b-bdb3-cd4f0785e4ac/populating-collection-with-rendertargetbitmap-via-thread-causes-memory-leak

We really hope the team investigates some time in XPS as it is really terrible, and it always being updated to "Future", "Future", "Future" 😠 .
See also:

• XPS huge memory issues #6296

• XPS documents memory leak (System.Windows.ContextLayoutManager+LayoutQueue+Request and XpsDocument.GetFixedDocumentSequence()) runtime#66756

• XPS documents from print driver are terribly slow runtime#51930

• XPS documents in .NET Core can't be opened #3546

• XPS documents in .NET Core can't be opened runtime#51929 (comment)

• .NET 5 - Printing - XPS requires STA Threads #4000

• [Question] [XPS] Drawing a Visual is always being rasterized #4466

• Printing XPS files fails on Windows 8.1 #2106

• Update XPS Serialization Stack for Packaging/Compression Semantic Changes #2085

• Possible unmanaged memory leak when running code using Print API(DocumentPaginator) on Windows 10 #3754

[wstaelens]

@gurpreet-wpf let me know if you need anything...

[wstaelens]

I don't know if this helps @gurpreet-wpf ?

00007ffcf5b83288 407286 16291440 System.Windows.ContextLayoutManager+LayoutQueue+Request
00007ffcf5b1b290 407286 19549728 System.Windows.LayoutEventList+ListItem

😢

!dumpheap -stat

00007ffcf4e77768    68364      2734560 System.Xml.XmlAttribute
00007ffcf4e78038    50602      2833712 System.Xml.XmlElement
00007ffcf50144f0     5920      3173120 System.Xml.XmlName[]
00007ffcf4e78848    80200      3208000 System.Xml.XmlText
00007ffcf1edb578    31071      4030104 System.Object[]
00007ffcf23b3fa8    11843      4831304 System.Xml.NameTable+Entry[]
00007ffcf5014408    71028      5114016 System.Xml.XmlName
00007ffcf203e4f8        3      5543112 System.Diagnostics.Tracing.EventSource+EventMetadata[]
00007ffcf20362d0    66407      6906328 System.Reflection.RuntimeMethodInfo
00007ffcf1f88410    28047      9811424 System.Int32[]
00007ffcf23b3f18   266454     10658160 System.Xml.NameTable+Entry
00007ffcf5b83288   407286     16291440 System.Windows.ContextLayoutManager+LayoutQueue+Request
00007ffcf1f8d698   191265     18064820 System.String
00007ffcf5b1b290   407286     19549728 System.Windows.LayoutEventList+ListItem
00007ffcf5b151e0        1     33554456 System.Runtime.CompilerServices.ConditionalWeakTable`2+Entry[[System.Object, System.Private.CoreLib],[System.Windows.Diagnostics.XamlSourceInfo, PresentationCore]][]
00007ffcf2033c98    19941     41548477 System.Byte[]
000001e3abe3b130     5026    257044104      Free


!DumpHeap /d -mt 00007ffcf5b83288

...
000001e3b91deab0 00007ffcf5b83288       40     
000001e3b91dead8 00007ffcf5b83288       40     
000001e3b91deb00 00007ffcf5b83288       40     
000001e3b91deb28 00007ffcf5b83288       40     

Statistics:
              MT    Count    TotalSize Class Name
00007ffcf5b83288   407286     16291440 System.Windows.ContextLayoutManager+LayoutQueue+Request
Total 407286 objects
Fragmented blocks larger than 0.5 MB:
            Addr     Size      Followed by
000001E3B84CA780    1.8MB         000001E3B8692E60 System.Windows.Media.Imaging.BitmapSizeOptions
000001E3B8693578    5.5MB         000001E3B8C118B8 System.Drawing.Bitmap
000001E3B8C118E0    2.3MB         000001E3B8E5F9A0 System.Text.RegularExpressions.Match
000001E3B8E5FA88    3.3MB         000001E3B91B2D40 System.Windows.Media.Imaging.BitmapSizeOptions
000001E3B926F210    2.6MB         000001E3B9505008 System.WeakReference
000001E3B95C0D50    1.6MB         000001E3B9761CE0 System.WeakReference
000001E3B98741F8    3.4MB         000001E3B9BDF1C8 System.Threading.ThreadPoolWorkQueue+WorkStealingQueue[]


0:000> !DumpObj /d 000001e3b91dead8
Name:        System.Windows.ContextLayoutManager+LayoutQueue+Request
MethodTable: 00007ffcf5b83288
EEClass:     00007ffcf5b5c7e0
Tracked Type: false
Size:        40(0x28) bytes
File:        C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\6.0.3\PresentationCore.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
00007ffcf3bf2b20  40031d3        8 ...Windows.UIElement  0 instance 0000000000000000 Target
00007ffcf5b83288  40031d4       10 ...youtQueue+Request  0 instance 000001e3b91deab0 Next
00007ffcf5b83288  40031d5       18 ...youtQueue+Request  0 instance 0000000000000000 Prev



0:000> !gcroot 000001e3b91dead8
HandleTable:
    000001E3AD931E58 (strong handle)
    -> 000001E3C5DACAE0 System.Object[]
    -> 000001E3AF847730 System.Collections.ArrayList
    -> 000001E3B5C08740 System.Object[]
    -> 000001E3B7DC0720 System.Windows.Media.MediaContext
    -> 000001E3B7DC0128 System.Windows.Threading.Dispatcher
    -> 000001E3B91D7830 System.EventHandler
    -> 000001E3B91D7808 System.Object[]
    -> 000001E3B91D77C8 System.EventHandler
    -> 000001E3B91D7758 System.Windows.ContextLayoutManager
    -> 000001E3B91DD340 System.Windows.ContextLayoutManager+InternalArrangeQueue
    -> 000001E3B91DEB28 System.Windows.ContextLayoutManager+LayoutQueue+Request
    -> 000001E3B91DEB00 System.Windows.ContextLayoutManager+LayoutQueue+Request
    -> 000001E3B91DEAD8 System.Windows.ContextLayoutManager+LayoutQueue+Request

Found 1 unique roots (run '!gcroot -all' to see all roots).

(@danmoseley @SamBent @singhashish-wpf @ThomasGoulet73 )

[wstaelens]

https://social.msdn.microsoft.com/Forums/en-US/home?category=&forum=windowsxps&filter=&sort=relevancedesc&brandIgnore=true&searchTerm=memory

I really hope we are not experiencing this from 2007...: https://social.msdn.microsoft.com/Forums/en-US/9a310989-23a8-497e-b3f2-b1f2d0883bbd/memory-leak-printing-xps-files-on-aspnet?forum=windowsxps

[wstaelens]

any updates @gurpreet-wpf ? @danmoseley ?

[arthurnardy]

Same issue with a png file exporter (oxyplot), which instantiates a LOT of :

• ContextLayoutManager+LayoutQueue+Request
• LayoutEventList+ListItem
• Dispatcher
• …etc…

It goes up, and up, and up -- and as it takes place in a test project, the memory is never released even between different tests when the runner runs multiple test in a row … On CI we need to restart the computer regularly to make sure the memory is freed. This is absolute non sense.

The impossibility to free the memory by hand is a absolute nightmare. This problem seems to be reported to Microsoft for at least a decade as I found out on internet and as @wstaelens has here documented.

Any news from the dev team ?

[hiteshkrmsft]

@wstaelens as this issue doesn’t have a minimal repro, I was not able to validate from my end if this fix fixes this issue. But by the look of it, the issue seems to be the same. Could you please validate and let us know if it works.

[wstaelens]

@hiteshwpfmsft hard to tell as this is 2 years old and happens in production. I guess when we will upgrade to .net 8/9 where these fixes will be added I hope it will be fixed.
Will take time for us to update and check again.

You may close this ticket and we will reopen/create a new one when we encounter it again in case it hasn't been fixed.

@hiteshwpfmsft this is more important as the STA requirement is a showstopper : #4000 (comment)

[hiteshkrmsft]

@wstaelens Understood; we’ll close this ticket for now. Feel free to reopen or create a new one if it reoccurs after your .NET 8/9 upgrade.

Thanks for the update!