Out of memory when printing PDFs

Having problems with building or using the JCEF Java binding? Ask your questions here.

Re: Out of memory when printing PDFs

Postby Phylanx » Fri Jan 13, 2017 8:18 am

ok, we'll give it a try and post the result :-)
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby Phylanx » Mon Jan 16, 2017 7:49 am

Hi!

In the meanwhile we had some further analysis done, here is the result.

The process would have enough native memory (594.880K), but the last free block of memory is only 119.040K big.
That's because 1024m are already reserved for the JVM, so ordering of 139 MB memory fails.
See the Xmx1024_beforePrint.jpg

The other screenshot (Xmx256_whilePrint.jpg) has much more free native memory (1.061.056K) and much bigger free memory blocks (353.856K).
So the allocation of 139MB works.
In this screenshot we also see that the 139MB of memory are reserved for writing a spool file (SPL66B0.tmp).

Unfortunatly the VMMap file (*.mmp) is too big (33 MB, 7zip 2.1 MB) to be attached.
If you need it, I would send it per mail.


We were also able to get a native debug stack.
The stack shows, that the code currently spools a page, rendering a printed page.
I hope the stack is helpful for you:

libcef.dll!logging::LogMessage::~LogMessage() Zeile 735 C++
libcef.dll!base::`anonymous namespace'::OnNoMemory(unsigned int size) Zeile 41 C++
libcef.dll!malloc(unsigned int size) Zeile 119 C++
[Externer Code]
> libcef.dll!printing::`anonymous namespace'::LazyEmf::LoadEmf(printing::Emf * emf) Zeile 277 C++
libcef.dll!printing::`anonymous namespace'::LazyEmf::SafePlayback(HDC__ * hdc) Zeile 252 C++
libcef.dll!printing::PrintedDocument::RenderPrintedPage(const printing::PrintedPage & page, HDC__ * context) Zeile 72 C++
libcef.dll!printing::PrintJobWorker::SpoolPage(printing::PrintedPage * page) Zeile 410 C++
libcef.dll!printing::PrintJobWorker::OnNewPage() Zeile 325 C++
libcef.dll!printing::PrintJobWorker::StartPrinting(printing::PrintedDocument * new_document) Zeile 279 C++
libcef.dll!base::internal::Invoker<base::IndexSequence<0,1>,base::internal::BindState<base::internal::RunnableAdapter<void (__thiscall IPC::internal::MessagePipeReader::*)(unsigned int)>,void __cdecl(IPC::internal::MessagePipeReader *,unsigned int),base::internal::UnretainedWrapper<IPC::internal::MessagePipeReader>,unsigned int const &>,base::internal::InvokeHelper<0,void,base::internal::RunnableAdapter<void (__thiscall IPC::internal::MessagePipeReader::*)(unsigned int)> >,void __cdecl(void)>::Run(base::internal::BindStateBase * base) Zeile 365 C++
libcef.dll!printing::`anonymous namespace'::HoldRefCallback(const scoped_refptr<printing::PrintJobWorkerOwner> & owner, const base::Callback<void __cdecl(void),1> & callback) Zeile 45 C++
libcef.dll!base::internal::RunnableAdapter<void (__cdecl*)(scoped_refptr<printing::PrintJobWorkerOwner> const &,base::Callback<void __cdecl(void),1> const &)>::Run<scoped_refptr<printing::PrintJob> const &,base::Callback<void __cdecl(void),1> const &>(const scoped_refptr<printing::PrintJob> & <args_0>, const base::Callback<void __cdecl(void),1> & <args_1>) Zeile 159 C++
libcef.dll!base::internal::Invoker<base::IndexSequence<0,1>,base::internal::BindState<base::internal::RunnableAdapter<void (__cdecl*)(scoped_refptr<printing::PrintJobWorkerOwner> const &,base::Callback<void __cdecl(void),1> const &)>,void __cdecl(scoped_refptr<printing::PrintJobWorkerOwner> const &,base::Callback<void __cdecl(void),1> const &),scoped_refptr<printing::PrintJob>,base::Callback<void __cdecl(void),1> >,base::internal::InvokeHelper<0,void,base::internal::RunnableAdapter<void (__cdecl*)(scoped_refptr<printing::PrintJobWorkerOwner> const &,base::Callback<void __cdecl(void),1> const &)> >,void __cdecl(void)>::Run(base::internal::BindStateBase * base) Zeile 365 C++
libcef.dll!base::debug::TaskAnnotator::RunTask(const char * queue_function, const base::PendingTask & pending_task) Zeile 51 C++
libcef.dll!base::MessageLoop::RunTask(const base::PendingTask & pending_task) Zeile 474 C++
libcef.dll!base::MessageLoop::DoWork() Zeile 595 C++
libcef.dll!base::MessagePumpDefault::Run(base::MessagePump::Delegate * delegate) Zeile 34 C++
libcef.dll!base::RunLoop::Run() Zeile 36 C++
libcef.dll!base::Thread::Run(base::MessageLoop * message_loop) Zeile 202 C++
libcef.dll!base::Thread::ThreadMain() Zeile 257 C++
libcef.dll!base::`anonymous namespace'::ThreadFunc(void * params) Zeile 86 C++
[Externer Code]
[Unten angegebene Rahmen sind möglicherweise nicht korrekt und/oder fehlen, keine Symbole geladen für kernel32.dll]
Attachments
Xmx256_whilePrint.jpg
Xmx256_whilePrint.jpg (482.77 KiB) Viewed 23580 times
Xmx1024_beforePrint.jpg
Xmx1024_beforePrint.jpg (487.48 KiB) Viewed 23580 times
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby magreenblatt » Tue Jan 17, 2017 11:21 am

Thanks for posting the call stack. It shows that the issue is in Chromium code. You can file a bug at http://crbug.com/new but they're likely to mark it as WontFix since the problem doesn't reproduce with Google Chrome.

I think your options at this point are to reduce the JVM reserved memory or use a 64-bit JVM.
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

Re: Out of memory when printing PDFs

Postby Phylanx » Wed Jan 18, 2017 4:08 am

Issue created for Chromium: https://bugs.chromium.org/p/chromium/is ... ?id=682156
Feel free to add any missing informations on it, you have a more detailed insight on the integration of Chromium/CEF/JCEF.

Maybe the issue itself is not caused by JCEF, but the architecture of JCEF enforces it.
For other operating systems I don't know, but for Windows it is java specific that reserving more memory for the JVM means less memory for the native part used by Chromium.
So in my opinion it is a problem that JCEF 32 bit has.
Isn't it possible to adapt the process model? If this logic would happen in a separate jcef_helper process, JCEF wouldn't have that problem.
Is there anything we can do about that?
This is really a severe problem for our customers and for us.

The two mentioned options are unfortunatly no options for us.
Upgrading to a 64 bit JVM cannot be done, because we also need to use other DLLs that are only available in 32 bit.
Workarounds with external processes,... are already estimated for us with over 50 days of implementation work, not to mention the risks.
Less Memory reserved for JVM is also difficult.
How much is less memory?
We also have seen this issue with Xmx768m and that's the minimum of memory needed for our application.
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby magreenblatt » Wed Jan 18, 2017 10:53 am

Thanks for creating the issue. We'll see what the Chromium devs say.
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

Re: Out of memory when printing PDFs

Postby Phylanx » Thu Jan 19, 2017 8:13 am

We did further analysis and found a TODO comment that shows exactly the problem we have:
https://cs.chromium.org/chromium/src/th ... dr=C&l=668
Our PDF has a masked image in it and the spool file is blown up extremly because of that.
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby Phylanx » Fri Jan 20, 2017 10:07 am

New comment on the chromium issue:

Comment 6 by rbpotter@chromium.org, Today (22 hours ago)
Cc: rbpotter@chromium.org
Labels: -Needs-Feedback
Status: Available

Since this PDF contains masks you are correct that that is likely the reason for the memory issue. We rasterize PDF pages containing image masks in order to get them to print correctly (see http://crbug.com/482206 and http://crbug.com/576175 for a few examples of problems that came up before we started doing this). We do plan to look into ways we could do this better: see http://crbug.com/674771 which was filed when the comment you reference was added.


Maybe we are lucky and get a fix for it! :-)
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby Phylanx » Mon Jan 23, 2017 7:04 am

After 3 days of trial and error (and now nearly 6h of "automate-git.py" script execution time), I'm able to build CEF without compile errors/link problems or other problems like that.

Until we get a solution for our PDF problems by Chromium, we will have to make a quick and dirty fix ourselves.
Is there a possibility to do a faster compile of parts of the code (specially pdfium/printing/...) than deleting the whole out folder and recompile and wait for 6 hours again?

When building the CEF code via "automate-git.py", is the building of chromium/PDFium also included?
Would changes in the PDFium code be included in the resulting binaries?

I'm asking because we have strange line number incompatibilities with the sources loaded in Visual Studio for debugging...
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

Re: Out of memory when printing PDFs

Postby magreenblatt » Mon Jan 23, 2017 11:47 am

The automate-git.py script can perform incremental builds of only the files that have changed. What arguments are you passing currently?
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

Re: Out of memory when printing PDFs

Postby Phylanx » Tue Jan 24, 2017 2:12 am

I use a small batch file that redirects the output into a logfile, but finally this statement is executed:

python D:\Git\cef\automate\automate-git.py --download-dir=D:\Git\cef\2704 --branch=2704 --depot-tools-dir=D:\Git\cef\2704\depot_tools

If I do not delete the out folder, I always get following output (even if I changed sources in chromium code, I'm not exactly sure if I changed CEF sources):

Building version 2704; logfile: CEFBuild_2704_20170124_08475343.log
--> Download Directory: D:\Git\cef\2704
--> Depot Tools Directory: D:\Git\cef\2704\depot_tools
--> Updating depot_tools
-------- Running "update_depot_tools.bat" in "D:\Git\cef\2704\depot_tools"...
-------- Running "D:\Git\cef\2704\depot_tools\git.bat config --get remote.origin.url" in "D:\Git\cef\2704\cef"...
--> CEF Branch: 2704
--> CEF URL: https://bitbucket.org/chromiumembedded/cef.git
--> CEF Source Directory: D:\Git\cef\2704\cef
-------- Running "D:\Git\cef\2704\depot_tools\git.bat rev-parse HEAD" in "D:\Git\cef\2704\cef"...
-------- Running "D:\Git\cef\2704\depot_tools\git.bat fetch" in "D:\Git\cef\2704\cef"...
-------- Running "D:\Git\cef\2704\depot_tools\git.bat rev-parse origin/2704" in "D:\Git\cef\2704\cef"...
--> CEF Current Checkout: ec3e9ed7fecc0d9f37a96951ba1bb26fd4f64dc7
--> CEF Desired Checkout: ec3e9ed7fecc0d9f37a96951ba1bb26fd4f64dc7 (origin/2704)
--> CEF Output Directory: D:\Git\cef\2704\out_2704
-------- Running "D:\Git\cef\2704\depot_tools\git.bat config --get remote.origin.url" in "D:\Git\cef\2704\chromium\src"...
--> Chromium URL: https://chromium.googlesource.com/chromium/src.git
-------- Running "D:\Git\cef\2704\depot_tools\git.bat rev-parse HEAD" in "D:\Git\cef\2704\chromium\src"...
-------- Running "D:\Git\cef\2704\depot_tools\git.bat rev-parse refs/tags/51.0.2704.103" in "D:\Git\cef\2704\chromium\src"...
--> Chromium Current Checkout: c4febd5ae385dee28312cfb1ce0c653b0919044a
--> Chromium Desired Checkout: c4febd5ae385dee28312cfb1ce0c653b0919044a (refs/tags/51.0.2704.103)
--> Not building. The source hashes have not changed and the output folder "D:\Git\cef\2704\chromium\src\out" already exists


Which sources have to be modified to get a continuous build?
Phylanx
Expert
 
Posts: 201
Joined: Thu Aug 11, 2016 8:17 am

PreviousNext

Return to JCEF Forum

Who is online

Users browsing this forum: No registered users and 15 guests