CEF Future Direction

[magreenblatt]

Hi All,

CEF is over a year old now and the time has come to discuss where it should be heading in the future. As many of you know, CEF is based on the test_shell application that is part of the Chromium project. The test_shell application is used by Chromium as the testing platform for the Chromium WebKit API layer. As a result of its origins CEF has a single-process architecture and does not support tabs, extensions or other browser-provided user interface features.

The Chromium developers are in the process of moving the Chromium WebKit API testing framework into the WebKit source tree [1] [2]. As a necessary aspect of this up-streaming effort important parts of the test_shell application are being generalized and re-implemented in ways that are not always performance-oriented. For this and other reasons CEF, like test_shell, suffers from a performance standpoint relative to the Chromium browser application [3]. Further complicating future CEF development is the possibility that the test_shell application may disappear completely [4]. If test_shell disappears or becomes severely limited it will be increasingly difficult for us to identify how CEF needs to change in order to keep up with underlying Chromium changes.

Instead of keeping CEF based on the test_shell application I propose that we build the future version of CEF as a customization of the Chromium browser application itself. CEF currently has tools that allow it to automatically patch Chromium source files and GYP configuration files as part of the build process. This patch capability allows us to more intrusively modify the Chromium source tree while still supporting those changes independently. Furthermore, I have demonstrated in the past [5] that it is possible to embed the Chromium browser application window in a separate client application.

Below is a short list of what I believe to be the main pros and cons of the Chromium customization approach.

PROS
a. All existing Chromium browser capabilities, including the multi-process framework, tabs, history, bookmarks, i18n, extensions, etc, can be made available to CEF clients.
b. CEF will offer equivalent performance to the Chromium browser.
c. The CEF code footprint may decrease leading to the potential for decreased development/support requirements.
d. Most of the existing CEF usage and APIs could remain the same.

CONS
a. A significant initial development effort will be required.
b. The CEF distributable size and complexity will increase, though this increase could be mitigated by selectively excluding Chromium features from the build.
c. The concept of embedded NPAPI plugins will need to be reworked.

What do you think?

Regards,
Marshall

[1] http://trac.webkit.org/wiki/Chromium
[2] http://www.chromium.org/developers/desi ... ayouttests
[3] http://groups.google.com/group/chromium ... e2024550a6
[4] http://groups.google.com/group/chromium ... e5591c87ca
[5] http://codereview.chromium.org/10973/show

[wayne]

Hi Marshall,

Thanks for a great framework. I adopted CEF almost a year ago for our mobione project and have branched and extended it further for our specific needs, e.g WebKit4SWT. As you indicate Chromium continues to change in ways that ripple through our implementations which greatly increases our maintenance and innovation costs. We have similar ideas as you but recognize the large redevelopment costs. Thus we have held off and continue to operate and maintain our CEF-alt branch and custom chromium builds. We are interested in brainstorming what is possible and how we may be able to help with a next generation CEF2.

Regards,

Wayne
genuitec.com

[magreenblatt]

Hi Wayne,

My initial goal will be porting the existing CEF APIs to the new multi-process framework. I believe this can be accomplished by keeping the existing libcef DLL interface and adding a new IPC layer between it and the Chromium executable where the actual work will be performed. Beyond the capabilities that Chromium and CEF currently offer are there any particular features or capabilities that you would like to see available in the next version of CEF?

Regards,
Marshall

[heshiming]

Hi Marshall,

Thanks for putting so much effort into this project. I understand what you are saying. So the next CEF will spawn background processes and talk to them via IPC pretty much the same way chromium.exe itself does. It does sound promising compared to basing off test_shell. But how are you going to start? Chromium has lots of things like locale, theme, graphics, menus & dialogs. Are you going to split them out, or create a new one from scratch? I suppose the IPC protocol can change too, you still need to catch up with that every now and then.

Could you please illustrate the development task a bit further? I'm willing to assist where I can.

And what do you mean by reworking NPAPI plugin concept?

[magreenblatt]

It does sound promising compared to basing off test_shell. But how are you going to start?

1. Implement something similar to http://codereview.chromium.org/10973/show for Chromium trunk that allows the hosting of Chromium windows in separate applications.
2. Implement a replacement for the existing CEF DLL that interfaces with 1 using an IPC layer and supports the basic CEF functionality. The existing CEF interfaces may be modified slightly to support the concept of multiple tabs.
3. Expand 1 and 2 to support all existing CEF functionality.
4. Create a customizable of minimum build of 1 that removes unused and/or undesired functionality.

I'll likely complete steps 1 and 2 privately before making the first public release. By then I should have a clear idea of what's required for steps 3 and 4 and I'll provide guidance for contributions from the community.

Chromium has lots of things like locale, theme, graphics, menus & dialogs. Are you going to split them out, or create a new one from scratch?

I'd like to keep the changes to Chromium as minimal as possible. CEF will provide most of the existing Chromium capabilities to host applications unchanged. Anything that we don't end up using can be excluded from the build in step 4.

And what do you mean by reworking NPAPI plugin concept?

CEF currently supports the concept of "embedded NPAPI plugins" where the host application provides CEF with pointers to an NPAPI plugin implementation that is part of the host application. This approach will no longer be possible once the CEF browser is hosted in a separate process. CEF will provide interfaces that simplify IPC communication to help mitigate the loss of capability.

[JoeAndrieu]

I would like to be able to use CEF in a transparent window, with transparent HTML showing through to the rest of the desktop.

Windows does this with UpdateLayeredWindow() instead of using Paint().

Could we use CEF as part of a layered window (WS_EX_LAYERED)? Looking at the examples, I don't think this would actually work, although I will apologize for not having actually tried it. It is in my queue however, because my app would very much like to render HTML in a non-square window using transparency in the HTML to control the visual shape.

If there isn't a way to do that yet, I would love to see it in a future CEF.

[magreenblatt]

I would like to be able to use CEF in a transparent window, with transparent HTML showing through to the rest of the desktop.

Do the HTML elements and desktop icons need to be functional at the same time? If so, then you will need to do some creative management of UI events so that they end up at the correct window, and then do some type of blending so that everything draws correctly. If only one or the other will be functional then take a screen shot of the non-functional one and use that as the background for the other.

[JoeAndrieu]

Yes, I would like them to be both be active at the same time.

For example, I would like to be able to use a pop-up HTML window as a custom pie menu (written in HTML). See http://www.piemenu.com for examples of what I mean.

I've implemented a custom Win32 window which handles similar pie menu functionality with GDI+ function calls, heavily relying on WS_EX_LAYERED and UpdateLayeredWindow. The event handling and blending were easy (once I figured out how to use UpdateLayeredWindow). Events automatically passed through when they occur over "transparent" parts of the window and Windows takes care of painting the buffer to the screen, blending the transparent parts appropriately (including full alpha blending if used). In short, Windows takes care of the hard parts if you restructure your OnPaint-based approach to rendering to use UpdateLayeredWindow instead. This is essentially just a double-buffered mechanism that passes events through the transparent areas. This is the "standard" way to make non-rectangular windows in Windows.

I do not know how to do anything similar on Mac or Linux, so doing this in a cross-platform way might be tricky.

I'd be interested in helping on this if there's someone who understands the current OnPaint-based rendering architecture in Chromium and wants to take the lead. I'm guessing this may be more of a Chromium modification than CEF, but I think the justification for it is probably easier here, since CEF is more likely to be used outside of a normal browser context where this sort of transparency is more likely.

[kevingadd]

I would like to be able to use CEF in a transparent window, with transparent HTML showing through to the rest of the desktop.

Windows does this with UpdateLayeredWindow() instead of using Paint().

Could we use CEF as part of a layered window (WS_EX_LAYERED)? Looking at the examples, I don't think this would actually work, although I will apologize for not having actually tried it. It is in my queue however, because my app would very much like to render HTML in a non-square window using transparency in the HTML to control the visual shape.

If there isn't a way to do that yet, I would love to see it in a future CEF.

I did some experimenting with this in a different Chromium embedding framework, but my findings are based on how Chromium actually implements transparency, so they might be helpful:

Chromium understands the concept of a 'transparent' document, in which case the renderer will produce an alpha channel for the document when painting it. Combine this with the ability to paint to an offscreen buffer instead of a window, and you could feasibly combine this with UpdateLayeredWindow to have transparent, interactive HTML windows. However: On Win32, there are known limitations - in particular, when GDI renders text and themed UI elements (like buttons), it wipes out the contents of the alpha channel on the target surface. As a result, if you try and create a transparent window using HTML in Chromium right now, any text or themed UI shows up as wholly transparent. There are ways to trick Chromium into preserving the alpha channel, and I've tested those out successfully to composite transparent HTML over a Win32 UI (though not using UpdateLayeredWindow) - for example, if you set the opacity of the element containing the text to 0.99, the renderer will do some tricks to preserve opacity data (but the text will look funky).

There are ways you could work around this, though; skia and chromium are robust enough that you really just need to dive into the renderer internals and fiddle with things to get the result you want.

[Phil]

Hi Marshall,

Thanks again. this is a really interesting project.

I am just getting started, but I hope to investigate the possibility of rendering to a texture also, so I can use CEF to display on top of an OpenGl window.
I think that your suggestion to look into the printing functionality sounds like a great place to start.

My question however, is about the 'Webkit Inspector'. I would very much like to be able to have this very powerful feature in our app, and I know that test_shell supports it.
Would it be difficult to enable CEF to pop open the Inspector? Can it already do this? I haven't found any reference to the Inspector.

Thanks
Phil