accelerated video playback with pepper - possible?

[richeakin]

Hi all,

Our team is using CEF in a windows application that will require very high resolution video and chromium's current decoders aren't quite getting us the consistent playback that we need. We're looking at alternatives and hoping to get some feedback / advice from this community before continuing down any one path.

One approach we've been exploring is to add a custom MIME handler / plug-in that uses Direct Show / Direct3D 11 to decode and render the video with hardware acceleration. It looks like the 'official' way to add plug-ins is to use Native Client, however I've read (in this native-client-discuss post) that, we don't have access to the HWND's so this won't be possible. Can we get around this?

This relatively old pepper / PPAPI documentation suggests that if we declare the plug-in as 'trusted' then we can get around the sandbox, however I don't really know how to utilize this or if / how it'd allow us to get the HWND handle we need.

Can anyone advise on how to connect the dots here? Seems there are a few magic raw pointers and handles floating around in the pepper api, but I can't make much of them yet. Our other options don't have the nice and tight integration that this route would have (we'd get to use keep using html5 video tags). Although I'm open to others opinions if they know a decent route for getting accelerated video playback on Windows.

thanks,
Rich

[magreenblatt]

Good hardware accelerated video playback is possible using a windowed NPAPI plugin, but of course NPAPI is going away some time in 2014. That leaves two future-proof browser technologies -- Pepper plugins and HTML5 video.

Unfortunately Pepper is not in a great state for supporting hardware accelerated video decoding/playback at this time. Hardware acceleration is accomplished using DXVA on Windows (Vista+) and VDA on OS X. Pepper does not currently support VDA (see bug 133828) so you're out of luck on OS X. DXVA is supported on Windows but you can't access it directly from the plugin process due to sandbox restrictions and the GPU architecture design -- instead you need to use the Pepper video decoder API. This API only supports hardware decoding currently so you'll need to provide your own alternative implementation for software decoding. Even after you've implemented all that your playback will likely be mediocre from a performance and resource usage standpoint as indicated by the outstanding bugs.

In conclusion, your only option at this time is HTML5 video if you want to use existing browser technologies. If that isn't sufficient and you need overlays or CSS effects (which both Pepper and HTML5 video support) then you're pretty much out of luck.

If you're willing to accept the lack of overlays/effects (meaning the video window is always rectangular and on top of other content) then you can create your own media player window. Your application can then position this window on top of the browser view -- for example, using JavaScript to adjust the positioning relative to an underlying div element. This is very similar to how windowed NPAPI plugins work, except in this case your application is wholly responsible for creating, positioning and destroying the window. There will probably be a slight noticeable delay when positioning the window relative to the underlying web content, but that may be an acceptable trade-off for high-quality/low-resource-usage video playback.

[magreenblatt]

In case my response sounds overly negative about Pepper (can you tell I'm a bit frustrated by the state of video playback? ) I'll remind everyone that Chromium is open source and patches to make video playback awesome would likely be warmly welcomed by the Chromium team.

[richeakin]

Thanks for the detailed information, very enlightining on the current situation with video.

Hm, it seems like Pepper may be a decent route in the future (once people like Netflix are forced to use it, and then google finishes it ... but at the moment it seems like quite a few hoops just to be stuck with similar issues as HTMLVideoElement's.

We're exploring overlayed videos at the moment to see what sort of performance we find there. Actually, we'll most likely have (very large) video playing underneath the web content, so the tricky part will be seeing how compositing is effected...

Hopefully someone finds the time / resources to improve the video performance situation... this is all relatively young so my bet is it'll happen.

cheers,
Rich