Support for screen sharing in CEF

Having problems with building or using CEF's C/C++ APIs? This forum is here to help. Please do not post bug reports or feature requests here.

Re: Support for screen sharing in CEF

Postby linuxcef9 » Tue Aug 20, 2019 5:13 pm

callum wrote:Those WebRTC demos are very neat - surprised to see they appear to work in Chrome 76 without any extensions installed locally but not across the wider internet - is that expected? Can they be made to work remotely too?


What do you mean by 'made to work remotely' ?

Also, here's another observation:

With chrome 76,

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --auto-select-desktop-capture-source="Some native window" https://webrtc.github.io/samples/src/co ... playmedia/

works fine. I don't see any prompts and I see recording of "Some native window".

However,

cefclient.exe --enable-media-stream --auto-select-desktop-capture-source="Some native window" --url=https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia/

does NOT work with cef_binary_76.1.9+g2cf916e+chromium-76.0.3809.87_windows64_client openspotify build. I can see uncaught exception in console logs and there's no recording of Some native window.

Also, adding --use-fake-ui-for-media-stream to above command line doesn't work as expected because it records entire primary screen instead of just the application.

Any ideas as to why auto-select-desktop-capture-source flag doesn't work with cefclient?
linuxcef9
Expert
 
Posts: 143
Joined: Tue Nov 06, 2018 3:08 pm

Re: Support for screen sharing in CEF

Postby callum » Tue Aug 20, 2019 5:41 pm

linuxcef9 wrote:
callum wrote:Those WebRTC demos are very neat - surprised to see they appear to work in Chrome 76 without any extensions installed locally but not across the wider internet - is that expected? Can they be made to work remotely too?

What do you mean by 'made to work remotely' ?


I create a shared screen session on my machine and send the numeric ID to a friend in another city who opens the same demo and uses the ID I sent them to view my screen.
callum
Expert
 
Posts: 323
Joined: Mon Feb 23, 2015 6:19 pm

Re: Support for screen sharing in CEF

Postby Czarek » Wed Aug 21, 2019 2:00 am

The --auto-select-desktop-capture-source flag may work when used along with the --use-views flag, test to see.
Maintainer of the CEF Python, PHP Desktop and CEF C API projects. My LinkedIn.
User avatar
Czarek
Virtuoso
 
Posts: 1927
Joined: Sun Nov 06, 2011 2:12 am

Re: Support for screen sharing in CEF

Postby JasonLP » Wed Aug 21, 2019 10:05 am

linuxcef9 wrote:@jasonLP, how did you share specific screen with getUserMedia?

Also, even I'd like to know how we can share any other screen besides default screen via getDisplayMedia.



I had to write my own screen and window picker, use the result from that to pass into getUserMedia(), which meant changing the flow of the javascript when run in CEF.

It would be nice if with getDisplayMedia there were a hook where we'd be called to show the screen/window picker or just return the device id so the javascript could remain unchanged.
JasonLP
Mentor
 
Posts: 98
Joined: Fri Jul 25, 2014 10:10 am

Re: Support for screen sharing in CEF

Postby linuxcef9 » Wed Aug 21, 2019 12:24 pm

The --auto-select-desktop-capture-source flag may work when used along with the --use-views flag, test to see.


It doesn't work unfortunately.

cefclient.exe --enable-media-stream --use-views --auto-select-desktop-capture-source="Some native window" --url=https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia/

that's the command line I used but it gives same errors as before.
Also, I don't want to use the view framework as my application creates the top-level window.

I had to write my own screen and window picker, use the result from that to pass into getUserMedia(), which meant changing the flow of the javascript when run in CEF.

It would be nice if with getDisplayMedia there were a hook where we'd be called to show the screen/window picker or just return the device id so the javascript could remain unchanged.


I see. Looks like you did something like : http://arkenthera.github.io/blog/Enabli ... Media-API/
Btw, if I understand correctly, your flow of application would've been :
1) Let user select a button to start screen sharing
2) You show your screen and window picker
3) User selects the screen/window and you pass the value to getUserMedia in chromeMediaSourceId parameter

So I am guessing your logic would work only for website/app that you write since you can control when to invoke getUserMedia?

I agree with the hook idea for getDisplayMedia but the spec explicitly disallows passing device id afaik.

https://www.w3.org/TR/screen-capture/#d ... dentifiers mentions "Display capture sources must not be selected with the deviceId constraint, since this would allow applications to influence selection"

(Edited to include link to spec)
linuxcef9
Expert
 
Posts: 143
Joined: Tue Nov 06, 2018 3:08 pm

Re: Support for screen sharing in CEF

Postby JasonLP » Wed Aug 21, 2019 2:07 pm

linuxcef9 wrote:So I am guessing your logic would work only for website/app that you write since you can control when to invoke getUserMedia?

That is exactly correct. We have a website that also runs in the context of our CEF application for additional features, and the javascript side knows what context it's running in.

We also have a requirement when run in desktop mode that the screen sharing can happen without any user interaction and can automatically pick which screen or window to share without showing any UI.


I agree with the hook idea for getDisplayMedia but the spec explicitly disallows passing device id afaik.

https://www.w3.org/TR/screen-capture/#d ... dentifiers mentions "Display capture sources must not be selected with the deviceId constraint, since this would allow applications to influence selection"

(Edited to include link to spec)

This is a shame. But at some point Chrome is presenting a picker and passing the result of that to the rest of the code. Wondering if there is some mechanism to hook into that and do our own thing there.

Until then, as long as getUserMedia() is not deprecated I guess we can continue to use that.
JasonLP
Mentor
 
Posts: 98
Joined: Fri Jul 25, 2014 10:10 am

Re: Support for screen sharing in CEF

Postby linuxcef9 » Wed Aug 21, 2019 2:58 pm

This is a shame. But at some point Chrome is presenting a picker and passing the result of that to the rest of the code. Wondering if there is some mechanism to hook into that and do our own thing there.


Yeah. But marshall and maybe czarek are in better position to answer that.

Until then, as long as getUserMedia() is not deprecated I guess we can continue to use that.


Unfortunately, that will not work on sites whose javascript we don't control for e.g the link I used for testing.
Also, the --auto-select-desktop-capture-source works fine with chrome and so I thought it should work fine with CEF too but something might be missing.
If that works, it'll still be an improvement over current behvior.
linuxcef9
Expert
 
Posts: 143
Joined: Tue Nov 06, 2018 3:08 pm

Re: Support for screen sharing in CEF

Postby magreenblatt » Wed Aug 21, 2019 4:14 pm

I haven’t looked at this particular code in depth, but in general we can’t use Chrome’s dialogs in CEF due to the dependency on Chrome-specific code or frameworks (Views, GTK3, etc). Additional callbacks to expose functionality that clients can then implement may be accepted if submitted as a PR.
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

Re: Support for screen sharing in CEF

Postby linuxcef9 » Thu Aug 22, 2019 12:26 pm

I haven’t looked at this particular code in depth, but in general we can’t use Chrome’s dialogs in CEF due to the dependency on Chrome-specific code or frameworks (Views, GTK3, etc). Additional callbacks to expose functionality that clients can then implement may be accepted if submitted as a PR.


Thanks marshall for the explanation. Keeping aside the dialog part, do you've any idea what's preventing --auto-select-desktop-capture-source flag from working in CEF ? With that flag, we can specify a window name or "Entire Screen" and getDisplayMedia will start capturing the same.
That switch works fine with chrome.
linuxcef9
Expert
 
Posts: 143
Joined: Tue Nov 06, 2018 3:08 pm

Re: Support for screen sharing in CEF

Postby magreenblatt » Thu Aug 22, 2019 1:03 pm

linuxcef9 wrote:
I haven’t looked at this particular code in depth, but in general we can’t use Chrome’s dialogs in CEF due to the dependency on Chrome-specific code or frameworks (Views, GTK3, etc). Additional callbacks to expose functionality that clients can then implement may be accepted if submitted as a PR.


Thanks marshall for the explanation. Keeping aside the dialog part, do you've any idea what's preventing --auto-select-desktop-capture-source flag from working in CEF ? With that flag, we can specify a window name or "Entire Screen" and getDisplayMedia will start capturing the same.
That switch works fine with chrome.

Support for that flag is implemented in the Chrome layer, so it's not available to CEF by default: https://cs.chromium.org/search/?q=kAuto ... um&type=cs
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

PreviousNext

Return to Support Forum

Who is online

Users browsing this forum: No registered users and 36 guests