Off-screen rendering 200-1800 FPS

[Alex]

Hi,

This one is tricky. I am aware this is not the intended use case for a browser.

I am looking to produce overlay using CEF out of real-time, as fast as performance allows and still utilize Javascript and HTML to produce overlay and do external composition on top of CEF's composited image.

Ideally I would like to invoke JS to prepare next frame then cause render to happen, extract image and repeat this cycle as fast as hardware allows.

Does anyone know some methods of speeding up the time for chromium&CEF?

Is it possible to request render output by API call? This looks complicated by several processes doing graphics operations, maybe there is a way to turn off process spawning and loose some performance.

Thanks,
Alex

[KatrinaS]

I guess you could render 'whatever' to a Canvas element and grab its content as a PNG on completion (but don't forget that whatever is returned may well be transparent, but that can be handy for compositing), and of course the next problem is getting the Data out of the Browser but depending on the language that's fairly straightforward

[Alex]

Hi KatrinaS, nice idea!

Looking at [toBlob()](https://developer.mozilla.org/en-US/doc ... ent/toBlob) I would expect following steps to take place:
- queue image capture event on specified canvas
- wait for next render cycle to happen
- capture raw image from canvas after render cycle
- encode raw image using specified quality
- queue JS callback passing captured BLOB (stored in ram or in file)

If this is correct then canvas capture pace is also limited by chromium render pace.

[Alex]

Update: found commandline params `--disable-frame-rate-limit --disable-gpu-vsync`.

`--disable-frame-rate-limit` breaks latest stable CEF render This post viewtopic.php?f=6&t=19466 seem to be related.

[KatrinaS]

If you want, and there are numerous ways, set up a custom protocol handler (sorry, 'scheme handler'), i.e. 'data://', when the Canvas has finished its loop of 'whatever' get it to (effectively) Post the Data to the handler using JavaScript in Base64 ('canvas.toDataURL();'), you decode, do the 'whatever', then bing bang bong rinse and repeat and then everything is (should be) synchronised (within reason). You need to set it up and stick to a reasonable framerate, the refresh rate of the Monitor'll do,, unless you have a RTX4090XT 3000i SUX

Anyway have fun with it

Oh. PNG's only have one render quality, and they're free of artefacts, JPG can get pretty mashie really and they have no decent Alpha channel available for further compositing. Edit, just did a side by side comparison, at the 'default' quality JPG is in deed mashie when compared to the same image in PNG

Oh. So basically the 'scheme handler' (or whichever is the fastest route you decide) does 'whatever' and dumps the result back to the Canvas, the Canvas does 'whatever' and dumps the result back to the scheme hander (or whatever), which dumps 'whatever' back to the Canvas, which dumps it back to the ... ad nauseum. And frames are generated as fast as the Canvas renders and the scheme handler (or whichever mechanism you choose) can pick up and process the result back and trigger the next iteration. I guess it's at least a decent exercise in seeing / measuring just how fast you can get out of this tight loop

[Alex]

Update: cef_binary_85.3.13+gcd6cbe0+chromium-85.0.4183.121_linux64.tar.bz2 with `--disable-frame-rate-limit --disable-gpu-vsync` gets up to 1100 FPS on https://testufo.com/

[KatrinaS]

Seems adequate