Contribution: Wait for browser finished loading function

Having problems with building or using the CefSharp .NET binding? Ask your CEF-related questions here. Please ask general usage questions on StackOverflow.

Moderator: amaitland

Contribution: Wait for browser finished loading function

Postby andla71 » Sat Oct 31, 2020 12:40 pm

Hi everyone my first post here.
I had been struggling with making CEF wait until browser was finished.
I'm creating my own macro tool and I just wanted a simple wait.
I will contribute a solution that worked like a charm for me. (but it needs further testing)
Be prepared because it's gonna be a bumpy ride. Here is the code:
Code: Select all
     public bool WaitForBrowser(ChromiumWebBrowser chrome, int timeoutcheckload)
        {
            bool didload = false;


            var frames = chrome.GetBrowser().GetFrameNames();

            bool hasmorecontent = false;
            int totalsize = 0;
            Stopwatch stopWatch = new Stopwatch();
            //stopWatch.Start();
            bool please_continue = true;
            do
            {


                do
                {
                    IFrame iframe = null;
                    string contentloading = "";
                    bool framesuccess = false;
                    foreach (string frame in frames)
                    {

                            iframe = chrome.GetBrowser().GetFrame(frame);


                        if (iframe != null)
                        {
                            framesuccess = true;
                            var task = iframe.EvaluateScriptAsync(@"document.getElementsByTagName ('html')[0].innerHTML").ContinueWith(x =>
                            {
                                var response = x.Result;
                                if (response.Success)
                                {
                                    contentloading += response.Result.ToString();
                                }
                            });
                            task.Wait(100);
                        }
                       
                    }
                    SleepMillisec(100);
                    if (totalsize < contentloading.Length)
                    {
                        totalsize = contentloading.Length;
                        hasmorecontent = true;
                        stopWatch.Reset();
                    }
                    else
                    {
                        hasmorecontent = false;
                        if(!stopWatch.IsRunning)
                        {
                            stopWatch.Start();
                        }
                    }

                } while (hasmorecontent);

                TimeSpan ts = stopWatch.Elapsed;
             
                if (ts.TotalMilliseconds > timeoutcheckload)
                {
                    please_continue = false;
                    didload = true;
                    stopWatch.Stop();
                }
            }
            while (please_continue);


            return didload;
        }

Updated and fixed.
What it does is compare the length of all frame html until the length stop increasing. Then wait timeout in case there are more.
Simple as that.


This is my old code that just simply sucks:
Code: Select all
       //public bool WaitForBrowser(ChromiumWebBrowser chrome, int timeoutcheckload)
        //{
        //    try
        //    {
        //        bool didload = false;
        //        using (var waitHandle = new System.Threading.AutoResetEvent(false))
        //        {

        //            EventHandler<FrameLoadStartEventArgs> frameloadstart = null;
        //            EventHandler<LoadingStateChangedEventArgs> loadingHandler = null;
        //            bool isloading = false;
        //            frameloadstart = (sender, e) =>
        //            {
        //                isloading = true;
        //            };
        //            loadingHandler = (sender, e) =>
        //            {
        //                try
        //                {


        //                    if (!e.IsLoading)
        //                    {
        //                        if (chrome != null)
        //                        {
        //                            chrome.LoadingStateChanged -= loadingHandler;
        //                            chrome.FrameLoadStart -= frameloadstart;

        //                        }
        //                        waitHandle.Set();
        //                        isloading = false;
        //                    }
        //                    else
        //                    {
        //                        isloading = true;
        //                    }
        //                }
        //                catch (Exception ex)
        //                {

        //                }
        //            };

        //            chrome.LoadingStateChanged += loadingHandler;
        //            chrome.FrameLoadStart += frameloadstart;
        //            waitHandle.WaitOne(timeoutcheckload);

        //            if (isloading)
        //            {

        //                int timeout2 = timeoutcheckload;
        //                if(timeout2<5000)
        //                {
        //                    timeout2 = 200000;
        //                }
        //                // System.Threading.Thread.Sleep(5000);
        //                didload = true;
        //                waitHandle.WaitOne(timeout2);
        //            }
        //            else
        //            {
        //                chrome.LoadingStateChanged -= loadingHandler;
        //                chrome.FrameLoadStart -= frameloadstart;
        //            }
        //        }

        //        return didload;
        //    }
        //    catch (Exception ex)
        //    {

        //        CheckLDM("WaitForBrowser (not a cmd): Exception'" + ex.Message + "' ");
        //    }
        //    return false;

        //}


I found a part of it online and maybe I did somehting wrong and it works fine if pages load fast but messes up somehow when they are slower.
after adding a waitHandle.close() it just messed things up even more and I decided to just go with it and check if content is loading.

Please let me know if there is something in my function that will kick my behind so to say.
Appreciate your input.
Last edited by andla71 on Sat Oct 31, 2020 4:12 pm, edited 1 time in total.
andla71
Newbie
 
Posts: 6
Joined: Sat Oct 31, 2020 12:19 pm

Re: Contribution: Wait for browser finished loading function

Postby magreenblatt » Sat Oct 31, 2020 3:11 pm

Many websites load data or resources asynchronously. The definition of “done” depends on the website, and requires a website-specific signal (like calling a JS function).
magreenblatt
Site Admin
 
Posts: 10939
Joined: Fri May 29, 2009 6:57 pm

Re: Contribution: Wait for browser finished loading function

Postby andla71 » Sat Oct 31, 2020 4:17 pm

magreenblatt wrote:Many websites load data or resources asynchronously. The definition of “done” depends on the website, and requires a website-specific signal (like calling a JS function).

Sure but how do you, as a human browsing the internet, determine when the webpage has finished loading?
I myself check when things stop moving and loading.
I'm not aware if any javascript is loading anything.
I can only see what is in front of me.
What better way is there then check if html starts to build.
andla71
Newbie
 
Posts: 6
Joined: Sat Oct 31, 2020 12:19 pm

Re: Contribution: Wait for browser finished loading function

Postby amaitland » Sat Oct 31, 2020 4:45 pm

CEF/Chromium are designed to be asynchronous, executing your script in a loop with a delay largely ignores this. Chromium uses multiple threads/processes and sitting there polling isn't great. You are attempting to return innerhtml, which depending on the page could be very large.

At a minimum you should wait for LoadingStateChange IsLoading false for the initial resources to finish loading. Then perform any additional checks you require. A MutationObserver can be useful in detecting DOM changes, you can combine this with a promise. In the context of CefSharp you can now call EvaluateScriptAsPromiseAsync execute JavaScript that awaits the resolution/rejection of a promise. https://github.com/cefsharp/CefSharp/wi ... s-a-result

From a .Net point of view i personally prefer to use a TaskCompletionSource to provide an asynchronous method. This is my recommendation when wrapping code that needs to wait for completion.

To be honest I see people writing this sort of code a lot in the context of CefSharp and I cringe every time. Task.Wait in a loop is horrible code in my opinion. It is very difficult to write a one size fits all method as there are many variances in how webpages are constructed.
Maintainer of the CefSharp project.
amaitland
Master
 
Posts: 969
Joined: Wed Jan 14, 2015 2:35 am

Re: Contribution: Wait for browser finished loading function

Postby andla71 » Sat Oct 31, 2020 6:30 pm

amaitland wrote:CEF/Chromium are designed to be asynchronous, executing your script in a loop with a delay largely ignores this. Chromium uses multiple threads/processes and sitting there polling isn't great. You are attempting to return innerhtml, which depending on the page could be very large.

At a minimum you should wait for LoadingStateChange IsLoading false for the initial resources to finish loading. Then perform any additional checks you require. A MutationObserver can be useful in detecting DOM changes, you can combine this with a promise. In the context of CefSharp you can now call EvaluateScriptAsPromiseAsync execute JavaScript that awaits the resolution/rejection of a promise. https://github.com/cefsharp/CefSharp/wi ... s-a-result

From a .Net point of view i personally prefer to use a TaskCompletionSource to provide an asynchronous method. This is my recommendation when wrapping code that needs to wait for completion.

To be honest I see people writing this sort of code a lot in the context of CefSharp and I cringe every time. Task.Wait in a loop is horrible code in my opinion. It is very difficult to write a one size fits all method as there are many variances in how webpages are constructed.


Thank you for the response amaitland. You're the CEF God in my opinion and I've seen you helping a lot of people. Awesome to meet you :)
I myself is just a simple programmer/youtube influencer and I've been postponing my migration from the old IExplorer browser component to something modern. RIP IE.
Finally my migration is pretty much done.
What I do is building my own macro software to automate stuff.
The code I use gets a decent number of websites to test on.
I'm fine with things not being optimal because I Know If I'm making more complicated things like diving into multithreading, async support then I simply don't get very productive.
I say thank God Task.Wait exists. I need the macro command line work in synchronious sequence so that I can instantly see where something goes wrong.
Anyway CEF/CEFSharp is awesome and I'm very happy that I have come so far with the migration as well as meeting you Amaitland.
andla71
Newbie
 
Posts: 6
Joined: Sat Oct 31, 2020 12:19 pm

Re: Contribution: Wait for browser finished loading function

Postby amaitland » Tue Nov 03, 2020 8:17 pm

andla71 wrote: You're the CEF God in my opinion and I've seen you helping a lot of people.


I think you've got me mistaken for @magreenblatt the creator/maintainer of the `Chromium Embedded Framework(CEF)`. I'm probably more in the class of power user :)

andla71 wrote:Awesome to meet you :)


Thanks for the kind words, even if I think they're a little misplaced :) It's very much appreciated when people stop and say thanks!

andla71 wrote:I say thank God Task.Wait exists.


I wish there was an option to have the framework throw an exception if `Wait` is called :lol: Everyone has a different opinion, I appreciate you are being pragmatic and going with something that works for you.

andla71 wrote:Anyway CEF/CEFSharp is awesome


@magreenblatt Deserves the bulk of the credit in my opinion.
Maintainer of the CefSharp project.
amaitland
Master
 
Posts: 969
Joined: Wed Jan 14, 2015 2:35 am

Re: Contribution: Wait for browser finished loading function

Postby Czarek » Thu Nov 05, 2020 5:42 am

Agree. Alex has been with us for many years and he is doing a great job!
Maintainer of the CEF Python, PHP Desktop and CEF C API projects. Connect with me on LinkedIn.
User avatar
Czarek
Virtuoso
 
Posts: 1913
Joined: Sun Nov 06, 2011 2:12 am

Re: Contribution: Wait for browser finished loading function

Postby magreenblatt » Thu Nov 05, 2020 11:19 am

@magreenblatt Deserves the bulk of the credit in my opinion.

Your contributions are also very much appreciated, Alex. I think there’s plenty of credit (and work!) to go around :)
magreenblatt
Site Admin
 
Posts: 10939
Joined: Fri May 29, 2009 6:57 pm

Re: Contribution: Wait for browser finished loading function

Postby andla71 » Sat Nov 07, 2020 10:19 pm

Thank you for the response.
@magreenblatt huge cred sent to you as well.
You are all awesome and I'm not that awesome maybe a little bit ;)

BTW I believe you are very right about the LoadingStateChange.
I did some tests without it and in some cases, if the javascript is accessed to soon it kind of corrupts/messes with the DOM.
For example copy to clipboard didn't work. Another page went blank.
So accessing the jscript to early is not that brilliant.

Here is my contribution to LoadingStateChange (cleaned and simplified)
Code: Select all
                try
                {

                    using (var waitHandle = new System.Threading.AutoResetEvent(false))
                    {
                        EventHandler<LoadingStateChangedEventArgs> loadingHandler = null;
                       
                        loadingHandler = (sender, e) =>
                        {

                            if (!e.IsLoading)
                            {

                                waitHandle.Set();

                            }
                            else
                            {
                               

                            }
                        };


                        if (chrome != null)
                        {
                            chrome.Load(url);
                            chrome.LoadingStateChanged += loadingHandler;
                            waitHandle.WaitOne(80000);
                            chrome.LoadingStateChanged -= loadingHandler;
                        }
                    }

                }
                catch (Exception ex)
                {

                }
andla71
Newbie
 
Posts: 6
Joined: Sat Oct 31, 2020 12:19 pm


Return to CefSharp Forum

Who is online

Users browsing this forum: No registered users and 1 guest