Google says they will be blocking sign-ins from CEF

Do not post support requests, bug reports or feature requests. Discuss CEF here. Non-CEF related discussion goes in General Discussion!

Re: Google says they will be blocking sign-ins from CEF

Postby linuxcef9 » Fri Aug 30, 2019 4:26 pm

I am using - for different reasons - a patched CEF, which overrides user agent on a per request/url basis.


Hmm thanks for the workaround!
linuxcef9
Expert
 
Posts: 143
Joined: Tue Nov 06, 2018 3:08 pm

Re: Google says they will be blocking sign-ins from CEF

Postby rjxray » Mon Feb 17, 2020 10:08 am

I've been trying to implement the user-agent patch posted above by ndesktop
At first I thought I had to do it in the cef source in my chromium build and then make a new binary distribution but after looking at it for while I thought I could do it without rebuilding cef like this:

Code: Select all
class MyCefResourceRequestHandler : public CefResourceRequestHandler {
public:
   CefResourceRequestHandler::ReturnValue OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      CefRefPtr<CefRequestCallback> callback) OVERRIDE;

   bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefDictionaryValue> request_info,
      CefString& overrideUserAgent);
};


and this just for test for test purposes initially
Code: Select all
CefResourceRequestHandler::ReturnValue MyCefResourceRequestHandler::OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
   CefRefPtr<CefFrame> frame,
   CefRefPtr<CefRequest> request,
   CefRefPtr<CefRequestCallback> callback) {

   CefRefPtr<CefDictionaryValue> request_info = CefDictionaryValue::Create();
   if (request_info.get()) {
      request_info->SetString(CefString("url"), request->GetURL());
   }
   CefRequest::HeaderMap headers;
   request->GetHeaderMap(headers);

   CefString overrideUserAgent;
   if (GetOverrideUserAgent(browser, request_info, overrideUserAgent))
   {
      headers.erase("User-Agent");
      headers.insert(std::make_pair("User-Agent", overrideUserAgent));

      request->SetHeaderMap(headers);
   }

   return RV_CONTINUE;
}

bool MyCefResourceRequestHandler::GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
   CefRefPtr<CefDictionaryValue> request_info,
   CefString& overrideUserAgent) {
   // spoof user-agent to Firefox for proof of concept
   overrideUserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0";
   return true;
}


But the user-agent is unchanged and it seems that OnBeforeResourceLoad() is never called.
I saw it would be called on the IO thread and I have enabled child process debugging but I still don't see it being called

So it seems that I am fundamentally misunderstanding something - I do struggle to understand the way cef is structured and operates

Any help or suggestions would be greatly appreciated

I am using the 3770 branch under Windows with VS2019
rjxray
Expert
 
Posts: 115
Joined: Wed Jun 07, 2017 4:31 am

Re: Google says they will be blocking sign-ins from CEF

Postby rjxray » Mon Feb 17, 2020 10:22 am

OK, I initially tried implementing this in my CefClient which I am creating for my browser. I guess I somehow need to add it to the browser itself.

After further investigation I see that you can pass a CefRefPtr<CefRequestContext> to CreatBrowserSync() - I am currently passing NULL
CefRequestContext has a CreateContext() method that takes a CefRequestContextHandler parameter.
A CefRequestContextHandler has a virtual GetResourceRequestHandler() method that i can make return my class which has the implementation of OnBeforeResourceLoad()

At least I think that's how it might work, I’ll try it tomorrow.
rjxray
Expert
 
Posts: 115
Joined: Wed Jun 07, 2017 4:31 am

Re: Google says they will be blocking sign-ins from CEF

Postby ndesktop » Mon Feb 17, 2020 3:13 pm

My implementation is this (branch 3904 - this combined with the relatively new network service might differ than the previous net implementation):
- CefRequestHandler
Code: Select all
virtual bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
                                    CefRefPtr<CefDictionaryValue> request_info,
                                    CefString& overrideUserAgent) { return false; }


I'm using the request_info dictionary so I can pass various things there from client without need to modify the prototype.

- libcef/browser/net_service/resource_request_handler_wrapper.cc
helper function:
Code: Select all
std::string GetUserAgentFromBrowser(CefRefPtr<CefBrowserHostImpl> browser,
                                    const GURL& url) {
  std::string browser_user_agent = "";

  if(browser) {
    CefRefPtr<CefClient> client = browser->GetClient();
    if(client) {
      CefRefPtr<CefRequestHandler> request_handler =
        client->GetRequestHandler();
      if(request_handler) {
        // build dictionary (for now just URL)
        // to avoid modifying prototypes later
        CefRefPtr<CefDictionaryValue> request_info =
          CefDictionaryValue::Create();
        if(request_info.get()) {
          request_info->SetString(CefString("url"), url.spec());
        }

        CefString overrideUserAgent;
        if(request_handler->GetOverrideUserAgent(browser.get(),
          request_info, overrideUserAgent)) {
          browser_user_agent = overrideUserAgent.ToString();
        }
      }
    }
  } 
 
  return browser_user_agent;
}


used in OnBeforeRequest (I'm pasting my current implementation, might differ since I'm not on the last 3904 revision):
Code: Select all
// InterceptedRequestHandler methods:
  void OnBeforeRequest(const RequestId& id,
                       network::ResourceRequest* request,
                       bool request_was_redirected,
                       OnBeforeRequestResultCallback callback,
                       CancelRequestCallback cancel_callback) override {
    CEF_REQUIRE_IOT();

    if (shutting_down_) {
      // Abort immediately.
      std::move(cancel_callback).Run(net::ERR_ABORTED);
      return;
    }

    if (!init_state_) {
      // Queue requests until we're initialized.
      pending_requests_.push_back(std::make_unique<PendingRequest>(
          id, request, request_was_redirected, std::move(callback),
          std::move(cancel_callback)));
      return;
    }

    // State may already exist for restarted requests.
    RequestState* state = GetOrCreateState(id);

    // Add standard headers, if currently unspecified.
    request->headers.SetHeaderIfMissing(
        net::HttpRequestHeaders::kAcceptLanguage,
        init_state_->accept_language_);

    request->headers.SetHeaderIfMissing(net::HttpRequestHeaders::kUserAgent,
                                        init_state_->user_agent_);
    // if browser says otherwise, override with it
    std::string browser_user_agent =
      GetUserAgentFromBrowser(init_state_->browser_, request->url);
   if(!browser_user_agent.empty()) {
      request->headers.SetHeader(net::HttpRequestHeaders::kUserAgent,
                                 browser_user_agent);
   }
    const bool is_external = IsExternalRequest(request);

    // External requests will not have a default handler.
    bool intercept_only = is_external;

    CefRefPtr<CefRequestImpl> requestPtr;
    CefRefPtr<CefResourceRequestHandler> handler =
        GetHandler(id, request, &intercept_only, requestPtr);

    CefRefPtr<CefSchemeHandlerFactory> scheme_factory =
        init_state_->resource_context_->GetSchemeHandlerFactory(request->url);
    if (scheme_factory && !requestPtr) {
      requestPtr = MakeRequest(request, id.hash(), true);
    }

    // True if there's a possibility that the client might handle the request.
    const bool maybe_intercept_request = handler || scheme_factory;
    if (!maybe_intercept_request && requestPtr)
      requestPtr = nullptr;

    // May have a handler and/or scheme factory.
    state->Reset(handler, scheme_factory, requestPtr, request_was_redirected,
                 std::move(cancel_callback));

    if (handler) {
      state->cookie_filter_ = handler->GetCookieAccessFilter(
          init_state_->browser_, init_state_->frame_, requestPtr.get());
    }

    auto exec_callback =
        base::BindOnce(std::move(callback), maybe_intercept_request,
                       is_external ? true : intercept_only);

    if (!maybe_intercept_request) {
      // Cookies will be handled by the NetworkService.
      std::move(exec_callback).Run();
      return;
    }

    MaybeLoadCookies(id, state, request, std::move(exec_callback));
  }



- this gets implemented in client:
cef/tests/cefclient/browser/client_handler.h:
Code: Select all
bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
    CefRefPtr<CefDictionaryValue> request_info,
    CefString& overrideUserAgent) OVERRIDE;


cef/tests/cefclient/browser/client_handler.cc:
Code: Select all
bool ClientHandler::GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
                                         CefRefPtr<CefDictionaryValue> request_info,
                                         CefString& overrideUserAgent) {
    do {
        if(!request_info.get()) {
            LOG(INFO) << "[cefclient] request_info is nullptr";
            break;
        }
        if(!request_info->HasKey("url")) {
            LOG(INFO) << "[cefclient] request_info does not have the key 'url'";
            break;
        }
        CefString url = request_info->GetString("url");
        if(url.empty()) {
            LOG(INFO) << "[cefclient] request_info 'url' key value is empty";
            break;
        }
        std::wstring sUrl = url.ToWString();
        std::wstring sDomain;
        bool fGetDomain = GetDomainOfUrl(sDomain, url.c_str());
        if(fGetDomain) {
            if(sDomain == L"get.adobe.com") {
                if(wcsstr(GetCommandLineW(), L"--getflash-as-firefox") != NULL) {
                    overrideUserAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0";
                    std::wstring str = L"Posing as Firefox on get.adobe.com: UA=";
                    str += overrideUserAgent.ToWString(); // temp object, but
                    str += L"\r\n";
                    ::OutputDebugStringW(str.c_str());
                    return true;
                }
            }
        }
#pragma warning(disable: 4127)
    } while(0);
#pragma warning(default: 4127)
    return false;
}


This is a legacy implementation started by the need of supporting NPAPI longer than the deprecation, so on get.adobe.com (modified cefclient shows this usage) UA posed as Firefox to let the user install NPAPI Flash. Obviously this was years ago, now it has other purposes (such as GMail signin).
ndesktop
Master
 
Posts: 748
Joined: Thu Dec 03, 2015 10:10 am

Re: Google says they will be blocking sign-ins from CEF

Postby rjxray » Tue Feb 18, 2020 6:47 am

I made my original scheme work - simple and no need to mess around with context

Code: Select all
class MyCefResourceRequestHandler : public CefResourceRequestHandler {
public:
   //CefResourceRequestHandler methods
   CefResourceRequestHandler::ReturnValue OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      CefRefPtr<CefRequestCallback> callback) OVERRIDE;

   bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefDictionaryValue> request_info,
      CefString& overrideUserAgent);

   IMPLEMENT_REFCOUNTING(MyCefResourceRequestHandler);
};

Code: Select all
class MyCefRequestHandler : public CefRequestHandler {
public:
   MyCefRequestHandler() {
      resourceRequesthandler = new MyCefResourceRequestHandler();
   }

   CefRefPtr<CefResourceRequestHandler> GetResourceRequestHandler(
      CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      bool is_navigation,
      bool is_download,
      const CefString& request_initiator,
      bool& disable_default_handling) OVERRIDE {
      return resourceRequesthandler;
   }

private:
   CefRefPtr<MyCefResourceRequestHandler> resourceRequesthandler;

   IMPLEMENT_REFCOUNTING(MyCefRequestHandler);
};


then in my CefClient:

add the requesthandler to the class
Code: Select all
CefRefPtr <MyCefRequestHandler> requestHandler_;

initialise it in the constructor
Code: Select all
requestHandler_ = new MyCefRequestHandler();

Implement GetRequestHandler
Code: Select all
   CefRefPtr<CefRequestHandler> GetRequestHandler() {
      return requestHandler_;
   }


OnBeforeResourceLoad and GetOverrideUserAgent are unchanged from my post yesterday and they are now called and the supplied user-agent is used by Cef.
Now I can modify it to be different for different sites
rjxray
Expert
 
Posts: 115
Joined: Wed Jun 07, 2017 4:31 am

Re: Google says they will be blocking sign-ins from CEF

Postby doberso » Fri May 01, 2020 5:05 am

Whilst the details are vague at the moment, Marshall may have some more insight. It would be nice if Google presented a standard approach for when the request is blocked so it could be managed in a seamless fashion. Perhaps a predictable URL pattern (urls may potentially vary by region), or a set URL and expected error code.
doberso
Newbie
 
Posts: 1
Joined: Thu Apr 09, 2020 5:31 am

Re: Google says they will be blocking sign-ins from CEF

Postby kbourro » Wed May 06, 2020 8:53 am

It seems that changing user agent doesn't' t work anymore. Already tried with the latest Firefox version user-agent.
kbourro
Newbie
 
Posts: 1
Joined: Wed May 06, 2020 8:52 am

Re: Google says they will be blocking sign-ins from CEF

Postby hisan » Sun Jun 21, 2020 3:26 pm

Searching for the good working solution for CEF to login to Google account. Ready to pay good money. Please pm me.
hisan
Newbie
 
Posts: 1
Joined: Sun Jun 21, 2020 10:55 am

Re: Google says they will be blocking sign-ins from CEF

Postby ajitchavan » Thu Dec 02, 2021 11:08 pm

Anyone found solution for this. we were able to login using User agent
Code: Select all
 "Mozilla/5.0 (X11; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"

But now that also not working.
ajitchavan
Techie
 
Posts: 27
Joined: Mon Mar 16, 2020 7:03 am

Re: Google says they will be blocking sign-ins from CEF

Postby ndesktop » Mon Dec 06, 2021 7:44 am

Just logged on using
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0"
user agent.
Patched CEF 95.42.14, using an internal implementation posted above to override user agent.
ndesktop
Master
 
Posts: 748
Joined: Thu Dec 03, 2015 10:10 am

Previous

Return to CEF Discussion

Who is online

Users browsing this forum: No registered users and 5 guests