Page 2 of 2

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Fri Aug 30, 2019 4:26 pm
by linuxcef9
I am using - for different reasons - a patched CEF, which overrides user agent on a per request/url basis.


Hmm thanks for the workaround!

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Mon Feb 17, 2020 10:08 am
by rjxray
I've been trying to implement the user-agent patch posted above by ndesktop
At first I thought I had to do it in the cef source in my chromium build and then make a new binary distribution but after looking at it for while I thought I could do it without rebuilding cef like this:

Code: Select all
class MyCefResourceRequestHandler : public CefResourceRequestHandler {
public:
   CefResourceRequestHandler::ReturnValue OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      CefRefPtr<CefRequestCallback> callback) OVERRIDE;

   bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefDictionaryValue> request_info,
      CefString& overrideUserAgent);
};


and this just for test for test purposes initially
Code: Select all
CefResourceRequestHandler::ReturnValue MyCefResourceRequestHandler::OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
   CefRefPtr<CefFrame> frame,
   CefRefPtr<CefRequest> request,
   CefRefPtr<CefRequestCallback> callback) {

   CefRefPtr<CefDictionaryValue> request_info = CefDictionaryValue::Create();
   if (request_info.get()) {
      request_info->SetString(CefString("url"), request->GetURL());
   }
   CefRequest::HeaderMap headers;
   request->GetHeaderMap(headers);

   CefString overrideUserAgent;
   if (GetOverrideUserAgent(browser, request_info, overrideUserAgent))
   {
      headers.erase("User-Agent");
      headers.insert(std::make_pair("User-Agent", overrideUserAgent));

      request->SetHeaderMap(headers);
   }

   return RV_CONTINUE;
}

bool MyCefResourceRequestHandler::GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
   CefRefPtr<CefDictionaryValue> request_info,
   CefString& overrideUserAgent) {
   // spoof user-agent to Firefox for proof of concept
   overrideUserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0";
   return true;
}


But the user-agent is unchanged and it seems that OnBeforeResourceLoad() is never called.
I saw it would be called on the IO thread and I have enabled child process debugging but I still don't see it being called

So it seems that I am fundamentally misunderstanding something - I do struggle to understand the way cef is structured and operates

Any help or suggestions would be greatly appreciated

I am using the 3770 branch under Windows with VS2019

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Mon Feb 17, 2020 10:22 am
by rjxray
OK, I initially tried implementing this in my CefClient which I am creating for my browser. I guess I somehow need to add it to the browser itself.

After further investigation I see that you can pass a CefRefPtr<CefRequestContext> to CreatBrowserSync() - I am currently passing NULL
CefRequestContext has a CreateContext() method that takes a CefRequestContextHandler parameter.
A CefRequestContextHandler has a virtual GetResourceRequestHandler() method that i can make return my class which has the implementation of OnBeforeResourceLoad()

At least I think that's how it might work, I’ll try it tomorrow.

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Mon Feb 17, 2020 3:13 pm
by ndesktop
My implementation is this (branch 3904 - this combined with the relatively new network service might differ than the previous net implementation):
- CefRequestHandler
Code: Select all
virtual bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
                                    CefRefPtr<CefDictionaryValue> request_info,
                                    CefString& overrideUserAgent) { return false; }


I'm using the request_info dictionary so I can pass various things there from client without need to modify the prototype.

- libcef/browser/net_service/resource_request_handler_wrapper.cc
helper function:
Code: Select all
std::string GetUserAgentFromBrowser(CefRefPtr<CefBrowserHostImpl> browser,
                                    const GURL& url) {
  std::string browser_user_agent = "";

  if(browser) {
    CefRefPtr<CefClient> client = browser->GetClient();
    if(client) {
      CefRefPtr<CefRequestHandler> request_handler =
        client->GetRequestHandler();
      if(request_handler) {
        // build dictionary (for now just URL)
        // to avoid modifying prototypes later
        CefRefPtr<CefDictionaryValue> request_info =
          CefDictionaryValue::Create();
        if(request_info.get()) {
          request_info->SetString(CefString("url"), url.spec());
        }

        CefString overrideUserAgent;
        if(request_handler->GetOverrideUserAgent(browser.get(),
          request_info, overrideUserAgent)) {
          browser_user_agent = overrideUserAgent.ToString();
        }
      }
    }
  } 
 
  return browser_user_agent;
}


used in OnBeforeRequest (I'm pasting my current implementation, might differ since I'm not on the last 3904 revision):
Code: Select all
// InterceptedRequestHandler methods:
  void OnBeforeRequest(const RequestId& id,
                       network::ResourceRequest* request,
                       bool request_was_redirected,
                       OnBeforeRequestResultCallback callback,
                       CancelRequestCallback cancel_callback) override {
    CEF_REQUIRE_IOT();

    if (shutting_down_) {
      // Abort immediately.
      std::move(cancel_callback).Run(net::ERR_ABORTED);
      return;
    }

    if (!init_state_) {
      // Queue requests until we're initialized.
      pending_requests_.push_back(std::make_unique<PendingRequest>(
          id, request, request_was_redirected, std::move(callback),
          std::move(cancel_callback)));
      return;
    }

    // State may already exist for restarted requests.
    RequestState* state = GetOrCreateState(id);

    // Add standard headers, if currently unspecified.
    request->headers.SetHeaderIfMissing(
        net::HttpRequestHeaders::kAcceptLanguage,
        init_state_->accept_language_);

    request->headers.SetHeaderIfMissing(net::HttpRequestHeaders::kUserAgent,
                                        init_state_->user_agent_);
    // if browser says otherwise, override with it
    std::string browser_user_agent =
      GetUserAgentFromBrowser(init_state_->browser_, request->url);
   if(!browser_user_agent.empty()) {
      request->headers.SetHeader(net::HttpRequestHeaders::kUserAgent,
                                 browser_user_agent);
   }
    const bool is_external = IsExternalRequest(request);

    // External requests will not have a default handler.
    bool intercept_only = is_external;

    CefRefPtr<CefRequestImpl> requestPtr;
    CefRefPtr<CefResourceRequestHandler> handler =
        GetHandler(id, request, &intercept_only, requestPtr);

    CefRefPtr<CefSchemeHandlerFactory> scheme_factory =
        init_state_->resource_context_->GetSchemeHandlerFactory(request->url);
    if (scheme_factory && !requestPtr) {
      requestPtr = MakeRequest(request, id.hash(), true);
    }

    // True if there's a possibility that the client might handle the request.
    const bool maybe_intercept_request = handler || scheme_factory;
    if (!maybe_intercept_request && requestPtr)
      requestPtr = nullptr;

    // May have a handler and/or scheme factory.
    state->Reset(handler, scheme_factory, requestPtr, request_was_redirected,
                 std::move(cancel_callback));

    if (handler) {
      state->cookie_filter_ = handler->GetCookieAccessFilter(
          init_state_->browser_, init_state_->frame_, requestPtr.get());
    }

    auto exec_callback =
        base::BindOnce(std::move(callback), maybe_intercept_request,
                       is_external ? true : intercept_only);

    if (!maybe_intercept_request) {
      // Cookies will be handled by the NetworkService.
      std::move(exec_callback).Run();
      return;
    }

    MaybeLoadCookies(id, state, request, std::move(exec_callback));
  }



- this gets implemented in client:
cef/tests/cefclient/browser/client_handler.h:
Code: Select all
bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
    CefRefPtr<CefDictionaryValue> request_info,
    CefString& overrideUserAgent) OVERRIDE;


cef/tests/cefclient/browser/client_handler.cc:
Code: Select all
bool ClientHandler::GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
                                         CefRefPtr<CefDictionaryValue> request_info,
                                         CefString& overrideUserAgent) {
    do {
        if(!request_info.get()) {
            LOG(INFO) << "[cefclient] request_info is nullptr";
            break;
        }
        if(!request_info->HasKey("url")) {
            LOG(INFO) << "[cefclient] request_info does not have the key 'url'";
            break;
        }
        CefString url = request_info->GetString("url");
        if(url.empty()) {
            LOG(INFO) << "[cefclient] request_info 'url' key value is empty";
            break;
        }
        std::wstring sUrl = url.ToWString();
        std::wstring sDomain;
        bool fGetDomain = GetDomainOfUrl(sDomain, url.c_str());
        if(fGetDomain) {
            if(sDomain == L"get.adobe.com") {
                if(wcsstr(GetCommandLineW(), L"--getflash-as-firefox") != NULL) {
                    overrideUserAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0";
                    std::wstring str = L"Posing as Firefox on get.adobe.com: UA=";
                    str += overrideUserAgent.ToWString(); // temp object, but
                    str += L"\r\n";
                    ::OutputDebugStringW(str.c_str());
                    return true;
                }
            }
        }
#pragma warning(disable: 4127)
    } while(0);
#pragma warning(default: 4127)
    return false;
}


This is a legacy implementation started by the need of supporting NPAPI longer than the deprecation, so on get.adobe.com (modified cefclient shows this usage) UA posed as Firefox to let the user install NPAPI Flash. Obviously this was years ago, now it has other purposes (such as GMail signin).

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Tue Feb 18, 2020 6:47 am
by rjxray
I made my original scheme work - simple and no need to mess around with context

Code: Select all
class MyCefResourceRequestHandler : public CefResourceRequestHandler {
public:
   //CefResourceRequestHandler methods
   CefResourceRequestHandler::ReturnValue OnBeforeResourceLoad(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      CefRefPtr<CefRequestCallback> callback) OVERRIDE;

   bool GetOverrideUserAgent(CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefDictionaryValue> request_info,
      CefString& overrideUserAgent);

   IMPLEMENT_REFCOUNTING(MyCefResourceRequestHandler);
};

Code: Select all
class MyCefRequestHandler : public CefRequestHandler {
public:
   MyCefRequestHandler() {
      resourceRequesthandler = new MyCefResourceRequestHandler();
   }

   CefRefPtr<CefResourceRequestHandler> GetResourceRequestHandler(
      CefRefPtr<CefBrowser> browser,
      CefRefPtr<CefFrame> frame,
      CefRefPtr<CefRequest> request,
      bool is_navigation,
      bool is_download,
      const CefString& request_initiator,
      bool& disable_default_handling) OVERRIDE {
      return resourceRequesthandler;
   }

private:
   CefRefPtr<MyCefResourceRequestHandler> resourceRequesthandler;

   IMPLEMENT_REFCOUNTING(MyCefRequestHandler);
};


then in my CefClient:

add the requesthandler to the class
Code: Select all
CefRefPtr <MyCefRequestHandler> requestHandler_;

initialise it in the constructor
Code: Select all
requestHandler_ = new MyCefRequestHandler();

Implement GetRequestHandler
Code: Select all
   CefRefPtr<CefRequestHandler> GetRequestHandler() {
      return requestHandler_;
   }


OnBeforeResourceLoad and GetOverrideUserAgent are unchanged from my post yesterday and they are now called and the supplied user-agent is used by Cef.
Now I can modify it to be different for different sites

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Fri May 01, 2020 5:05 am
by doberso
Whilst the details are vague at the moment, Marshall may have some more insight. It would be nice if Google presented a standard approach for when the request is blocked so it could be managed in a seamless fashion. Perhaps a predictable URL pattern (urls may potentially vary by region), or a set URL and expected error code.

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Wed May 06, 2020 8:53 am
by kbourro
It seems that changing user agent doesn't' t work anymore. Already tried with the latest Firefox version user-agent.

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Sun Jun 21, 2020 3:26 pm
by hisan
Searching for the good working solution for CEF to login to Google account. Ready to pay good money. Please pm me.

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Thu Dec 02, 2021 11:08 pm
by ajitchavan
Anyone found solution for this. we were able to login using User agent
Code: Select all
 "Mozilla/5.0 (X11; Linux x86_64; rv:92.0) Gecko/20100101 Firefox/92.0"

But now that also not working.

Re: Google says they will be blocking sign-ins from CEF

PostPosted: Mon Dec 06, 2021 7:44 am
by ndesktop
Just logged on using
"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0"
user agent.
Patched CEF 95.42.14, using an internal implementation posted above to override user agent.