5938 (117.0.5938.150) compilation fails on windows

[linuxcef9]

Hello team,

I am building CEF from source on windows using automated method for release branch 5938(117.0.5938.150) using below command:

Code: Select all

python ..\automate\automate-git.py --no-depot-tools-update --download-dir=%CHROMIUM_GIT_DIR% --depot-tools-dir=%DEPOT_TOOLS_DIR% --force-clean --no-distrib --no-build --branch=5938 --chromium-checkout=refs/tags/117.0.5938.150 --verbose-build --checkout=946dde2e17880748cc0dbbb516a86c359eda3bc7 --with-pgo-profiles

But the compilation is failing with below error:

application.go:144] fatal error: failed to build venv: failed to build package : cipkg.Derivation{Name:"python_venv", Platform:"os=windows,arch=amd64", Builder:"C:.vpython-root\\store\\cpython-e6tenrlsesftvg6k08ajgkk7b4\\contents\\bin\\python3.exe", Args:[]string{"-c", "# Copyright 2022 The Chromium Authors\n# Use of this source code is governed by a BSD-style license that can be\n# found in the LICENSE file.\n\nimport glob\nimport os\nimport subprocess\nimport sys\n\nISOLATION_FLAG = '-I' if sys.version_info[0] > 2 else '-sSE'\n\n# Create virtual environment in ${out} directory\nvirtualenv = glob.glob(\n os.path.join(r'C:.vpython-root\\store\\virtualenv-qb7nsruah38d8pbovm856nd258\\contents', '*', 'virtualenv.py*'))[0]\nsubprocess.check_call([\n sys.executable, ISOLATION_FLAG, virtualenv,\n '--no-download', '--always-copy', os.environ['out']\n])\n\n# Install wheels to virtual environment\nif 'wheels' in os.environ:\n pip = glob.glob(os.path.join(os.environ['out'], '*', 'pip*'))[0]\n subprocess.check_call([\n pip,\n 'install',\n '--isolated',\n '--compile',\n '--no-index',\n '--find-links',\n os.path.join(os.environ['wheels'], 'wheels'),\n '--requirement',\n os.path.join(os.environ['wheels'], 'requirements.txt'),\n ])\n\n# Generate all .pyc in the output directory. This prevent generating .pyc on the\n# fly, which modifies derivation output after the build.\n# It may fail because lack of permission. Ignore the error since it won't affect\n# correctness if .pyc can't be written to the directory anyway.\ntry:\n subprocess.check_call([\n sys.executable, ISOLATION_FLAG, '-m', 'compileall', os.environ['out']\n ])\nexcept subprocess.CalledProcessError as e:\n print('complieall failed and ignored: {}'.format(e.returncode))\n"}, Env:[]string{"depsHostTarget=C:.vpython-root\\store\\cpython-e6tenrlsesftvg6k08ajgkk7b4\\contents;C:.vpython-root\\store\\virtualenv-qb7nsruah38d8pbovm856nd258\\contents"}, Inputs:[]string{"cpython-e6tenrlsesftvg6k08ajgkk7b4", "virtualenv-qb7nsruah38d8pbovm856nd258"}}: failed to build derivation: fork/exec C:.vpython-root\store\cpython-e6tenrlsesftvg6k08ajgkk7b4\contents\bin\python3.exe: The system cannot find the path specified.

followed by error for command ""gclient sync --nohooks --with_branch_heads --jobs 16":

subprocess.CalledProcessError: Command '['gclient', 'sync', '--nohooks', '--with_branch_heads', '--jobs', '16']' returned non-zero exit status 1

Can someone help me with the fix for this issue?

[magreenblatt]

Are you building in a cmd.exe shell? Remove any python installations from your PATH, and make sure it includes the depot_tools path first.

[linuxcef9]

Thanks for your quick response.
Yes, we are building in cmd.exe shell. We will try your suggestion.
Not sure how it is working with depot tools of 5790 branch in same environment. Any idea?

[magreenblatt]

Not sure how it is working with depot tools of 5790 branch in same environment. Any idea?

Maybe the Go dependency was added between those versions?

[linuxcef9]

But latest depot tool should have all the dependencies, right?

[magreenblatt]

But latest depot tool should have all the dependencies, right?

Yes, if you're using the correct depot_tools version. Why are you passing --no-depot-tools-update? Also, don't set the DEPOT_TOOLS_UPDATE=0 env.

[linuxcef9]

Issue still persist even with correct/latest depot tools (skipped --no-depot-tools-update param, also not setting DEPOT_TOOLS_UPDATE=0 env).
In the first error, I can see '\' is missing after 'C:' in 'Builder' path, any idea who sets this env variable? Can this cause failure, where chromium src is not getting downloaded?

application.go:144] fatal error: failed to build venv: failed to build package : cipkg.Derivation{Name:"python_venv", Platform:"os=windows,arch=amd64", Builder:"C:.vpython-root\\store\\cpython-e6tenrlsesftvg6k08ajgkk7b4\\contents\\bin\\python3.exe", Args:[]string{"-c", "# Copyright 2022 The Chromium Authors\n# Use of this source code is governed by a BSD-style license that can be\n# found in the LICENSE file.\n\nimport glob\nimport os\nimport subprocess\nimport sys\n\nISOLATION_FLAG = '-I' if sys.version_info[0] > 2 else '-sSE'\n\n# Create virtual environment in ${out} directory\nvirtualenv = glob.glob(\n os.path.join(r'C:.vpython-root\\store\\virtualenv-qb7nsruah38d8pbovm856nd258\\contents', '*', 'virtualenv.py*'))[0]\nsubprocess.check_call([\n sys.executable, ISOLATION_FLAG, virtualenv,\n '--no-download', '--always-copy', os.environ['out']\n])\n\n# Install wheels to virtual environment\nif 'wheels' in os.environ:\n pip = glob.glob(os.path.join(os.environ['out'], '*', 'pip*'))[0]\n subprocess.check_call([\n pip,\n 'install',\n '--isolated',\n '--compile',\n '--no-index',\n '--find-links',\n os.path.join(os.environ['wheels'], 'wheels'),\n '--requirement',\n os.path.join(os.environ['wheels'], 'requirements.txt'),\n ])\n\n# Generate all .pyc in the output directory. This prevent generating .pyc on the\n# fly, which modifies derivation output after the build.\n# It may fail because lack of permission. Ignore the error since it won't affect\n# correctness if .pyc can't be written to the directory anyway.\ntry:\n subprocess.check_call([\n sys.executable, ISOLATION_FLAG, '-m', 'compileall', os.environ['out']\n ])\nexcept subprocess.CalledProcessError as e:\n print('complieall failed and ignored: {}'.format(e.returncode))\n"}, Env:[]string{"depsHostTarget=C:.vpython-root\\store\\cpython-e6tenrlsesftvg6k08ajgkk7b4\\contents;C:.vpython-root\\store\\virtualenv-qb7nsruah38d8pbovm856nd258\\contents"}, Inputs:[]string{"cpython-e6tenrlsesftvg6k08ajgkk7b4", "virtualenv-qb7nsruah38d8pbovm856nd258"}}: failed to build derivation: fork/exec C:.vpython-root\store\cpython-e6tenrlsesftvg6k08ajgkk7b4\contents\bin\python3.exe: The system cannot find the path specified.

[ndesktop]

Somehow vpython-root does not look like the correct depot_tools path. Use something like

Code: Select all

SET DEPOT_TOOLS=C:\depot_tools
SET PATH=%DEPOT_TOOLS%;%PATH%

before invoking automate-git.py.

[linuxcef9]

Yes, I am setting depot tool dir path as a first entry in PATH env before invoking automate-git.py.
Any idea, who sets this vpython-root?

[linuxcef9]

Fixed this issue by setting VPYTHON_VIRTUALENV_ROOT env variable to temp dir.

application.go determines VirtualEnv base directory path using os.UserCacheDir(). UserCacheDir() is supposed to return %LocalAppData% on windows.
But its not returning proper value in my case, which in turn formed the invalid base directory path for VirtualEnv (c:.vpython-root).

If VPYTHON_VIRTUALENV_ROOT is set then application.go picks this value and proceeds without any error.