Building CEF with media codecs on Windows Server 2012

[callum]

I have been able to get builds of the Chromium/CEF trunk with proprietary media codecs enabled using my own dev box - Windows 10/64.

I want to get automated builds using TeamCity working now but the build hosts it uses are all Windows Server 2012. I have the right Windows 10 SDK installed as well as Visual Studio 2017.

The TeamCity setup seems to exceed the specs laid on on the CEF build page https://bitbucket.org/chromiumembedded/cef/wiki/AutomatedBuildSetup.md - namely Windows 7/64 or later.
It's hard to read the TeamCity logs but the error appears to be happening when it starts to build the project files - I see "Generating CEF project files..." just before the error then lines like:

Code: Select all

Creating C:\code\chromium_git\chromium\src\out\Release_GN_x86 directory.
Traceback (most recent call last):
   File "C:/code/chromium_git/chromium/src/build/vs_toolchain.py", line 566, in <module>
     sys.exit(main())
   File "C:/code/chromium_git/chromium/src/build/vs_toolchain.py", line 562, in main
     return commands[sys.argv[1]](*sys.argv[2:])
   File "C:/code/chromium_git/chromium/src/build/vs_toolchain.py", line 383, in CopyDlls
     _CopyRuntime(target_dir, runtime_dir, target_cpu, debug=False)
   File "C:/code/chromium_git/chromium/src/build/vs_toolchain.py", line 358, in _CopyRuntime suffix)
   File "C:/code/chromium_git/chromium/src/build/vs_toolchain.py", line 296, in _CopyUCRTRuntime
     assert len(ucrt_files) > 0
AssertionError
ERROR at //build/toolchain/win/BUILD.gn:49:3: Script returned non-zero exit code.


Has anyone been able to get a build on Windows Server 2012?

[magreenblatt]

Is Visual Studio installed to the default location on the server? If not, you’ll need to set the environment variables described on the AutomatedBuildSetup wiki page.

[callum]

Yes, as far as I can tell. It's installed via an AMI image and I had no part in generating it but comparing directories, it appears to be identical to my local Windows 10 install.

[HarmlessDave]

How much RAM, swap space, and disk space does the instance have? A few years ago I could just barely get older versions of CEF3 to build with 8 GB RAM and a lot of swap space (12 GB) but for more recent versions you should have at least 16 GB RAM. Besides RAM + swap, you also need something like 80 GB of free disk space (so the instance needs more for swap, OS, temp files, ...)

[callum]

Good question re resources. I know the disk space is a little tight - there is 139GB free on the drive and my local builds consumed about 110GB. I’ll look at the other specs once I can revive the instance again.

[magreenblatt]

You can look at the code in vs_toolchain.py to see what files/paths it’s trying to copy and then verify that those exist in the expected locations on your server.

[callum]

Code: Select all

    1   1   services/viz/privileged/interfaces/compositing/layered_window_updater.mojom
 
    15   0   ui/compositor/compositor.h
 
    7   2   ui/compositor/host/host_context_factory_private.cc
 
 ... successfully applied.
 
 
 
 63 patches total (63 applied, 0 skipped, 0 failed)
 
 
 Generating CEF project files...
 Creating D:\cef3809_64\chromium_git\chromium\src\out\Release_GN_x86 directory.
 Traceback (most recent call last):
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 294, in <module>
 
     main()
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 220, in main
 
     env = _LoadToolchainEnv(cpu, win_sdk_path, target_store)
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 151, in _LoadToolchainEnv
 
     variables = _LoadEnvFromBat(args)
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 84, in _LoadEnvFromBat
 
     raise Exception('"%s" failed with error %d' % (args, popen.returncode))
 
 Exception: "['C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Auxiliary\\Build\\vcvarsall.bat', 'amd64_x86', '&&', 'set']" failed with error 255
 
 ERROR at //build/config/win/BUILD.gn:323:27: Script returned non-zero exit code.
   vcvars_toolchain_data = exec_script("../../toolchain/win/setup_toolchain.py",
                           ^----------


I've managed to resolve about a dozen gotchas with this TeamCity build of Win64 + media codecs (branch 3809) but this one has me licked. Building on an Amazon EC2 c3.4xlarge so there should be plenty of CPU, memory etc.

Builds work as expected when run from command prompt on but fails when run inside TeamCity.

As you can see from the fragment of log file, the build proceeds as normal, then gets to the "Generating CEF project files..." stage then fails quickly because running and capturing the environment changes from 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat', 'amd64_x86' fails for open.communicate() with an error 255 (-1 ???)

That's valid and running that line manually again, works as expected.

Long shot I know but has anyone experienced something similar and can point me at something to investigate?

[linuxcef9]

Code: Select all

    1   1   services/viz/privileged/interfaces/compositing/layered_window_updater.mojom
 
    15   0   ui/compositor/compositor.h
 
    7   2   ui/compositor/host/host_context_factory_private.cc
 
 ... successfully applied.
 
 
 
 63 patches total (63 applied, 0 skipped, 0 failed)
 
 
 Generating CEF project files...
 Creating D:\cef3809_64\chromium_git\chromium\src\out\Release_GN_x86 directory.
 Traceback (most recent call last):
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 294, in <module>
 
     main()
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 220, in main
 
     env = _LoadToolchainEnv(cpu, win_sdk_path, target_store)
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 151, in _LoadToolchainEnv
 
     variables = _LoadEnvFromBat(args)
 
   File "D:/cef3809_64/chromium_git/chromium/src/build/toolchain/win/setup_toolchain.py", line 84, in _LoadEnvFromBat
 
     raise Exception('"%s" failed with error %d' % (args, popen.returncode))
 
 Exception: "['C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Auxiliary\\Build\\vcvarsall.bat', 'amd64_x86', '&&', 'set']" failed with error 255
 
 ERROR at //build/config/win/BUILD.gn:323:27: Script returned non-zero exit code.
   vcvars_toolchain_data = exec_script("../../toolchain/win/setup_toolchain.py",
                           ^----------


I've managed to resolve about a dozen gotchas with this TeamCity build of Win64 + media codecs (branch 3809) but this one has me licked. Building on an Amazon EC2 c3.4xlarge so there should be plenty of CPU, memory etc.

Builds work as expected when run from command prompt on but fails when run inside TeamCity.

As you can see from the fragment of log file, the build proceeds as normal, then gets to the "Generating CEF project files..." stage then fails quickly because running and capturing the environment changes from 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat', 'amd64_x86' fails for open.communicate() with an error 255 (-1 ???)

That's valid and running that line manually again, works as expected.

Long shot I know but has anyone experienced something similar and can point me at something to investigate?

I had to debug similar kinda issues. Here's what I can suggest you to try :

1) Figure out from which directory was the script being invoked
2) Write a simple python script (or copy it from setup_toolchain.py or something) which just uses popen to execute the script. Make sure you give same relative path

That should narrow down the scope and you should be able to see at what point it fails. It'll save you from doing your entire build all over again.

[callum]

Good idea - thank you.

The only complication is I need to insert the test Python script into my project build scripts, check it in and ask TeamCity to build it - otherwise, if I run it on the box directly, I'm sure it will work as expected.

A bit fiddly but well worth a try - thank you.

[linuxcef9]

Good idea - thank you.

The only complication is I need to insert the test Python script into my project build scripts, check it in and ask TeamCity to build it - otherwise, if I run it on the box directly, I'm sure it will work as expected.

A bit fiddly but well worth a try - thank you.

Just make sure you mimic the original script as closely as possible - taking into account function parameters, environment variables etc
Otherwise your script will execute fine but the TeamCity build will continue to fail, making it harder to debug.