aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-03-04Snap for 11526323 from 229e46055785e329ca397400bd4bfba35a7ec1db to ↵simpleperf-releaseAndroid Build Coastguard Worker
simpleperf-release Change-Id: I3f786968fe8de6ceeb4585432b15cae5ad404b77
2024-02-28Merging 182 commit(s) from Chromium's toolchain-utilszijunzhao
Merged commit digest: 61074474 Dedupe addition of crash-diagnostics-dir c75f0ff3 llvm_tools: Fix cros workon invocation e632e6f8 Commit patches instead of using patch 74d1998a pgo_rust: remove code for rust-artifacts ebfdad7a rust_uprev: remove code for rust-artifacts 9bff7a7c compiler_wrapper: automatic sync fd01d9a8 llvm_tools: parse default-fatal warnings properly 6dbf24aa llvm_tools: add aggregate board root canonicalization 674ee6b0 compiler_wrapper: automatic sync f1aac8ff llvm_tools: add -Werror log fetching functionality 8446d3d7 compiler_wrapper: automatic sync b715ea44 toolchain-utils: Fix /home paths in rust analyzer wrapper e3acf484 auto_update_rust_bootstrap: fix dir detection logic 6ed3a301 toolchain-utils: Fix paths related to sysroot in rust-analyzer wrapper 6ecb1100 Clean up dead toolchain scripts 62b68a9e bugs: add Rust-y metabugs 6600638a auto_update_rust_bootstrap: fix whitespace in commit message ee70ef2a rust_uprev: don't update rust-bootstrap 282120e8 rust_uprev: refactor repetitive routines regarding rusty revisions 0a091272 rust_uprev: refactor create_ebuild() to take strings instead of a path c2b9e7a4 pgo_rust: explicitly build rust-artifacts bf1e26ec rust_uprev: use stdout=PIPE instead of capture_output 7d31a04d rust_uprev: remove update_manifest() 36aedb0f rust_tools: link rust upgrades to the rust maintenance metabug 88ec70a9 cros_utils: add parent_bug support to `bugs` 9910f177 cros_utils: fix minor issues 18e8096c toolchain-utils: Update rust-analyzer README 211f63aa llvm_tools: add setup_for_workon.py 73f2f02c compiler_wrapper: automatic sync 7868f472 rust_uprev: make find_ebuild_for_rust_version() call find_ebuild_path() 1333c307 rust_uprev: only return base ebuilds from find_rust_versions() 9f1cac16 rust_uprev: rewrite find_rust_versions() to use Path instead of os.* da46b240 rust_uprev_test: stop importing RustVersion e6052e7d compiler_wrapper: automatic sync 8a15d25d toolchain-utils: Fix clippy warnings for wrapper ce175473 toolchain-utils: Fix rust-analyzer-chromiumos-wrapper d01e0900 nightly_revert_checker: refactor find_shas f750cb6d nightly_revert_checker: add stale HEAD notifications 4093f8fd nightly_revert_checker: use a State object 46b7a614 nightly_revert_checker: import directly from `typing` b5900731 llvm_tools: Fix get_changed_packages in get_patch 521f6bf0 afdo_metadata: Publish the new kernel profiles 920d0402 fetch_cq_size_diff: add docs to README 68d56fca fetch_cq_size_diff: add debuginfo checking f005cb1a fetch_cq_size_diff: refactor to facilitate debuginfo d6925f77 llvm_tools: add a -Werror log parsing tool 567a3a8d pgo_tools: add update_llvm_manifest c3c9c087 llvm_tools: Force patch_utils.py to sort metadata f032a972 llvm_tools: Fix get_patch.py for cherrypicks 389fa0b2 llvm_tools: Add GitHub API usage to get_patch.py f94c1fdf llvm_tools: get_patch.py now applies patches f6ebdbbf llvm_tools: Add skeleton for get_patch.py 9ae061aa llvm_tools: add fetch_cq_size_diff 74c4d9da llvm_tools: add cros_cls f06e93eb compiler_wrapper: automatic sync ec7fdc05 afdo_metadata: Publish the new kernel profiles d6e8b62a toolchain_utils_githooks: add script to mypy checks 7fa195dc llvm_tools: Keep patches_info as PatchInfo 6ec59ddb rust_tools: fix bug in prebuilt autodetection 3d9f44f9 afdo_metadata: Publish the new kernel profiles 316be047 contrib: add bgtask d1ecff2d check_portable_toolchains: add sdk autodetection ff53a87d llvm_tools: opt into static checking for all Python code c681e245 llvm_tools/get_upstream_patch.py: fix type errors c5011c60 cros_utils/tiny_render.py: fix mypy complaints 0138c609 llvm_tools/llvm_project.py: opt into static type checking a117ba3f cros_utils/tiny_render.py: remove coding cookie 62443eee llvm_tools: accept Path or str in git_llvm_rev.LLVMConfig 85d69d3f llvm_tools/check_clang_diags.py: fix mypy complaints 7e5f7547 cros_utils/bugs.py: fix type annotations and opt into static checking f0c424df cros_utils/bugs.py: fix cros lint complaints 65570114 llvm_tools/modify_a_tryjob.py: opt into static type checking 290d35da llvm_tools: Remove "verbose" in places where it does nothing 012ab521 llvm_tools/update_packages_and_run_tests.py: add type annotations 4130ffa2 llvm_tools/update_chromeos_llvm_hash.py: fix type errors and unit test c5db9bcd llvm_tools/manifest_utils.py: opt into static type checking 3866411f llvm_tools/patch_{manager,utils}.py: opt into static type checking 8e7e57a7 llvm_tools/get_llvm_hash.py: annotate and opt into static type checking 97d35514 llvm_tools/git.py: opt into static type checking 0c334bf3 llvm_tools: remove duplication among chroot.py and git.py 30180e95 llvm_tools/atomic_write_file.py: fix type annotation errors d735864c afdo_metadata: Publish the new kernel profiles dcced50e llvm_tools/git_llvm_rev.py: opt into static type checking 3470bcec llvm_tools: Don't uprev LLVM in get_upstream_patch d0ae8571 compiler_wrapper: automatic sync 5ca2f14c llvm_tools/chroot: add type annotations and opt into static checking aaf2973f auto_update_rust_bootstrap: cosmetic bugfixes 11af312a rust_tools: add ebuild deletion to rust-bootstrap script 0bd6b24f rust_tools: rename auto_upload_rust_bootstrap 15daabda afdo_metadata: Publish the new kernel profiles ebe1777b rust_tools: add a tracking bug to auto_upload_rust_bootstrap 1dc476ec rust_tools: add automatic rust-bootstrap updating 0907a751 rust_tools: refactor auto_upload_rust_bootstrap slightly b1e6c0f3 afdo_metadata: Publish the new kernel profiles 90f011ee check-presubmit: use mypy to check everything in rust_tools 679eee7e rust_tools: fix mypy type complaints 763a108b rust_tools: fix lint errors in rust_watch.py 09d24dbd rust_tools: add auto_upload_rust_bootstrap eeadef2b afdo_metadata: Publish the new kernel profiles d23edc69 afdo_metadata: Publish the new kernel profiles bb3898c1 compiler_wrapper: automatic sync b20ec7ea afdo_metadata: Publish the new kernel profiles f55bdac8 update_chromeos_llvm_hash: add an option to skip patch application 17e4022a afdo_metadata: Publish the new kernel profiles 3d0d43c7 pgo_tools: don't upload profiles as (pretend) tarballs 61b08d5a llvm_tools: add revert report generation e3d6a223 pgo_tools: remove mention of llvm-next_pgo_use f7b7c6e4 check-presubmit: move `get_from_pip` under `get_pip` check 958ebe23 check-presubmit: put cros_utils on PYTHONPATH 7ecfa889 check-presubmit: silence mypy on imported modules 3bb829cc patch_sync: Add README.md cc9e607b patch_sync: Add automation shiboleth for AOSP 83a8ff6a afdo_metadata: Publish the new kernel profiles e9028c37 pgo_tools: Use home path for hyperfine 2b3dbd00 copy_rust_bootstrap: use lbzip2 instead of pbzip2 acf15930 copy_rust_bootstrap: add missing `stdin` for gsutil 8f2c2657 patch_sync: Don't uprev by default 9e86601c nightly_revert_checker: use github links instead of phabricator a3fafb9b afdo_metadata: Publish the new kernel profiles 2f0a643f create_chroot_and_generate_pgo_profile: provide --out-dir 36de6197 compiler_wrapper: automatic sync 14ee7bb9 nightly_revert_checker: remove reviewers from email cc lines d7f84c5c afdo_metadata: Publish the new kernel profiles 1490f1c0 pgo_tools: finish a refactor 9bb87f21 pgo_tools: add automation to ensure PGO remains a win e3edb8b9 pgo_tools: refactor benchmarking to be usable as a module e539461c pgo_tools: refactor chroot checking into a shared function 397b1bf9 llvm_tools: remove mention of llvm-tot f3607d6a rust_uprev: cache sudo credentials 0baa32a0 git_llvm_rev: add caching 3a104f71 compiler_wrapper: automatic sync 11cdd46c afdo_metadata: Publish the new kernel profiles 5a9032ff pgo_tools: add new PGO scripts 4716c2de check-presubmit.py: support directories in mypy allowlist 06115d3e pgo_tools: delete old monitoring/merging scripts 3d8fdf8d compiler_wrapper: automatic sync 39b17698 check-presubmit: complain about changes to compiler_wrapper 4e5ad1c5 compiler_wrapper: use -D_FORTIFY_SOURCE=3 by default 6b06f503 Revert "compiler_wrapper: manually apply -D_FORTIFY_SOURCE=2" b0269024 afdo_metadata: Publish the new kernel profiles 4976f274 compiler_wrapper: remove -Wno-string-concatenation e9441706 compiler_wrapper: remove -Wno-gnu-offsetof-extensions c89a3484 llvm_tools: Fix cq-depend None bug 81cc9288 llvm_tools: refactor to fix unexpected None in a List[str] 0cc50104 llvm_tools: add a few types d6426ec3 llvm_tools: use utf-8 consistently 1c3292a5 subprocess_helpers: fix lints 0643bf41 manifest_utils_unittest: fix lints, mark as executable 9b5404ac get_llvm_hash: fix lint errors 25e67eba check_clang_diags: fix lints; reformat messaging; change cc 0b0c58f7 llvm_tools: fix tiny lints in files 26b37ade llvm_tools: address atomic_write_file lints ef94419d afdo_metadata: Publish the new kernel profiles 409bba31 llvm_tools: pass a list instead of a string 994c900b llvm_tools: rename extra_commit_msg for clarity 0f121ea3 llvm_tools: add a few function types & modernize Cq-Depend 67c552a3 llvm_tools: address lints fdc70e83 llvm_tools: format update hash scripts & test 6b56a7d7 llvm_tools: update_chromeos_llvm_hash cq-depend d5bd6461 llvm_tools: update_chromeos_llvm_hash no_delete_branch e7d3b21b afdo_metadata: Publish the new kernel profiles f7312293 orderfile: remove subdir 62ae06cb check_portable_toolchains: Print clang version 795d7eb7 Update check_portable_toolchains in python 06f69dcd llvm_tools: update_chromeos_llvm_hash: reversion -> revision a5861631 afdo_metadata: Publish the new kernel profiles c0ab4a07 update_kernel_afdo: Remove kernel 4.14. 54aae7c0 llvm_tools: Version sys-devel/llvm same as others af0e5ae3 githooks: import typenames instead of "import typing as t" 7b10332e githooks: fix type errors 50db271f llvm_tools: Fix static check errors for nightly_revert_checker 039ee2b2 githooks: add Python type checking using mypy a4bfa9d8 rust_uprev: avoid serialization error after PGO_RUST invocations 6db0bf33 rust_uprev.py: run outside the chroot 64722563 patch_sync: Rename ANDROID_MAIN_BRANCH to main c534313c compiler_wrapper: Remove -Wno-deprecated-copy 3043ebb5 compiler_wrapper: Remove -Wno-compound-token-split-by-space cf422856 compiler_wrapper: Remove -Wno-array-parameter 43418f57 toolchain_utils: Fix lints in run_tests_for.py ea8a42a1 toolchain-utils: Replace pipes import 62be7cfe compiler_wrapper: remove custom default c++ std 2a71e5f6 toolchain_utils_githooks: rewrite pip installation logic Change-Id: Ib6bb58803c2b4daab832402c2ba94637aa12f47f
2024-02-28Dedupe addition of crash-diagnostics-dirEric Lok
For chromeos-chrome builds, we need to remove any unnecessary {PVR} like info from the build cmd in order for cache hits across builds. chromeos-chrome already defines "crash_diagnostics-dir" in such a manner, but compiler-wrapper adds a duplicate definition of this same flag but with {PVR} like info. The fix here is to update compiler-wrapper so that it does not add "crash-diagnostics-dir" if it is already defined. BUG=b:326847828 TEST=go test -vet=all (and added more tests) Change-Id: I81c4dbb29241e85cf78659f35ac9dd93999b6b01 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5321366 Tested-by: Eric Lok <lokeric@google.com> Commit-Queue: Eric Lok <lokeric@google.com> Auto-Submit: Eric Lok <lokeric@google.com> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: Ryan Beltran <ryanbeltran@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Reviewed-by: Raul Rangel <rrangel@chromium.org>
2024-02-23llvm_tools: Fix cros workon invocationJordan R Abrahams-Whitehead
At present, we invoke `cros workon` using the `cros-workon` style syntax. This unfortunately: 1. Is going to be deprecated. 2. Doesn't work outside of the chroot. 3. Assumes you're working on the --host, when in fact you may not want to workon the host, you may want to workon a board. This CL changes this behaviour, by not enabling cros workon by default (warning the user it's not workon-ed), and using the more modern cros workon syntax. BUG=None TEST=llvm_tools/setup_for_workon.py \ --workon-board volteer-scudo \ --package sys-libs/scudo TEST=llvm_tools/setup_for_workon.py \ --package sys-libs/scudo Change-Id: I4f65c73ef12e64da973a14ec1ea9b9ea7b70b35f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5321923 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com>
2024-02-20Commit patches instead of using patchzijunzhao
Bug: b/287560304 Test: ./patch_manager/utils_unittest.py Change-Id: I3ce32822029d0760d3909fce2230b471d975d805 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5213948 Tested-by: Zijun Zhao <zijunzhao@google.com> Auto-Submit: Zijun Zhao <zijunzhao@google.com> Commit-Queue: Zijun Zhao <zijunzhao@google.com> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com>
2024-02-20pgo_rust: remove code for rust-artifactsBob Haarman
rust-artifacts no longer exists, so don't try to build it. BUG=b:297387332 TEST=pgo_rust.py generate Change-Id: Ifc5d8cd020d5b06eafb84a25dcf2e46277308116 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5301348 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-02-20rust_uprev: remove code for rust-artifactsBob Haarman
rust-artifacts no longer exists, so remove the code from rust_uprev that deals with it. The Manifest that used to live in rust-artifacts now lives in rust-host, so update rust_uprev accordingly. BUG=b:297387332 TEST=create a rust uprev with the new code Change-Id: If0a315d19eac31ce3a7d6b3e309ce2c80e5eff91 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5300550 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-02-19compiler_wrapper: automatic syncmobiletc-prebuild
This CL automatically brings toolchain-utils' compiler_wrapper/ directory in sync with chromiumos-overlay's. Please see go/crostc-repo/+/main:sync_compiler_wrapper_within_cros.sh for more info on this process. BUG=None TEST=None Change-Id: Ife77212a517d7ed5ce54415738e81374474a71eb Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5306972 Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Auto-Submit: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com>
2024-02-16llvm_tools: parse default-fatal warnings properlyGeorge Burgess IV
We have a class of warnings that don't print `,-Werror` in the warning specification brackets. Those should be handled just as well as the ones that do. BUG=b:325463152 TEST=Unittests Change-Id: I9bd8351a6ff14c6e7820d9992b145d70b0860169 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5299451 Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org>
2024-02-16llvm_tools: add aggregate board root canonicalizationGeorge Burgess IV
Many error messages fron boards will talk about absolute paths, like /build/atlas/foo/bar/baz.cc. When aggregating errors across many boards, this can lead to the same warning appearing quite often at paths that only differ in the board's name. Allowing translation of `/build/atlas` to `/build/{board}` makes this easier to deal with. BUG=b:316172255 TEST=Unittests Change-Id: I0312b782b6aa4754034236605d1311f51a249216 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5296918 Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2024-02-15compiler_wrapper: automatic syncmobiletc-prebuild
This CL automatically brings toolchain-utils' compiler_wrapper/ directory in sync with chromiumos-overlay's. Please see go/crostc-repo/+/main:sync_compiler_wrapper_within_cros.sh for more info on this process. BUG=None TEST=None Change-Id: I8e43f915da44101b5b4a28d9fccb198ecdcac2cf Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5299664 Tested-by: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Auto-Submit: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com>
2024-02-14llvm_tools: add -Werror log fetching functionalityGeorge Burgess IV
This allows us to fetch -Werror logs from CLs, or specific cq-orchestrator runs. These are unpacked so one can later `analyze` them. BUG=b:316172255 TEST=Ran (slightly modified) on a CQ that produced many Clang crash TEST=artifacts. Change-Id: Ice9a142a98951d7baf79fd8509b6be0b91c10a19 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5296915 Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2024-02-12compiler_wrapper: automatic syncmobiletc-prebuild
This CL automatically brings toolchain-utils' compiler_wrapper/ directory in sync with chromiumos-overlay's. Please see go/crostc-repo/+/main:sync_compiler_wrapper_within_cros.sh for more info on this process. BUG=None TEST=None Change-Id: I5c416f548d5ceeac2834c1f1b5bc2ba47d127131 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5285300 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Auto-Submit: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com>
2024-02-07toolchain-utils: Fix /home paths in rust analyzer wrapperBastian Kersting
In some places (e.g. rust registry at /home/$USER/.cargo/registry), the rust-analyzer inside chroot points to /home. This is visible outside the chroot under "<chromiumos-root>/out/home". BUG=b:320674356 TEST=manual Change-Id: I50b4d30d3f3578e8a8ee54c3d98f62fcdd5b183c Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5277036 Tested-by: Bastian Kersting <bkersting@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bastian Kersting <bkersting@google.com> Reviewed-by: Allen Webb <allenwebb@google.com>
2024-02-06auto_update_rust_bootstrap: fix dir detection logicGeorge Burgess IV
Apparently I had `__name__ == "__main__"` on my mind when writing this? :) BUG=b:323850985 TEST=Ran ./rust_tools/auto_update_rust_bootstrap.py; it printed the TEST=right `my_dir` value. Change-Id: I0987cd659489eb0fd525bad0b5b7de5f77292333 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5269401 Commit-Queue: Bob Haarman <inglorion@chromium.org> Auto-Submit: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org>
2024-02-06toolchain-utils: Fix paths related to sysroot in rust-analyzer wrapperBastian Kersting
Currently when trying to open e.g. a standard library type's source in VSCode, rust-analyzer-chromiumos-wrapper will point to the wrong path. That is, because inside chroot, rust's sysroot lives under /usr, while outside of the chroot it usually lives under $HOME/.rustup. This patch adds the capability for translating these sysroot related paths accordingly. BUG=b:324045239, b:320674356 TEST=manual Change-Id: Id0a01409a8622159cad136485f7b8c077a19fbc6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5272425 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bastian Kersting <bkersting@google.com> Tested-by: Bastian Kersting <bkersting@google.com>
2024-02-02Clean up dead toolchain scriptsJordan R Abrahams-Whitehead
Each of these scripts are unrunnable currently, have been unrunnable for at least 6 months, and have not been updated in years. BUG=None TEST=run_tests_for.py llvm_tools/* cros_utils/* Change-Id: I00da0677699f35e827e899d01372ca1c029aba5b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5262987 Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com>
2024-02-02Snap for 11400057 from 8eed62767c6a80c3c26359d38d4961069da83813 to ↵Android Build Coastguard Worker
simpleperf-release Change-Id: Ied9f25e50c60da2077053ff599873bb16ce7ebed
2024-02-02bugs: add Rust-y metabugsGeorge Burgess IV
These need to be used from internal tooling. Migrate/centralize them here for convenience. BUG=b:323522852 TEST=repo upload Change-Id: I8fb0317fe92773ffacd68daf8d31379155ee2c3d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5262323 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Auto-Submit: George Burgess <gbiv@chromium.org>
2024-02-01auto_update_rust_bootstrap: fix whitespace in commit messageBob Haarman
auto_update_rust_bootstrap.py writes commit messages using f-strings and textwrap.dedent. However, dedent only removes whitespace from the beginning of lines, and the code used line continuations that cause repeated whitespace inside of lines. This change moves things around a bit so that messages are formatted as expected. BUG=None TEST=Ran the script, checked the resulting commit message. Change-Id: Ia9ba93b64bcb214d17e90e96567349c3bd7fdeb6 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5259243 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-02-01rust_uprev: don't update rust-bootstrapBob Haarman
rust_uprev.py contains code that updates the rust-bootstrap ebuild. This was previously necessary, because, without it, the version of rust-bootstrap needed to build the other rust packages would not be available. However, this is no longer true: We now update rust-bootstrap to the same version as dev-lang/rust and the others, which means that when we advance the non-bootstrap version, it will only be one step ahead of rust-bootstrap and therefore work. auto_update_rust_bootstrap.py will then take care of updating rust-bootstrap. As a result, we don't need logic for that in rust_uprev.py anymore, and this change removes it. Some code that refers to rust-bootstrap is still preserved, namely the code updates the bootstrap version encoded in cros-rustc.eclass and the code that verifies that rust-bootstrap sources and prebuilts are available on the mirror. BUG=None TEST=presubmit Change-Id: I0959bfc6d442467622b0403e6cb975d5b579aad4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5259526 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-02-01rust_uprev: refactor repetitive routines regarding rusty revisionsBob Haarman
rust_uprev handles several ebuilds: dev-lang/rust, dev-lang/rust-artifacts, dev-lang/rust-host, and dev-lang/rust-bootstrap. Excepting the last, the script generally needs to process all of them the same way. Previously, this was done by repeating code snippets for each package. This change instead puts designators for the packages to process in a list, then iterates over that list and runs the same code on each. This makes it easier to ensure that all packages are indeed processed the same way, instead of, say, adding code that creates a new ebuild for a package, but not adding code that removes the old version. BUG=b:322348600 TEST=presubmit, create a rust uprev with it Change-Id: Ide5b2dab00e88633c8e5b80c1ee465adee14917b Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5255518 Tested-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2024-02-01rust_uprev: refactor create_ebuild() to take strings instead of a pathBob Haarman
create_ebuild() used to take a path to an ebuild file to copy and a pkgatom which it would split and use to compute the new ebuild path from. This change refactors create_ebuild() to take the ebuild's category and name as separate strings (to avoid having to split strings), and computes both the path to the source ebuild and the path to the ebuild to be created (to avoid the inconsistency of computing one but requiring the other to be passed in). This refactor will be built upon by an upcoming refactor of create_rust_uprev(), for which it is convenient not to have to compute the source ebuild path before calling create_ebuild(). BUG=b:322348600 TEST=presubmit Change-Id: I000e45423e7e33b110d60bd04a59c9f2863597b2 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5255378 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2024-02-01pgo_rust: explicitly build rust-artifactsBob Haarman
pgo_rust builds dev-lang/rust and dev-lang/rust-host a couple of times with different USE flags (instrumented for frontend profiling, instrumented for LLVM profiling) and generates profile data with each configuration. However, USE flags are not taken into account when deciding if dependencies need to be rebuilt. This resulted in dev-lang/rust-artifacts not being rebuilt with the changed USE flags, and profiles not being generated correctly. This change explicitly rebuilds dev-lang/rust-artifacts each time we change the flags, so that profiles are generated correctly. BUG=b:322998657 TEST=pgo_rust.py generate; see that profiles are generated successfully Change-Id: I28265c8d5cf98c4abf751f681e7a361be587359a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5258978 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2024-02-01rust_uprev: use stdout=PIPE instead of capture_outputBob Haarman
When running equery fails, we currently don't get an error message, because it is swallowed by subprocess.run. This change makes it so that the error message is shown on standard error instead. BUG=None TEST=made it fail, saw error message Change-Id: Ib057f3215505d5824593d387c22df5f95b163921 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5259297 Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2024-02-01rust_uprev: remove update_manifest()Bob Haarman
update_manifest() takes a path, which we then translate to a package name, which we then look up in the filesystem to get the path to the ebuild file. This change removes a level of indirection by calling ebuild_actions() with the package name directly. BUG=None TEST=Create a rust uprev Change-Id: I1b1651cc7d1fc735c353b55fa28c5766015e4829 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5255377 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2024-01-30rust_tools: link rust upgrades to the rust maintenance metabugGeorge Burgess IV
BUG=b:322398536 TEST=Unittests Change-Id: I3ba346dc2e23cbc2876f8f2578048b24da620fb3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5249777 Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2024-01-30cros_utils: add parent_bug support to `bugs`George Burgess IV
This adds support for the JSON bits added in cl/601848281. BUG=b:322398536 TEST=Unittests Change-Id: I07b8ebc05dbf8c99d36bb0a014f504d873bd8e6a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5249776 Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org>
2024-01-30cros_utils: fix minor issuesGeorge Burgess IV
The comment block was missing `Args`. Add it for consistency and added lint checking. Slight fixups in bugs_test are due to feedback from `cros lint` on module importing. BUG=b:322398536 TEST=None Change-Id: I09571313fb27e4d5e20ca70a11489ef70da6323a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5249775 Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2024-01-26toolchain-utils: Update rust-analyzer READMEBastian Kersting
This change updates the rust-analyzer-chromiumos-wrapper README file, since it contains outdated information. BUG=b:320674356 TEST=CQ Change-Id: Ia9199e968cdfab89a8320e8970d8a5c191117702 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5233611 Commit-Queue: Bastian Kersting <bkersting@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Bastian Kersting <bkersting@google.com>
2024-01-25llvm_tools: add setup_for_workon.pyGeorge Burgess IV
This adds a script to complement the changes made in crrev.com/c/5120358. The intent is for devs to run this script prior to hacking on llvm. There are no tests in this CL since this script is _mostly_ just a series of `subprocess.run`s with a tiny bit of intervening logic. It's also 100% intended for interactive use. BUG=b:315871489 TEST=Ran the script. Change-Id: I2d516b22ba7f7d95d3a3666cc2b6ab51810cf7ee Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5119366 Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org>
2024-01-25compiler_wrapper: automatic syncmobiletc-prebuild
This CL automatically brings toolchain-utils' compiler_wrapper/ directory in sync with chromiumos-overlay's. Please see go/crostc-repo/+/main:sync_compiler_wrapper_within_cros.sh for more info on this process. BUG=None TEST=None Change-Id: I57353526b8f113892556bb4f4c07e10ecb42ce00 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5237730 Commit-Queue: Bob Haarman <inglorion@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org> Tested-by: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Auto-Submit: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com>
2024-01-25rust_uprev: make find_ebuild_for_rust_version() call find_ebuild_path()Bob Haarman
Previously, find_ebuild_for_rust_version() enumerated all versions of dev-lang/rust, then filtered them to only those that match the specified version. Instead, call find_ebuild_path(), which finds the requested version directly and also contains an implementation of the logic that tests only one version exists and properly accounts for symlinks. BUG=b:318424456 TEST=presubmit, use the new code to create a rust uprev Change-Id: Id7a058cb58a94cfa8a208d92ba6fc24b6344a683 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5233111 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com>
2024-01-25rust_uprev: only return base ebuilds from find_rust_versions()Bob Haarman
find_rust_versions() previously returned values for both rust-x.y.z.ebuild and rust-x.y.z-rn.ebuild if both existed. We commonly have x.y.z-rn.ebuild be a symlink to x.y.z.ebuild, and the code that calls find_rust_versions() actually needs only the base file and will not work correctly if both are returned. This change makes find_rust_versions() only return non-symlinks and adds documentation and a test. BUG=b:318424456 TEST=./rust_tools/rust_uprev_test.py Change-Id: I18b0795fb93c9a9d5a81424ebaf07ba43476ba3f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5233610 Tested-by: Bob Haarman <inglorion@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2024-01-25rust_uprev: rewrite find_rust_versions() to use Path instead of os.*Bob Haarman
This is a small refactor ahead of a change to the logic. The rewritten find_rust_versions() returns a list of tuples of (RustVersion, Path) instead of (RustVersion, str) as before, and updates the call sites and test to match. BUG=b:318424456 TEST=presubmit type checks and tests Change-Id: If5e26ee3c40265031c43fe68d921d43405a44e3a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5233609 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-01-25rust_uprev_test: stop importing RustVersionBob Haarman
cros lint does not like us to import rust_uprev.RustVersion and wants us to import only rust_uprev. This change does that. BUG=None TEST=presubmit Change-Id: I8b1b26113b673acdc4c153b9ce0221abf1330f8d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5233608 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org> Tested-by: Bob Haarman <inglorion@chromium.org>
2024-01-22compiler_wrapper: automatic syncmobiletc-prebuild
This CL automatically brings toolchain-utils' compiler_wrapper/ directory in sync with chromiumos-overlay's. Please see go/crostc-repo/+/main:sync_compiler_wrapper_within_cros.sh for more info on this process. BUG=None TEST=None Change-Id: I24075774fe1646895ad7c39fd646f293ef76ddca Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5211566 Tested-by: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Auto-Submit: mobiletc-prebuild Role Account <mobiletc-prebuild@google.com> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: Bob Haarman <inglorion@chromium.org>
2024-01-22toolchain-utils: Fix clippy warnings for wrapperBastian Kersting
BUG=None TEST=CQ Change-Id: I0c203c05bd39ce6e8f508af311ef10c9db1af8ec Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5214370 Reviewed-by: George Burgess <gbiv@chromium.org> Tested-by: Bastian Kersting <bkersting@google.com> Commit-Queue: Bastian Kersting <bkersting@google.com>
2024-01-22toolchain-utils: Fix rust-analyzer-chromiumos-wrapperBastian Kersting
Currently rust-analyzer-chromiumos-wrapper is broken, since it translates Paths in the wrong way. 1. It only replaces paths with the `file://` prefix, which is not correct for various rust-analyzer specific settings. 2. It wrongfully doesn't replace the rust-analyzer.server.path` setting` and thus lets the chroot internal rust-analyzer believe in a wrong server path. 3. It assumes the `rootPath` and `uri` property ends in a `/` while it actually doesn't. This CL fixes the issues and together with crrev/c/5210755 (which adds back a rust-analyzer in chroot) makes using rust-analyzer for chromiumos development possible again. BUG=b:320674356 TEST=check out this patch and crrev/c/320674356. Execute `emerge dev-lang/rust-host`. Install the wrapper using `cargo install --path src/third_party/toolchain-utils/rust-analyzer-chromiumos-wrapper`. Then for use with VSCode set the rust-analyzer.server.path setting in `settings.json` to `$CARGO/bin/rust-analyzer-chromiumos-wrapper`. After restarting the server, start editing a rust file and it works! Change-Id: I58bf2fcfb36ce94ceb62a46171c180981b7511f0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5212068 Reviewed-by: Allen Webb <allenwebb@google.com> Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Bastian Kersting <bkersting@google.com> Tested-by: Bastian Kersting <bkersting@google.com>
2024-01-19nightly_revert_checker: refactor find_shasGeorge Burgess IV
Jordan commented on this being confusing, and I agree. Refactor this function to not update a parameter on the fly, and name it more precisely for what it's trying to do. BUG=b:319635292 TEST=Unittests Change-Id: Iee7ec49f909cc593102eb8881d3ea2fda05dc697 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5217602 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: George Burgess <gbiv@chromium.org>
2024-01-19nightly_revert_checker: add stale HEAD notificationsGeorge Burgess IV
This notifies teams when this tool doesn't observe their LLVM revision moving for a while. The intent isn't to put pressure on LLVM upgrade rotations; more to catch bugs in this checker earlier rather than later. BUG=b:319635292 TEST=Unittests Change-Id: I211b8803f483a96952f5d573bdb083ac723d3ca4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5200618 Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2024-01-19nightly_revert_checker: use a State objectGeorge Burgess IV
I plan to add more to State in a later CL. It's much easier to modify State if it's a dataclass than if it's a dict. This also generally helps with readability. BUG=b:319635292 TEST=./nightly_revert_checker_test.py Change-Id: I6b57719239fc6483dbac9c9feb4fd7b25f7cfb3f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5200617 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Commit-Queue: George Burgess <gbiv@chromium.org>
2024-01-19nightly_revert_checker: import directly from `typing`George Burgess IV
`t.` was written here since there wasn't style guidance on using `from typing import ...`. Since there's now that guidance, freshen this up. BUG=b:319635292 TEST=./nightly_revert_checker_test.py Change-Id: I8211067081bdaa81c2256360736c1d1fd14cc2e5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5200616 Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: George Burgess <gbiv@chromium.org>
2024-01-18llvm_tools: Fix get_changed_packages in get_patchJordan R Abrahams-Whitehead
get_changed_packages was conducting a diff between the base and the head ref. However, given the head ref could be very distant from the base, the expected diff ends up being much larger than the final merged diff. Instead compare between the base and the new HEAD after making the squash commit. Also while we're here, actually compute the valid patch range and also remove the PatchApplicationError. Also add --verbose flag for command parsing, so we can enable the more verbose debugging logs. BUG=None TEST=run_tests_for.py llvm_tools/get_patch.py TEST=llvm_tools/get_patch.py -s HEAD -l ~/llvm-project p:74531 Change-Id: I192295f47172013d75e31768789ace04182850f3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5208750 Reviewed-by: George Burgess <gbiv@chromium.org> Commit-Queue: Jordan Abrahams-Whitehead <ajordanr@google.com> Tested-by: Jordan Abrahams-Whitehead <ajordanr@google.com>
2024-01-17afdo_metadata: Publish the new kernel profilesmobiletc-prebuild
Update amd profile on chromeos-kernel-5.4 Update amd profile on chromeos-kernel-5.10 Update amd profile on chromeos-kernel-5.15 Update arm profile on chromeos-kernel-5.15 BUG=None TEST=Verified in kernel-release-afdo-verify-orchestrator Change-Id: Ia1ac7f7baba11797d41ed9512b7b7a93603da44e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5201263 Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: George Burgess <gbiv@chromium.org>
2024-01-11fetch_cq_size_diff: add docs to READMEGeorge Burgess IV
BUG=b:316172272 TEST=None Change-Id: I0ea0d1f0e6eb72b5b584d9b4adeb605c160300a1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5154267 Tested-by: George Burgess <gbiv@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org>
2024-01-11fetch_cq_size_diff: add debuginfo checkingGeorge Burgess IV
BUG=b:316172272 TEST=Ran the script with `--debuginfo` Change-Id: I1e90028357bc62e54a4ee70e7b825cf82a84e644 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5154135 Reviewed-by: Bob Haarman <inglorion@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org>
2024-01-11fetch_cq_size_diff: refactor to facilitate debuginfoGeorge Burgess IV
Checking debuginfo for the Mage's test matrix is very similar in many ways to checking image sizes. Refactor a bit to take advantage of this. BUG=b:316172272 TEST=Ran the script. Change-Id: Idd1dc82a38e2b501d1dfe26bf514904a592f4996 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5154134 Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Bob Haarman <inglorion@chromium.org> Commit-Queue: George Burgess <gbiv@chromium.org>
2024-01-10llvm_tools: add a -Werror log parsing toolGeorge Burgess IV
This is the first "half" of points 3 and 4 in the description of b/316172255. This is useful on its own, since if a Mage runs a local `FORCE_DISABLE_WERROR` build, they can point this tool at their build tree and get a full report of all of the suppressed warnings. BUG=b:316172255 TEST=Unittests; ran on a few `FORCE_DISABLE_WERROR` trees Change-Id: I95bfc5de70f8614bf0ae7570fd5ab31766ce405d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5180185 Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org>
2024-01-09pgo_tools: add update_llvm_manifestGeorge Burgess IV
Realistically, adding llvm-next PGO profiles isn't a _difficult_ or _toilsome_ thing, but it's nice to be able to say "just run this script." Plus, scraping these links from the ebuild is pretty trivial, and they're unlikely to change much as time goes on. BUG=b:315642924 TEST=./update_llvm_manifest.py --llvm-next Change-Id: I754445fc67474507d92809980da85be20a4d79aa Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/third_party/toolchain-utils/+/5170318 Commit-Queue: George Burgess <gbiv@chromium.org> Tested-by: George Burgess <gbiv@chromium.org> Reviewed-by: Ryan Beltran <ryanbeltran@chromium.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-30 00:40:18 +0000