Skip to content

Fix artifact download step key for PyTorch and QA test pipelines#2982

Merged
edsavage merged 4 commits intoelastic:mainfrom
edsavage:fix/artifact-download-step-key
Mar 12, 2026
Merged

Fix artifact download step key for PyTorch and QA test pipelines#2982
edsavage merged 4 commits intoelastic:mainfrom
edsavage:fix/artifact-download-step-key

Conversation

@edsavage
Copy link
Contributor

Summary

  • After the build/test step split, the run_pytorch_tests.yml.sh and run_qa_tests.yml.sh pipelines were still downloading build artifacts from the test step (build_test_linux-x86_64-RelWithDebInfo), which doesn't upload build artifacts.
  • Updated the --step argument to reference the build step (build_linux-x86_64-RelWithDebInfo), which is the step that uploads build/distributions/*.zip.
  • The depends_on remains on the test step to ensure downstream tests only trigger after unit tests pass.

Fixes: https://buildkite.com/elastic/ml-cpp-pr-builds/builds/2286#019cd609-00cd-424f-aee3-6d5985b979af

Test plan

  • Verify the nightly PyTorch test pipeline can successfully download build artifacts
  • Verify the QA test pipeline can successfully download build artifacts

Made with Cursor

@prodsecmachine
Copy link

prodsecmachine commented Mar 10, 2026

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Copy link
Contributor

@valeriy42 valeriy42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for fixing this. 🚀

After the build/test step split, build artifacts are uploaded by the
build step (key: build_linux-x86_64-RelWithDebInfo), not the test step
(key: build_test_linux-x86_64-RelWithDebInfo). Update the artifact
download commands to reference the correct step key.

Made-with: Cursor
Made-with: Cursor
The build/test split (PR elastic#2914) renamed build step keys from
build_test_{platform} to build_{platform}, breaking downstream
pipelines (notably appex-qa-stateful-custom-ml-cpp-build-testing)
that download artifacts using the old step key.

Restore the build_test_ prefix on build steps and give test steps
a distinct test_ prefix:

  Build steps:  build_test_{platform}  (compile, upload artifacts)
  Test steps:   test_{platform}        (download bundle, run tests)

This fixes:
- QA pipeline artifact download (uses --step build_test_linux-*)
- sccache config in post-checkout (build_test_* wildcard now
  matches build steps that actually compile, not test steps)
- Analytics tokens now correctly target test_ steps
- DRA creation waits for test_ steps to complete
- ES integration tests depend on build_test_ steps

Made-with: Cursor
The test_linux-x86_64-RelWithDebInfo-debug step was missing from
the BUILDKITE_ANALYTICS_TOKEN conditions, causing the
test-collector plugin to fail with exit status 1.

Made-with: Cursor
@edsavage edsavage force-pushed the fix/artifact-download-step-key branch from 167bc06 to 3ed28e0 Compare March 11, 2026 23:11
@edsavage edsavage merged commit 6f8b14e into elastic:main Mar 12, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants