justinp-tt 7e8a7d7336 Flaky Test Detector (#6645) ...

## What changed?
<!-- Describe what has changed in this PR -->
Add flaky test detection by running modified test suites multiple times

- Create `find_altered_tests.go` to gather modified test suites per
category.

- Update Makefile:  
  - Modify test targets to accept `TEST_ARGS` where it was omitted
- Run modified test suites multiple times using `go test
-count=$(N_RUNS)` to detect flaky tests.

- Update GitHub Actions workflows:
- Update `set-up-single-test` job to collect modified test suites and
pass them to test jobs.
- Adjust test jobs to run modified test suites multiple times per
category.
- Optimize test execution by running tests once per category with all
modified suites.

## Why?
<!-- Tell your future self why have you made these changes -->
Pragmatic steps to reduce introduction of new flaky tests

## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->

## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->

## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->

## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->

---------

Co-authored-by: David Reiss <david@temporal.io>
Co-authored-by: Roey Berman <roey@temporal.io>

2024-10-15 22:31:59 -05:00

..

server

Import alignment (#6426)

2024-08-23 14:48:10 -06:00

tools

Flaky Test Detector (#6645)

2024-10-15 22:31:59 -05:00