DAG-1879: Add support for generating burn_in_tests #39
Conversation
dbradf
requested review from
a team and
zituo-jin
and removed request for
a team
|
This still needs more testing, but I wanted to get it out so people could start taking a look. |
| @@ -44,8 +45,12 @@ impl EvgConfigService for EvgProjectConfig { | |||
| } | |||
|
|
|||
| /// Get a map of task name to task definitions. | |||
| fn get_task_def_map(&self) -> HashMap<String, &EvgTask> { | |||
| self.evg_project.task_def_map() | |||
| fn get_task_def_map(&self) -> HashMap<String, EvgTask> { | |||
There was a problem hiding this comment.
Since we will now be sharing this among threads, it was easier to take ownership of the EvgTask rather than having a reference to it.
| if !cmd.status.success() { | ||
| let error_message = String::from_utf8_lossy(&cmd.stderr).to_string(); | ||
|
|
||
| event!( |
There was a problem hiding this comment.
With this function we will log stderr output when we encounter an error trying to execute commands. We should be able to switch resmoke calls over to use this function and then they will have better error messages. I didn't add this here to keep this PR focused.
| test_list: vec![test.to_string()], | ||
| exclude_test_list: None, | ||
| origin_suite: origin_suite.clone(), | ||
| mv_exclude_tags: suite_info.multiversion_tags.clone(), |
There was a problem hiding this comment.
Did we do this thing with adding tags in the old burn in task generator?
If this is just keeping the same functionality, then it's fine. I just don't remember, if resmoke is able to exclude the test from running, if we pass the test path as an argument.
There was a problem hiding this comment.
This could be a problem that needs more investigation. I don't think burn_in has been running multiversion tests properly, so we likely haven't hit this before. But we should investigate how resmoke will behave. Ideally, resmoke would just no-op in this case, which is what I think we would want.
There was a problem hiding this comment.
I've checked --excludeWithAnyTags resmoke's flag, the resmoke no-oped and didn't run the test even if I passed it as an argument. So if mv_exclude_tags end up being in --excludeWithAnyTags list, then it should work.
python buildscripts/resmoke.py run --suites=replica_sets_jscore_passthrough_last_lts_old_new_new --dbpathPrefix=build/data/db --excludeWithAnyTags=multiversion_incompatible jstests/core/read_after_optime.js
[resmoke] 11:56:10.534Z verbatim resmoke.py invocation: buildscripts/resmoke.py run --suites=replica_sets_jscore_passthrough_last_lts_old_new_new --dbpathPrefix=build/data/db --excludeWithAnyTags=multiversion_incompatible jstests/core/read_after_optime.js
[resmoke] 11:56:10.538Z resmoke.py invocation for local usage: buildscripts/resmoke.py run --suites=replica_sets_jscore_passthrough_last_lts_old_new_new --excludeWithAnyTags=multiversion_incompatible --dbpathPrefix=build/data/db jstests/core/read_after_optime.js
[resmoke] 11:56:10.630Z YAML configuration of suite replica_sets_jscore_passthrough_last_lts_old_new_new
test_kind: js_test
selector:
roots:
- jstests/core/read_after_optime.js
executor:
archive:
hooks:
- CheckReplDBHashInBackground
- ValidateCollectionsInBackground
- CheckReplDBHash
- CheckReplOplogs
- ValidateCollections
config:
shell_options:
eval: testingReplication = true;
fixture:
class: ReplicaSetFixture
mixed_bin_versions: old_new_new
mongod_options:
set_parameters:
enableTestCommands: 1
num_nodes: 3
old_bin_version: last_lts
hooks:
- class: CheckReplDBHashInBackground
- class: ValidateCollectionsInBackground
- class: CheckReplOplogs
- class: CheckReplDBHash
- class: ValidateCollections
- class: CleanEveryN
n: 20
logging:
executor:
format: '[%(name)s] %(asctime)s %(message)s'
handlers:
- class: logging.StreamHandler
fixture:
format: '[%(name)s] %(message)s'
handlers:
- class: logging.StreamHandler
tests:
format: '[%(name)s] %(message)s'
handlers:
- class: logging.StreamHandler
[executor] 11:56:10.631Z Skipping js_test, no tests to run
[resmoke] 11:56:10.631Z ================================================================================
[resmoke] 11:56:10.631Z Summary of replica_sets_jscore_passthrough_last_lts_old_new_new suite: No tests ran.
Suite did not run any tests.
[resmoke] 11:56:10.631Z Exiting with code: 0
And without passing --excludeWithAnyTags resmoke did run the test.
| Ok(Box::new(GeneratedResmokeSuite { | ||
| task_name: "burn_in_tests".to_string(), | ||
| sub_suites, | ||
| use_large_distro: false, |
There was a problem hiding this comment.
I assume use_large_distro: false is also keeping the same as it was before, right?
There was a problem hiding this comment.
I don't think burn_in has ever supported "use_large_distro", it just generate tasks on the default distro for the build variant. I believe there is a request on the backlog to start supporting large distros, but we would need to think through how best to do that.
|
evergreen merge |
ghost
merged commit 
No description provided.