Skip to content

Allow specifying whether to make file paths absolute when parsing configs. #5805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

gsquared94
Copy link
Contributor

@gsquared94 gsquared94 commented May 7, 2021

Current behavior of the config parser is to preserve the original filepaths specified in the root skaffold.yaml file while setting all the imported configs' filepaths to absolute values immediately.
#5791 introduced the setting MakePathsAbsolute to make the parser substitute absolute file paths for all configs (including the root configs).
This PR changes the behavior of MakePathsAbsolute to allow specifying not to set absolute file paths for any config.

So, now the parser can:

  • Set absolute paths in imported configs, not in root configs.
    • This is required to keep backwards compatibility with the assumption that the filepaths in the root skaffold.yaml need to be relative to CWD where skaffold is invoked, and not the directory of the skaffold.yaml file.
  • Set absolute paths in all configs.
    • This is required for commands like skaffold diagnose that take a snapshot of the entire set of skaffold configs that can then be invoked from anywhere.
  • Preserve existing paths in all configs.
    • (This is new) For skaffold inspect commands that will recursively modify and write out the set of skaffold configs, it needs to preserve the original relative file paths specified.

@gsquared94 gsquared94 requested a review from nkubala May 7, 2021 21:04
@gsquared94 gsquared94 requested a review from a team as a code owner May 7, 2021 21:04
@google-cla google-cla bot added the cla: yes label May 7, 2021
@codecov
Copy link

codecov bot commented May 7, 2021

Codecov Report

Merging #5805 (e7bc70d) into master (8efc9ef) will decrease coverage by 0.01%.
The diff coverage is 66.66%.

❗ Current head e7bc70d differs from pull request most recent head cd0de04. Consider uploading reports for the commit cd0de04 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5805      +/-   ##
==========================================
- Coverage   71.00%   70.99%   -0.02%     
==========================================
  Files         438      439       +1     
  Lines       16497    16499       +2     
==========================================
- Hits        11714    11713       -1     
- Misses       3921     3923       +2     
- Partials      862      863       +1     
Impacted Files Coverage Δ
pkg/skaffold/config/options.go 100.00% <ø> (ø)
pkg/skaffold/parser/config.go 78.32% <62.50%> (-1.25%) ⬇️
cmd/skaffold/app/cmd/diagnose.go 64.51% <100.00%> (ø)
pkg/skaffold/deploy/deploy_problems.go 80.00% <0.00%> (-2.36%) ⬇️
pkg/skaffold/build/build_problems.go 94.64% <0.00%> (-0.19%) ⬇️
pkg/skaffold/errors/errors.go 95.31% <0.00%> (-0.15%) ⬇️
pkg/skaffold/initializer/init_problems.go 100.00% <0.00%> (ø)
pkg/skaffold/errors/problem_catalog.go 75.00% <0.00%> (ø)
pkg/skaffold/errors/problem.go 91.30% <0.00%> (+8.69%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb9c352...cd0de04. Read the comment docs.

@tejal29
Copy link
Contributor

tejal29 commented May 7, 2021

From the description, if setting absolute paths in all configs is only required in skaffold diagnose mode, then why not just resolve the paths just before printing out in doDiagnose

Copy link
Contributor

@nkubala nkubala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

}
// if current config is either a URL or STDIN, base path should be CWD
if util.IsURL(currentConfigPath) || currentConfigPath == "-" {
cwd, _ := util.RealWorkDir()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a comment on why we can safely ignore the error here?

Copy link
Contributor

@tejal29 tejal29 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering, if we could write something as
yaml.MarshalWithSeparatorWithAbsolutePaths using the yaml.processTags kind of utility.

@@ -91,6 +90,7 @@ type SkaffoldOptions struct {
RPCPort int
RPCHTTPPort int
BuildConcurrency int
MakePathsAbsolute *bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make it this a *bool? Would opts. MakePathsAbsolute false not suffice?

This is not a command line flag, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering, if we could write something as
yaml.MarshalWithSeparatorWithAbsolutePaths using the yaml.processTags kind of utility.

We actually need absolute path substitution for all real working modes like dev, debug, build and deploy. So it's not something that can be done only while writing output. The reverse might have worked MarshalWithSeparatorWithOriginalPaths but we don't retain the original paths anywhere, so even that can't be done.

Copy link
Contributor Author

@gsquared94 gsquared94 May 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make it this a *bool? Would opts. MakePathsAbsolute false not suffice?

I have 3 modes of parsing (see description). So nillable boolean nil, true, and false works out. Otherwise I can use an explicit struct for parsingMode.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, true is set in diagnose workflow. Its nil by default.
When is opts. MakePathsAbsolute set to false? - Do you plan to set it to false in inspect workflow?

Copy link
Contributor Author

@gsquared94 gsquared94 May 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it'll be set to false for skaffold inspect commands. I'm making this change to have a single parser handle all cases and not duplicate config parsing code.

@gsquared94 gsquared94 requested a review from tejal29 May 7, 2021 22:42
@gsquared94
Copy link
Contributor Author

From the description, if setting absolute paths in all configs is only required in skaffold diagnose mode, then why not just resolve the paths just before printing out in doDiagnose

We could do that. But I think having the setting in the parser is more generic and can be used elsewhere if the need arises in the future.

@tejal29
Copy link
Contributor

tejal29 commented May 7, 2021

From the description, if setting absolute paths in all configs is only required in skaffold diagnose mode, then why not just resolve the paths just before printing out in doDiagnose

We could do that. But I think having the setting in the parser is more generic and can be used elsewhere if the need arises in the future.

I would argue we are adding complexity to the code in present thinking about use case in future.
The code already exists.
If its not a huge effort, my vote is for removing opts.MakeAbsolutePaths and the if-else branches and instead move the logic to doDiagnose

@gsquared94 gsquared94 merged commit 22c0fe2 into GoogleContainerTools:master May 10, 2021
@gsquared94
Copy link
Contributor Author

will address requested changes around code complexity in follow up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants