proposal: spec: reduce error handling boilerplate using "? return" and "? err"

[apparentlymart]

The following is a language proposal that is strongly inspired by Ian's earlier proposal for error handling using ?, but with some slight modifications that I hope will help to address some of the most commonly-raised concerns in the discussion of Ian's proposal.

At this point Ian's proposal discussion has become very large and hard to consume for a newcomer, so for ease of consuming this the following are the specific complaints from that that I'm aiming to address here:

1. The shorthand ? syntax without an explicit exceptional block seems easy for a human to miss when trying to scan the body of a function to find all of its exit points.

   Making the error handling easier to ignore when you want to was one of the goals of the original proposal, and so this modified proposal aims to find a compromise that still pushes all of the error-handling content to the right but hopefully makes it still stand out a little more than in the original proposal: it now includes a mandatory return keyword for that case in addition to the question mark.

2. Ian's proposal included an implicit declaration of a local symbol called err in the exceptional block. However, the discussion also included a sub-thread with voting for two other possibilities: a predeclared identifier called erv, or an additional identifier before the block to declare the name explicitly.

   The second of those alternatives has a plurality of votes at the time I'm writing this, so I've adopted it here.

3. Ian's proposal discussion had a sub-thread for the question of whether it should be invalid for control to implicitly "fall through" the end of an exceptional block, and that idea seemed pretty popular based on the discussion and voting.

   This proposal adopts that requirement and makes a specific proposal for how it might be specified as a compile-time check.

As with Ian's discussion, let's please use this only to discuss advantages and disadvantages of this proposal and how they might be directly addressed by modifying this proposal, rather than discussing entirely different design ideas here. It's admittedly tough sometimes to differentiate between "new proposal" and "refinement of this proposal" but for the sake of this request I'd like to assert that any new idea that doesn't aim to meet all of the design goals mentioned throughout is off-topic for this discussion.

I have created this as an issue instead of a discussion because only the Go team can create new discussions, but I expect to quickly regret that decision since any proposal related to error handling tends to quickly attract a high volume of comments, many of which are redundant due to GitHub's poor handling of long issue discussions.

I know it's hard, but please try to make sure that any comment you are adding is adding new information that wasn't already covered by an earlier comment and could not be represented simply as an emoji reaction. In particular, "I think Go's current error handling is fine and no changes are needed" is best represented by voting 👎; no comment is needed because I am already very familiar with all of the objections of that kind from previous discussions. For what it's worth, I am also pretty skeptical that a change is needed, but I also acknowledge that the user study run by Todd Kulesza found that at least some newcomers stumble reading and understanding the current idiom.

I am nowhere near as familiar with the Go source code rewriting facilities as Ian, so I have not produced a tool to automatically implement these changes nor a copy of the standard library with the changes applied. For now I hope that this proposal is similar enough to Ian's that it's possible for a reader to mentally translate Ian's standard library patch series into the slightly-modified equivalents for this proposal. As before, let's please try to keep the discussion focused on real examples of code someone might want to write, rather than on highly contrived hypotheticals.

---

Background

The goal of this proposal is to introduce a new syntax that improves the readability and conciseness of code that checks errors, without obscuring control flow, and with an explicit distinction between mainline code that is "on the left" and exceptional code that is "on the right" or indented.

I readily acknowledge the various complaints in the earlier discussions asserting that Go's current error handling is fine already, or that returning errors without wrapping them is a "code smell", etc, but this proposal does not aim to address those concerns. If you did not like Ian's proposal for these reasons then this proposal does not substantially change those tradeoffs and so there is no need to repeat all of those objections here: a 👎 vote on this comment would suffice to represent that.

Refer to the previous proposal's discussion for more background information. This proposal is strongly inspired by Ian's and I have made only small changes to it here, based on the three specific concerns I enumerated above.

New syntax

This section is an informal description of the proposal, with examples. A more precise description appears below.

I propose permitting statements of the form:

	r, err := SomeFunction()
	if err != nil {
		return fmt.Errorf("something failed: %v", err)
	}

to be written as:

	r := SomeFunction() ? err {
		return fmt.Errorf("something failed: %v", err)
	}

The ? absorbs the error result of the function. It introduces a new exceptional block, which is executed if the error result is not nil. Within the new block, the identifier declared immediately after the ? token refers to the absorbed error result.

Similarly, statements of the form:

	if err := SomeFunction2(); err != nil {
		return fmt.Errorf("something else failed: %v", err)
	}

may be written as:

	r := SomeFunction() ? err {
		return fmt.Errorf("something failed: %v", err)
	}

Further, I propose a shorthand form which replaces the optional block and its declaration identifer with the return keyword. In this form it acts as though there were a block that simply returns the error from the function. For example, code like:

	if err := SomeFunction2(); err != nil {
		return err
	}

may in many cases be rewritten as:

	SomeFunction2() ? return

Formal proposal

This section presents the formal proposal.

An assignment or expression statement may be followed by a question mark (?). The question mark is a new syntactic element, the first permitted use of ? in Go outside of string and character constants. The ? causes conditional execution similar to an if statement.

A ? uses a value as described below, referred to here as the qvalue.

For a ? after an assignment statement, the qvalue is the last of the values produced by the right hand side of the assignment. The number of variables on the left hand side of the assignment must be one less than the number of values produced by the right hand side (the right hand side values may come from a function call as usual). It is not valid to use a ? if there is only one value on the right hand side of the assignment.

For a ? after an expression statement the qvalue is the last of the values of the expression. It is not valid to use a ? after an expression statement that has no values.

The qvalue must be of an interface type and must implement the predeclared type error. That is, it must have the method Error() string. In most cases it will simply be of type error.

A ? may be followed by either an identifier or by the return keyword. The return keyword is valid only if the statement using ? appears in the body of a function, and the enclosing function has at least one result, and the qvalue is assignable to the last result. (This means that the type of the last result must implement the predeclared type error, and will often simply be error.)

Execution of the ? depends on the qvalue. If the qvalue is nil, execution proceeds as normal, skipping over the exceptional block if there is one.

If the ? is followed by return, and the qvalue is not nil, then the function returns immediately. The qvalue is assigned to the final result. If the other results (if any) are named, they retain their current values. If they are not named, they are set to the zero value of their type. The results are then returned. Deferred functions are executed as usual.

If the ? is followed by an identifier -- the error variable name -- then the identifier must in turn by be followed by a block called the exceptional block. If the qvalue is not nil in this case then the block is executed. Within the block a new variable whose name matches the error variable name is declared, possibly shadowing a variable of the same name from an ancestor scope. The value and type of this variable will be those of the qvalue. The new variable is not available to the parent scope of the exception block.

It is a compile-time error if the exceptional block does not end with either a terminating statement or a fallthrough statement explicitly representing intent to continue execution with the first following after the block. Control flow statements inside the exceptional block behave equivalently to how they are specified in the current language specification with the exceptional block treated as the block from an if statement.¹

Discussion

This new syntax is partly inspired by Rust's question mark operator, though Rust permits ? to appear in the middle of an expression and does not support the optional block. Also, I am suggesting that gofmt will enforce a space before the ?, which doesn't is not how Rust is normally written. Rust's let .. else is similar to the form with an optional block, except that it does not allow the program to access the error value.

Absorbing the error returned by a function, and optionally returning automatically if the error is not nil, is similar to the earlier try proposal. However, it differs in that:

• ? is an explicit syntactic element, not a call to a predeclared function, and
• ? may only appear at the end of two specific kinds of statement, and not in the middle of an expression.

Declaring the error variable name

An identifier immediately after the ? symbol explicitly names the variable used to represent the error in the exceptional block. This is intentionally different to Ian's proposal which instead always implicitly chose the name err. Based on discussion from Ian's proposal, I hope that this one additional token remains true to the goal of reducing boilerplate while mitigating some of the concerns around implicit shadowing and of how code intelligence tools would interact with a local variable that has no explicit declaration site.

In practice, the variable will almost always be named err. However, authors working in codebases where shadowing is discouraged or prohibited may choose to use a different name whenever there is already a variable err in the parent scope, and in particular in the (hopefully-rare) situation where one exceptional block is nested inside another.

This proposal explicitly rejects Ian's alternative of a predeclared identifier named errval or erv due to concerns in the previous discussion that authors are likely to be confused by something which appears on initial inspection to be a local variable but actually behaves as a global, based in particular on experience with similar constructs in other languages such as JavaScript's this symbol. A local variable that is explicitly named in the program is a concept already used in many other parts of Go and so most likely to be understood by a newcomer.

I'm hopeful that even when using the common generic name err the presence of that name will help an unfamiliar reader who is already somehow familiar with that name being an abbreviation for "error" (either from previous Go experience or from experience in another language with a similar naming convention) to more quickly infer that this new language feature represents error handling.

Pros and cons

(For the advantages and disadvantages that are common between this proposal and Ian's previous proposal I have retained Ian's numbering just so that we might be able to cross-reference the feedback from the previous discussion that this proposal has or has not addressed. This means that there are some gaps in the numbering, which are intentional.)

Pros

• Advantage 1: Rewriting

  r, err := SomeFunction()
  if err != nil {
  	return fmt.Errorf("something failed: %v", err)
  }

  to

  r := SomeFunction() ? err {
  	return fmt.Errorf("something failed: %v", err)
  }

  reduces the error handling boilerplate from 9 tokens to 6, and 3 boilerplate lines to 2.

  Rewriting

  r, err := SomeFunction()
  if err != nil {
  	return err
  }

  to

  r := SomeFunction() ? return

  reduces boilerplate from 9 tokens to 2, and 3 boilerplate lines to 0.

• Advantage 2: This change turns the main code flow into a straight line with no intrusive if err != nil statements and no obscuring if v, err = F() { ... } statements. All error handling either disappears or is indented into a block.

• Advantage 3: That said, when a block is used the } remains on a line by tself, unindented, as a signal that something is happening. (This is also listed as a disadvantage below.)

• Advantage 4: Unlike the try proposal and some other earlier error handling proposals, there is no hidden control flow. The control flow is called out by an explicit ? symbol that can't be in the middle of an expression, followed by either a code block or a return keyword to make it more visible.

• Advantage 5: To some extent this reduces a couple of common error handling patterns to just one, as there is no need to decide between the if v, err = F(); err != nil form and the v, err := F(); if err != nil form.

  Instead, people can consistently write:

  v := F() ? err {
  	...
  }

• Advantage 6: Someone who is intending to focus only on the main code (the "happy path") and disregard the details of the error handling can more readily assume that a ? always diverts control flow somewhere other than the statement following the one containing ?, unless there is an exception block and it ends with the fallthrough keyword.

  Subjectively, this reduces the cognitive load of reviewing each error-handling branch at least enough to determine whether it diverts control flow, and thus the reader can assume that whatever represents "success" for the operation in question is definitely true when reviewing the following statements, at least for otherwise-linear code within a single block.

Cons

• Disadvantage 1: This is unlike existing languages, which may make it harder for newcomers to understand. As noted above, it is cosmetically similar to the Rust ? operator, but still different. However, it may not be too bad: Todd Kulesza ran a user study and discovered that people unfamiliar with the syntax were able to see that code from Ian's earlier proposal had to do with error handling, and this proposal only slightly modifies Ian's proposal.

• Disadvantage 2: The benefits of this proposal are not so clear in more complicated situations where the error must be stored somewhere other than an exception-block-scoped local variable.

  For example, in fmt/scan.go:

  for n = 1; !utf8.FullRune(r.buf[:n]); n++ {
  	r.buf[n], err = r.readByte()
  	if err != nil {
  		if err == io.EOF {
  			err = nil // must change outer err
  			break
  		}
  		return
  	}
  }
  // code that later returns err

  In this example the assignment err = nil must change the err variable that exists outside of the for loop. Using the ? symbol would force declaring a new local variable inside the block that is separate from the outer one, and a newcomer naively applying the usual pattern would probably use err for their error variable name and inadvertently shadow the err from the containing scope.

  To successfully rewrite this example the author must choose a different name for either the parent scope's error or the exception block's error:

  for n = 1; !utf8.FullRune(r.buf[:n]); n++ {
  	r.buf[n] = r.readByte() ? readErr {
  		if readErr == io.EOF {
  			err = nil // changes the outer err
  			break
  		}
  		err = readErr // changes the outer err
  		return
  	}
  }
  // code that later returns err

  (Note for this comparing this to Ian's earlier proposal: the new ability to explicitly select a variable name for the exception block has at least introduced a potential workaround that doesn't involve choosing a less-conventional name for a named return value that might be part of a function's documented signature.)

• Disadvantage 3: When using an exception block, the } remains on a line by itself, taking up vertical space as pure boilerplate. (This is also listed as an advantage above.)

• (Disadvantages 4 and 5 from Ian's proposal do not seem to apply to this one. They both related to concerns caused by the exception block being optional, but the requirement for the return keyword in the shorthand case both avoids the accidental addition of a newline changing a valid program into another valid program with a different meaning, and hopefully makes the shorthand case more visible on the page by allowing editors/etc to highlight it as an already-familiar keyword.)

• Disadvantage 6: This proposal has no support for chaining function calls, as in F().G().H(), where F and G also have an error result.

• Disadvantage 7: This proposal makes it easier to simply return an error than to annotate the error, by using the ? return form with no exception block. This may encourage programmers to skip error annotations even when they are desirable.

  (Some in the previous discussion asserted that error annotations are always desirable, and that a lack of annotation is always an antipattern or code-smell. I find that framing too reductive and think this tradeoff has considerably more nuance, but I acknowledge and respect that others differ.)

• Disadvantage 8: We really only get one chance to change error handling syntax in Go. We aren't going to make a second change that touches 1.5% of the lines of existing Go code. Is this proposal the best that we can do?

• Disadvantage 9: We don't actually need to make any changes to error handling. Although it is a common complaint about Go, it's clear that Go is usable today. Perhaps no change is better than this change. Perhaps no change is better than any change.

• Disadvantage 10: As compared to Ian's previous proposal, the requirement that the end of the exception block be unreachable requires a redundant control flow statement in any situation involving a function that is documented not to return, because the Go language spec does not currently include any concept of a function that is guaranteed not to return.

  For example, in test code using t.Fatalf:

  SomethingUnderTest() ? err {
      // Fatalf is documented to prevent continued
      // execution of the subsequent statements...
      t.Fatalf("unexpected error: %s", err)
      // ...but the compiler doesn't know that, so
      // an unreachable-in-practice terminating
      // statement must appear to convince the
      // compiler of correctness.
      return
  }

  Under this proposal, if that situation were to become a concern then a separate proposal would need to somehow solve for the compiler recognizing calls to t.Fatalf (and many other functions with this characteristic, like os.Exit and log.Fatal) as another kind of terminating statement, which implies a rather complicated interaction between library code and the language specification that is perhaps unlikely to be accepted².

  (Ian proposed addressing this by handling this problem at runtime rather than compile time, such as by panicking if control reaches the end of an exceptional block. I am concerned that an automatic panic in a codepath already associated with error handling will mislead newcomers that this automatic panic is a desirable way to respond to the error, particularly if their previous experience is with languages that use structured exception handling as the primary error handling concept.)

Transition

If we adopt this proposal, we should provide tools that can be used to automatically rewrite existing Go code into the new syntax. Not everyone will want to run such a tool, but many people will. Using such a tool will encourage Go code to continue to look the same in different projects, rather than taking different approaches. This tool can't be gofmt, as correct handling requires type checking which gofmt does not do. It could be an updated version of go fix. See also modernizers.

We will need to update the go/ast package to support the use of ?, and we will need to update all packages that use go/ast to support the new syntax. That is a lot of packages.

We will also need to update the introductory documentation and the tour, and existing Go books will be out of date and will need updating by their authors. The change to the language and the compiler is likely to be the easiest part of the work.

Footnotes

1. I recall that there is an existing proposal somewhere for a shorthand form of return that allows automatically zeroing all but the final result values of the function, using syntax like return ..., err.

   Although that is not a part of this proposal, I think it would complement this proposal well by making an error-wrapping exceptional block focus only on the error and not include distracting expressions for the other results:

   MightFail() ? err {
       return ..., fmt.Errorf("while doing something: %w", err)
   }
   

   ↩

2. The earlier proposal https://github.com/golang/go/issues/69591 was declined, but it remains to be seen whether this proposal would constitute new information that would warrant revisiting that decision. I've opened https://github.com/golang/go/issues/71553 to find out. ↩

[gabyhelp]

Related Issues

• proposal: spec: reduce error handling boilerplate using ? #71203 (closed)
• proposal: spec: conditional return with return ?err? #70170 (closed)
• proposal: Go 2 : add support for conditional statements #25582 (closed)
• proposal: Go 2: `? :` for returning on function errors #65579 (closed)
• proposal: Go 2: new syntax for abbreviation of `if err != nil { return ..., err }` #51146 (closed)
• proposal: Go 2: Error Operator #65184 (closed)
• proposal: Go 2: Better way for error handling #64674 (closed)
• proposal: Go 2: add ? or ?() to the golang to handle the err return #66309 (closed)
• proposal: Go2: error handling shortening #62378 (closed)

Related Discussions

• discussion: spec: reduce error handling boilerplate using ?

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[flibustenet]

In Advantage 1. I believe you forgot return

r := SomeFunction() ? return

In Advantage 5. The err

v := F() ? err {
	...
}

[aksdb]

Since the usage of ? would be optional ("expression statement may be followed by a question mark"), why is Disadvantage 2 a disadvantage in the first place? Wouldn't this simply be a case where you would not use the short-cut but actually keep treating err as a variable like always? I would even consider this an advantage of this proposal; you can keep using the return values individually and don't have to treat errors specially.

It is a disadvantage for the transition though. Because if the intention is to apply this automatically to as many places as possible (gofix), then this tool would need to be able to discern those cases when it can safely simplify and when it needs to keep it untouched. Although I assume it would already help a lot if it finds the common cases of if err != nil { return fmt.Errorf(..., err) }

[jwebb]

Could you clarify if this would be permitted only at statement level, or could it be used inside expressions? E.g.

x := &SomeStruct{
    Field1: calc1() ? return,
    Field2: calc2() ? return,
}

For me, it's this sort of non-side-effecting but possibly failing logic where the current approach to error handling feels most egregiously verbose. To write the above currently, I need to either introduce variables duplicating the fields, or build the struct incrementally (increasing the risk of missing something if it grows more fields in future).

[apparentlymart]

@aksdb wrote:

Since the usage of ? would be optional ("expression statement may be followed by a question mark"), why is Disadvantage 2 a disadvantage in the first place? [...]

You are correct that not rewriting this at all remains an option. I included that disadvantage mainly because Ian's proposal had a similar disadvantage and since I didn't fully solve it with my small changes it seemed dishonest to delete it entirely.

With that said, I expect that in a hypothetical future world where documentation and books have all been updated to demonstrate this new approach as the primary/default way to handle errors that a newcomer would first try to write it with ? and would find that in a case like this the typical idiom doesn't work and they would then need to adapt. Whether they would adapt by renaming the local variable or by eschewing ? altogether is a fair question that I don't think I can answer, but you are right that both would be valid solutions under this proposal. (and that an automatic rewriting tool would need some sort of opinion about this embedded in it, which indeed might make some existing code worse if not designed carefully).

@jwebb wrote:

Could you clarify if this would be permitted only at statement level, or could it be used inside expressions?

As with Ian's proposal, this is allowed only at the very end of assignment expressions and expression statements. It is not an expression operator and is not valid to use within a nested expression. For your example, it would still be necessary to use some temporary variables from earlier statements that handle the errors:

field1 := calc1() ? return
field2 := calc2() ? return
x := &SomeStruct{
    Field1: field1
    Field2: field2,
}

The ability for something in the middle of an expression to cause control flow other than returning a value or panicking was one of the repeated concerns in previous error handling proposals, and so Ian's proposal aimed to avoid that by restricting it to the whole-statement level and my modifications further reinforce the idea of "predictable control flow" by promising that the exceptional block will definitely prevent sequential execution of the following statement unless clearly marked with a fallthrough statement. The overall goal in both cases is for the control flow to be highly visible on the page, but to also be predictable enough that a reader can safely make some assumptions about it when focusing primarily on the happy path.

[ncruces]

These are, IMO, all improvements over Ian's proposal, if the scope of the proposal is kept the same.

If the argument wins out that return is still too much noise compared with the optional block, I think I'd rather reduce scope, and adopt just the naked ? and forgo the block altogether.

All in all, IMO, this is the best version of Ian's proposal.

[chad-bekmezian-snap]

Seems like this proposal has reduced syntax as much as possible without creating an entirely unfamiliar experience in Go. It's still explicit enough that it's very easy to read the syntax, even for those that are unfamiliar. That being said, I still struggle with this and Ian's proposal in that, aside from reducing verbosity, I don't really feel it does a ton for improving error handling in Go in general.

[ianlancetaylor]

I'm somewhat concerned that if the syntax is ? return, it's a little hard to explain why we can't say the values that we aren't going return. And, of course, the reason for that is that we can't name the error value, so people need to instead write ? err { return fmt.Errorf("got error %v", err) }. It's going to be a constant question for newcomers to the language. But probably we can live with this approach, though I still think that the plain ? is OK.

I definitely concerned that the block after the ? must end with a terminating statement. Looking over at https://go.dev/cl/644076, for the #71460 syntax, the majority of the _test.go files have code like this:

	os.Symlink(target, link) ? {
		t.Fatal(err)
	}

With this proposal, that presumably would be written as

	os.Symlink(target, link) ? err {
		t.Fatal(err)
		panic("unreachable")
	}

That additional panic is fairly annoying, and it's only there to satisfy a compiler requirement. We would actually be better off with the original code:

	if err := os.Symlink(target, link); err != nil {
		t.Fatal(err)
	}

While I'm not personally fond of ? err {, I could get on board with that. But I don't think I can get on board with requiring that the compiler know that the ? block terminates.

[chad-bekmezian-snap]

I'm somewhat concerned that if the syntax is ? return, it's a little hard to explain why we can't say the values that we aren't going return. And, of course, the reason for that is that we can't name the error value, so people need to instead write ? err { return fmt.Errorf("got error %v", err) }. It's going to be a constant question for newcomers to the language. But probably we can live with this approach, though I still think that the plain ? is OK.

I thought that return values were just omitted from the example for brevity. I completely agree. If this syntax doesn't require specifying returned values e.g. return arg1, err or naked returns, I see now benefit in having a return included at all. I only see that as confusing, and would prefer the plain ?.