Tweaked overload conformance tests to permit errors to appear in a few additional places (#1956)

Tweaked overload conformance tests to permit errors to appear in a few additional places.
Updated conformance results for pyright, which passes the overload conformance tests in the latest version.

Author: erictraut

conformance/results/mypy/overloads_definitions.toml

@@ -5,20 +5,20 @@ Does not allow an overload with no implementation in an abstract base class.
 Allows @override to be on all overloads and implementation, instead of just implementation.
 """
 errors_diff = """
-Line 245: Expected 1 errors
-Line 49: Unexpected errors ['overloads_definitions.py:49: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]']
+Lines 220, 222, 226: Expected error (tag 'override_impl')
+Line 47: Unexpected errors ['overloads_definitions.py:47: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]']
 """
 output = """
-overloads_definitions.py:14: error: Single overload definition, multiple required  [misc]
-overloads_definitions.py:26: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
-overloads_definitions.py:49: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
-overloads_definitions.py:63: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
-overloads_definitions.py:78: error: Overload does not consistently use the "@staticmethod" decorator on all function signatures.  [misc]
-overloads_definitions.py:88: error: Overloaded function implementation does not accept all possible arguments of signature 1  [misc]
-overloads_definitions.py:88: error: Overloaded function implementation does not accept all possible arguments of signature 2  [misc]
-overloads_definitions.py:91: error: Overload does not consistently use the "@classmethod" decorator on all function signatures.  [misc]
-overloads_definitions.py:133: error: @final should be applied only to overload implementation  [misc]
-overloads_definitions.py:148: error: @final should be applied only to overload implementation  [misc]
-overloads_definitions.py:196: error: Cannot override final attribute "final_method" (previously declared in base class "Base")  [misc]
-overloads_definitions.py:211: error: Method "bad_override" is marked as an override, but no base method was found with this name  [misc]
+overloads_definitions.py:15: error: Single overload definition, multiple required  [misc]
+overloads_definitions.py:27: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
+overloads_definitions.py:47: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
+overloads_definitions.py:58: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
+overloads_definitions.py:71: error: Overload does not consistently use the "@staticmethod" decorator on all function signatures.  [misc]
+overloads_definitions.py:81: error: Overloaded function implementation does not accept all possible arguments of signature 1  [misc]
+overloads_definitions.py:81: error: Overloaded function implementation does not accept all possible arguments of signature 2  [misc]
+overloads_definitions.py:84: error: Overload does not consistently use the "@classmethod" decorator on all function signatures.  [misc]
+overloads_definitions.py:121: error: @final should be applied only to overload implementation  [misc]
+overloads_definitions.py:135: error: @final should be applied only to overload implementation  [misc]
+overloads_definitions.py:175: error: Cannot override final attribute "final_method" (previously declared in base class "Base")  [misc]
+overloads_definitions.py:190: error: Method "bad_override" is marked as an override, but no base method was found with this name  [misc]
 """

conformance/results/mypy/overloads_definitions_stub.toml

@@ -4,15 +4,15 @@ Allows @override to appear in a stub file not on the first overload.
 """
 conformance_automated = "Fail"
 errors_diff = """
-Line 168: Expected 1 errors
+Lines 143, 147, 149: Expected error (tag 'override_impl')
 """
 output = """
 overloads_definitions_stub.pyi:13: error: Single overload definition, multiple required  [misc]
-overloads_definitions_stub.pyi:37: error: Overload does not consistently use the "@staticmethod" decorator on all function signatures.  [misc]
-overloads_definitions_stub.pyi:38: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
-overloads_definitions_stub.pyi:46: error: Overload does not consistently use the "@classmethod" decorator on all function signatures.  [misc]
-overloads_definitions_stub.pyi:85: error: In a stub file @final must be applied only to the first overload  [misc]
-overloads_definitions_stub.pyi:101: error: In a stub file @final must be applied only to the first overload  [misc]
-overloads_definitions_stub.pyi:128: error: Cannot override final attribute "final_method" (previously declared in base class "Base")  [misc]
-overloads_definitions_stub.pyi:140: error: Method "bad_override" is marked as an override, but no base method was found with this name  [misc]
+overloads_definitions_stub.pyi:32: error: Overload does not consistently use the "@staticmethod" decorator on all function signatures.  [misc]
+overloads_definitions_stub.pyi:33: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
+overloads_definitions_stub.pyi:39: error: Overload does not consistently use the "@classmethod" decorator on all function signatures.  [misc]
+overloads_definitions_stub.pyi:71: error: In a stub file @final must be applied only to the first overload  [misc]
+overloads_definitions_stub.pyi:84: error: In a stub file @final must be applied only to the first overload  [misc]
+overloads_definitions_stub.pyi:107: error: Cannot override final attribute "final_method" (previously declared in base class "Base")  [misc]
+overloads_definitions_stub.pyi:120: error: Method "bad_override" is marked as an override, but no base method was found with this name  [misc]
 """

conformance/results/pyre/overloads_definitions.toml

@@ -5,28 +5,28 @@ Does not allow an overload with no implementation in a Protocol or an abstract b
 Expects @final/@override on all overloads and implementation, instead of implementation only.
 """
 errors_diff = """
-Line 245: Expected 1 errors
-Lines 148, 150: Expected error (tag 'invalid_final_2')
+Lines 135, 137, 142: Expected error (tag 'invalid_final_2')
+Lines 220, 222, 226: Expected error (tag 'override_impl')
 Line 40: Unexpected errors ['overloads_definitions.py:40:4 Missing overload implementation [42]: Overloaded function `MyProto.func3` must have an implementation.']
-Line 51: Unexpected errors ['overloads_definitions.py:51:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.func4` must have an implementation.']
-Line 128: Unexpected errors ['overloads_definitions.py:128:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
-Line 239: Unexpected errors ['overloads_definitions.py:239:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Line 49: Unexpected errors ['overloads_definitions.py:49:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.func4` must have an implementation.']
+Line 117: Unexpected errors ['overloads_definitions.py:117:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Line 215: Unexpected errors ['overloads_definitions.py:215:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
 """
 output = """
-overloads_definitions.py:15:0 Incompatible overload [43]: At least two overload signatures must be present.
-overloads_definitions.py:27:0 Missing overload implementation [42]: Overloaded function `func2` must have an implementation.
+overloads_definitions.py:16:0 Incompatible overload [43]: At least two overload signatures must be present.
+overloads_definitions.py:28:0 Missing overload implementation [42]: Overloaded function `func2` must have an implementation.
 overloads_definitions.py:40:4 Missing overload implementation [42]: Overloaded function `MyProto.func3` must have an implementation.
-overloads_definitions.py:51:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.func4` must have an implementation.
-overloads_definitions.py:64:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.not_abstract` must have an implementation.
-overloads_definitions.py:80:4 Incompatible overload [43]: The implementation of `C.func5` does not accept all possible arguments of overload defined on line `80`.
-overloads_definitions.py:85:4 Incompatible overload [43]: The implementation of `C.func5` does not accept all possible arguments of overload defined on line `85`.
-overloads_definitions.py:88:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions.py:97:4 Incompatible overload [43]: The implementation of `C.func6` does not accept all possible arguments of overload defined on line `97`.
-overloads_definitions.py:97:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions.py:128:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions.py:139:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions.py:204:4 Invalid override [40]: `overloads_definitions.Child.final_method` cannot override final method defined in `Base`.
-overloads_definitions.py:220:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions.py:220:4 Invalid override [40]: `overloads_definitions.Child.bad_override` is decorated with @override, but no method of the same name exists in superclasses of `Child`.
-overloads_definitions.py:239:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:49:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.func4` must have an implementation.
+overloads_definitions.py:59:4 Missing overload implementation [42]: Overloaded function `MyAbstractBase.not_abstract` must have an implementation.
+overloads_definitions.py:73:4 Incompatible overload [43]: The implementation of `C.func5` does not accept all possible arguments of overload defined on line `73`.
+overloads_definitions.py:78:4 Incompatible overload [43]: The implementation of `C.func5` does not accept all possible arguments of overload defined on line `78`.
+overloads_definitions.py:81:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:90:4 Incompatible overload [43]: The implementation of `C.func6` does not accept all possible arguments of overload defined on line `90`.
+overloads_definitions.py:90:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:117:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:127:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:181:4 Invalid override [40]: `overloads_definitions.Child.final_method` cannot override final method defined in `Base`.
+overloads_definitions.py:198:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions.py:198:4 Invalid override [40]: `overloads_definitions.Child.bad_override` is decorated with @override, but no method of the same name exists in superclasses of `Child`.
+overloads_definitions.py:215:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
 """

conformance/results/pyre/overloads_definitions_stub.toml

@@ -4,24 +4,22 @@ Expects @final and @override to be present on all overloads, not just first.
 """
 conformance_automated = "Fail"
 errors_diff = """
-Line 168: Expected 1 errors
-Lines 80, 82, 85, 87: Expected error (tag 'invalid_final')
-Lines 96, 98, 101: Expected error (tag 'invalid_final_2')
-Lines 122, 128, 129, 133: Expected error (tag 'override-final')
-Line 75: Unexpected errors ['overloads_definitions_stub.pyi:75:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
-Line 91: Unexpected errors ['overloads_definitions_stub.pyi:91:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
-Line 146: Unexpected errors ['overloads_definitions_stub.pyi:146:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
-Line 162: Unexpected errors ['overloads_definitions_stub.pyi:162:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
-Line 174: Unexpected errors ['overloads_definitions_stub.pyi:174:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Lines 67, 69, 71, 73: Expected error (tag 'invalid_final')
+Lines 80, 82, 84, 86: Expected error (tag 'invalid_final_2')
+Lines 102, 107, 108, 111, 113: Expected error (tag 'override-final')
+Line 63: Unexpected errors ['overloads_definitions_stub.pyi:63:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Line 76: Unexpected errors ['overloads_definitions_stub.pyi:76:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Line 125: Unexpected errors ['overloads_definitions_stub.pyi:125:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
+Line 138: Unexpected errors ['overloads_definitions_stub.pyi:138:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
 """
 output = """
 overloads_definitions_stub.pyi:14:0 Incompatible overload [43]: At least two overload signatures must be present.
-overloads_definitions_stub.pyi:43:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:52:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:75:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:91:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:142:4 Invalid override [40]: `overloads_definitions_stub.Child.bad_override` is decorated with @override, but no method of the same name exists in superclasses of `Child`.
-overloads_definitions_stub.pyi:146:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:162:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
-overloads_definitions_stub.pyi:174:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:37:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:44:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:63:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:76:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:122:4 Invalid override [40]: `overloads_definitions_stub.Child.bad_override` is decorated with @override, but no method of the same name exists in superclasses of `Child`.
+overloads_definitions_stub.pyi:125:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:138:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
+overloads_definitions_stub.pyi:147:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).
 """

conformance/results/pyre/version.toml

@@ -1,2 +1,2 @@
 version = "pyre 0.9.23"
-test_duration = 7.4
+test_duration = 6.3