[Bug]: Multiple openai endpoint Missing Content-Type Header

[tarukumar]

Your current environment

Not reletaed

🐛 Describe the bug

During property-based testing of the vLLM API, defined by an OpenAPI 3.1 schema, we observed a recurring issue with missing Content-Type headers across several endpoints.

1. GET /health:
Issue: Missing Content-Type header

Details: The test failed because the server expects a Content-Type: application/json header, but it was not included in the request.


2. GET /ping:
Issue: Missing Content-Type header

Details: Similar to the previous failure, the Content-Type: application/json header was missing for the request to /ping.



3. POST /ping:
Issue: Missing Content-Type header

Details: The failure is due to the absence of the Content-Type: application/json header in the POST request to /ping.

logs:

test_new.py u,uuuu,,uu,,,,u,,,u.                                                                                                                                                                                                       [100%]

================================================================================================================== FAILURES ==================================================================================================================
_______________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='GET /health') ________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X GET -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/health
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='GET /ping') _________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X GET -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/ping
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /ping') ________________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Missing Content-Type header
E       
E           The following media types are documented in the schema:
E           - `application/json`
E       
E       [200] OK:
E       
E           <EMPTY>
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/ping
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

______________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /tokenize') ______________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'messages': []})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200, 422
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' -d '{"messages": []}' --insecure http://127.0.0.1:8080/tokenize
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

_____________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /detokenize') _____________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'tokens': [-1]})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200, 422
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' -d '{"tokens": [-1]}' --insecure http://127.0.0.1:8080/detokenize
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/chat/completions') _________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:
E   schemathesis.exceptions.CheckFailed: Schemathesis found 2 distinct sets of failures.
E   ==================== 
E   
E   1. Server error
E   
E   2. Undocumented HTTP status code
E   
E       Received: 500
E       Documented: 200, 422
E   
E   [500] Internal Server Error:
E   
E       `Internal Server Error`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"messages": []}' --insecure http://127.0.0.1:8080/v1/chat/completions
E   
E   ==================== 
E   
E   1. Undocumented HTTP status code
E   
E       Received: 400
E       Documented: 200, 422
E   
E   [400] Bad Request:
E   
E       `{"object":"error","message":"list object has no element 0","type":"BadRequestError","param":null,"code":400}`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"messages": [{"role": "assistant"}]}' --insecure http://127.0.0.1:8080/v1/chat/completions

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

___________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/completions') ___________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:
E   schemathesis.exceptions.CheckFailed: Schemathesis found 2 distinct sets of failures.
E   ==================== 
E   
E   1. Undocumented HTTP status code
E   
E       Received: 400
E       Documented: 200, 422
E   
E   [400] Bad Request:
E   
E       `{"object":"error","message":"please provide at least one prompt","type":"BadRequestError","param":null,"code":400}`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"prompt": []}' --insecure http://127.0.0.1:8080/v1/completions
E   
E   ==================== 
E   
E   1. Server error
E   
E   2. Undocumented HTTP status code
E   
E       Received: 500
E       Documented: 200, 422
E   
E   [500] Internal Server Error:
E   
E       `Internal Server Error`
E   
E   Reproduce with: 
E   
E       curl -X POST -H 'Content-Type: application/json' -d '{"prompt": [-1]}' --insecure http://127.0.0.1:8080/v1/completions

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

______________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /v1/audio/transcriptions') _______________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'}, body={'file': Binary(data=b'')})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Undocumented HTTP status code
E       
E           Received: 400
E           Documented: 200, 422
E       
E       [400] Bad Request:
E       
E           `{"object":"error","message":"[{'type': 'value_error', 'loc': ('body', 'file'), 'msg': \"Value error, Expected UploadFile, received: <class 'str'>\", 'input': \"b''\", 'ctx': {'error': ValueError(\"Expected UploadFile, received: <class 'str'>\")}}]","type":"BadRequestError","param":null,"code":400}`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' -d file=b%27%27 --insecure http://127.0.0.1:8080/v1/audio/transcriptions
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

____________________________________________________________________________________ test_openapi_schema_with_fixture (verbose_name='POST /invocations') _____________________________________________________________________________________

    @wraps(test)
>   def test_function(*args: Any, **kwargs: Any) -> Any:

dsd/lib64/python3.12/site-packages/schemathesis/_hypothesis.py:83: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

case = Case(headers={'Content-Type': 'application/json'})

    @schema.parametrize()
    def test_openapi_schema_with_fixture(case):
        case.headers = {
            "Content-Type": "application/json",
        }
>       case.call_and_validate(verify=False)
E       schemathesis.exceptions.CheckFailed: 
E       
E       1. Server error
E       
E       2. Undocumented HTTP status code
E       
E           Received: 500
E           Documented: 200
E       
E       [500] Internal Server Error:
E       
E           `Internal Server Error`
E       
E       Reproduce with: 
E       
E           curl -X POST -H 'Content-Type: application/json' --insecure http://127.0.0.1:8080/invocations
E       
E       Falsifying example: test_openapi_schema_with_fixture(
E           case=,
E       )

test_new.py:46: CheckFailed
----------------------------------------------------------------------------------------------------------- Captured stdout setup ------------------------------------------------------------------------------------------------------------
http://127.0.0.1:8080
------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------

============================================================================================================== warnings summary ==============================================================================================================
dsd/lib64/python3.12/site-packages/schemathesis/generation/coverage.py:305
  /home/takumar/Downloads/dsd/lib64/python3.12/site-packages/schemathesis/generation/coverage.py:305: DeprecationWarning: jsonschema.exceptions.RefResolutionError is deprecated as of version 4.18.0. If you wish to catch potential reference resolution errors, directly catch referencing.exceptions.Unresolvable.
    ref_error: type[Exception] = jsonschema.RefResolutionError,

dsd/lib64/python3.12/site-packages/schemathesis/internal/deprecation.py:6
  /home/takumar/Downloads/dsd/lib64/python3.12/site-packages/schemathesis/internal/deprecation.py:6: DeprecationWarning: Argument `method` is deprecated and will be removed in Schemathesis 4.0. Use `include` and `exclude` methods instead.
    warnings.warn(

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[KubeKyrie]

Could I work on this? thanks