Make schema object compatible with JSON Schema draft4

[IvanGoncharov]

According to spec swagger inside schema object support only limited subset of JSON Schema draft4 and extend it with custom keywords.

This isssue was hidden, before 68c6777 commit any swagger model with arbitrary JSON schema was valid(against Swagger schema).

These create a problem, because in reality most of tools use standard JSON schema validators(for e.g. swagger-tools use z-schema, swagger-express-middleware use tv4). And I didn't found any true Swagger 2.0 validator in the wild.

Another problem is that you can't reference existing schemes inside your swagger files.
It became real problem, if you want to use some standard scheme or simply reuse your existing validation scheme. For example if you make API that return swagger definitions and want to reuse existing schema, you can't simply use '$ref' or even copy-paste it.

Finally it brake compatibility with all tooling around JSON Schema not only validators but also documentation generators, UI form generators, etc.

On technical side, this is list of Draft4 keywords doesn't supported by Swagger: id, $schema, additionalItems, definitions, patternProperties, dependencies, anyOf, oneOf, not.

Also there are "discriminator" and "readOnly" Swagger specific keywords that don't supported by JSON Schema validators. "discriminator" could be safely substitute with "oneOf" construction, as for "readOnly" same behaviour could be achieved by using "allOf" with "write part" and "read-only part".
My proposal it either to remove these keywords or make them optional to support.

Also there are "safe" custom keywords namely "example" and "externalDocs", they shouldn't cause any serious trouble, but should be submitted as requests for Draft5 to ensure future compatibility.

Last keyword is "xml", I didn't make any research about it, so I can't say if it used in any Swagger related project or if it brake JSON schema compatibility.

[davesag]

Hi @IvanGoncharov Looks like we've been having similar issues. I'm having trouble getting documents that lack properties that are not actually required to be correctly validated. The Ruby JSON::Validator.validate_fully method takes a schema and a document and validates that the document conforms to the schema. That worked when we were using additionalProperties to define optional properties (until we realised that that's not what additionalProperties is for.) Also the Swagger Codegen ignores any property defined under additionalProperties which is either just a current limitation or a bug. see http://stackoverflow.com/questions/29714534/swagger-json-optional-fields-being-treated-as-required

[webron]

The existing JSON Schema validators out there should have no problem validating Swagger definitions as-is. Using a subset of properties doesn't take away from that functionality.

There are explicit reasons for most excluded keywords as to why they were excluded, though some excluded keywords may have been excluded by mistake. We will not be able to provide support for most these keywords without finding proper solutions for the problems they may cause.

JSON Schema is extensible to the user's needs. We chose to extend it with our own keywords to better support the use cases that we require. There's no technical problem there. If there's a problem with a specific validator, it's a bug with the validator. By definition, the validator should ignore any additional keyword that's use that it is not aware of.

The discriminator absolutely cannot be replaced by oneOf as it is required for proper polymorphism support. oneOf will not guarantee that and could break support for strongly-typed languages.

You are partially right about the readOnly, however it significantly reduces the complexity of the schema. If anything, we may extend readOnly in future versions.

Given all the above, it does not mean we will not provide support for whichever JSON Schema version is available when we work on the next Swagger specification version. We would have to be sure that we can provide end-to-end (for producers and consumers) which we couldn't do in the last version of the spec.

[IvanGoncharov]

The existing JSON Schema validators out there should have no problem validating Swagger definitions as-is.

I'm not speaking about validating Swagger definitions instead my concern is about request/reply body validation.

There are explicit reasons for most excluded keywords as to why they were excluded

Can you please post link or short description, because it really affects further discussion.
It especially interesting, because competitive standards(API Blueprint, RAML) fully support JSON Schema without any problems.

JSON Schema is extensible to the user's needs. We chose to extend it with our own keywords to better support the use cases that we require. There's no technical problem there. If there's a problem with a specific validator, it's a bug with the validator. By definition, the validator should ignore any additional keyword that's use that it is not aware of.

If we speak only about JSON Schema standard than your right, validator could simply ignore non-standard keywords.

From Swagger point of view situation is different, lets consider the following Pet model as an example. If you simply ignore discriminator keyword than you will validate all objects only against Pet schema.

The discriminator absolutely cannot be replaced by oneOf as it is required for proper polymorphism support.

discriminator implement so-called ad hoc polymorphism, that is why it can be replaced with oneOf.
For example, I removed discriminator from Pet model example.
Same aproach used in Swagger schema to distinguish between different parameter types based on value of in property.

We would have to be sure that we can provide end-to-end (for producers and consumers) which we couldn't do in the last version of the spec.

Right now I'm aware only of two types of "consumers", which could be affected by such change:

• request/reply validators is already broken
• documentation generators(for e.g. swagger-ui) is unaffected because simply display schema

Do you now about any other affected "consumers"?

[paulhill]

I'm with @IvanGoncharov. (with respect to allowing all of json-schema). It's been a PITA to have swagger fail the spec validation because we use the same json-schema for model definition as validation. It's "thou shalt not use unblessed attributes" approach seems unnecessarily restrictive.

We don't need to add full blown json-schema support to the spec definition.
That's not what this is about.

We should relax the spec validation to allow valid json-schema in the model definitions. I've never understood what the harm is. So I include valid json schema attributes swagger should just ignore, what's the big deal?

[fehguy]

Folks, I do understand the concern. Historically, however, we have had invalid swagger specifications from a variety of tooling providers, which has made it very difficult for end-users to know "what is compatible with what". One of the most requested, and frankly most important features in 2.0 is a strict schema that should tell you whether or not your swagger specification is compliant with the spec.

If for example we started ignoring definitions in the schema, it would be up to the end user (i.e. consumer) to know great detail of the swagger specification, including what is allowed vs. what is not before being confident that the definition would work with the tooling ecosystem. This has proven over and over again to hurt the consumers of the spec.

To all you to do "whatever you want" you should consider using vendor extensions. They can be easy to produce or possibly more complicated depending on what you're looking to do.

[paulhill]

So use vendor extensions to support existing json-schema constructs?

x-anyOf ?

That seems a little... awkward, don't you think?
Not at all supported by existing json-schema validators.
Re-implement json-schema validation anyone?

:-)

[IvanGoncharov]

@fehguy I think we divert from original problem.
It's about JSON Schema keywords and only in scope of Schema object.

[IvanGoncharov]

I think we should divide problem into three parts:

1. Support for oneOf and anyOf.
   As I understand main problem is codegen for strongly typed languages.
2. Rest of unsupported JSON keywords, such as: id, definitions, etc.
   I don't see any reason why they should be excluded from Swagger spec.
3. Swagger specific extensions: discriminator and readOnly.
   I done quick search in swagger-ui, swagger-codegen and swagger-js and it seems that no one is using them. IMHO, it's look like artifacts from 1.x, which no one really use.

@fehguy, Can you please comment?

[webron]

1. this is not an issue for the codegen only. This affects all the tooling.
2. Some of the missing keywords were missed unintentionally. That said, we cannot add them not. In a future version? Sure. As for the other ones, we need to cover them and see how we can add proper support for them. Like 1) above, it's easy to say we 'support' it, but if we don't provide support for it in the tools or unable to do so, we're providing poor service.
3. readOnly is being added to the codegen as we speak. The whole model inheritance is unfortunately underdeveloped in the tools and we're behind.

[noirbizarre]

Hi !

I'm just adding my 2 cents contribution as it's an important issue to me.

I second @IvanGoncharov , supporting the full JSON Schema v4 specs is essential, in my opinion, for many reason, mostly for being able to:

• use standards and reusables validators
• reference existing schemas
• use all the primitive types (lack of null is really problematic in many cases)
• describe ALL the API (legacy and new ones)

As I understand the main caveat is that existing tooling won't be able to implement all the specs.
In my opinion, the specs should allows to describe all the APIs, the tooling need to adapt as they can.
When Swagger 2.0 specifications were released, the tooling was not ready, but the specifications permitted to every developper behind a tool to make it evolve in few monthes.

I develop a tool too (flask-restplus) and the impossibility to use existing JSON schema validator is a real handicap because it means one of these:

• I have to develop a new validator from scratch (which I won't)
• I need to monkey patch an existing one, which is dirty and cost a lot to maintain

I don't know if this the place to debate about this, if there is already a dedicated issue or a dedicated topic on the mailing list, I'll join the existing debate.

[webron]

@noirbizarre - this is definitely the place to debate it. I'd just like to clarify that for 2.0, this will not change. I completely understand the problems this causes, but no matter which way we go, it will cause problems. I'd like to try and find a hopefully generic solution as part of this ticket when the time comes to be able to provide adequate support across the board. We should thrive to evolve, but do it in a thoughtful process.