Merge branch 'master' into stability-indexing

* master:
  resize lightbow logo
  add lightbow logo
  add Lightbow as sponsor (#2822)
  [docs] Document new templating engine, adding ability to reference templates by file to comlete the example (#2773)
  [JAVA][KOTLIN] fix var Naming for all uppercase with Numbers (#2794)
  [Documentation] Add instructions to build the javascript client module (#2806)
  use mvn instead of run-in-docker (#2821)
  Better handling of form data (#2818)
  [haskell-servant] Add some missing types to the generated modules (#2675)
  Clarifies need to build project for sample output (#2819)
  [Elm] Bugfix .encode in modelTypeDiscriminator (#2807)
  Apply strict spec option to CodegenConfig instance (#2814)

Author: jimschubert

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,7 +1,7 @@
 ### PR checklist
 
 - [ ] Read the [contribution guidelines](https://github.com/openapitools/openapi-generator/blob/master/CONTRIBUTING.md).
-- [ ] Ran the shell script under `./bin/` to update Petstore sample so that CIs can verify the change. (For instance, only need to run `./bin/{LANG}-petstore.sh`, `./bin/openapi3/{LANG}-petstore.sh` if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in `.\bin\windows\`.
+- [ ] Ran the shell script under `./bin/` to update Petstore sample so that CIs can verify the change. (For instance, only need to run `./bin/{LANG}-petstore.sh`, `./bin/openapi3/{LANG}-petstore.sh` if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in `.\bin\windows\`. If contributing template-only or documentation-only changes which will change sample output, be sure to [build the project](https://github.com/OpenAPITools/openapi-generator#14---build-projects) first.
 - [ ] Filed the PR against the [correct branch](https://github.com/OpenAPITools/openapi-generator/wiki/Git-Branches): `master`~~, `3.4.x`, `4.0.x`~~. Default: `master`.
 - [ ] Copied the [technical committee](https://github.com/openapitools/openapi-generator/#62---openapi-generator-technical-committee) to review the pull request if your PR is targeting a particular programming language.
 

.travis.yml

@@ -121,7 +121,7 @@ script:
   # fail if generators contain tab '\t'
   - /bin/bash ./bin/utils/detect_tab_in_java_class.sh
   # run integration tests defined in maven pom.xml
-  - ./run-in-docker.sh mvn --quiet --batch-mode clean install
+  - mvn --quiet --batch-mode clean install
   - mvn --quiet --batch-mode verify -Psamples
 after_success:
   # push to maven repo

README.md

@@ -40,7 +40,7 @@ If you find OpenAPI Generator useful for work, please consider asking your compa
 #### Thank you to our bronze sponsors!
 
 [![NamSor](https://openapi-generator.tech/img/companies/namsor.png)](https://www.namsor.com/?utm_source=openapi_generator&utm_medium=github_webpage&utm_campaign=sponsor)
-
+[![LightBow](https://openapi-generator.tech/img/companies/lightbow.png)](https://www.lightbow.net/?utm_source=openapi_generator&utm_medium=github_webpage&utm_campaign=sponsor)
 
 ## Overview
 OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs,  documentation and configuration automatically given an [OpenAPI Spec](https://github.com/OAI/OpenAPI-Specification) (both 2.0 and 3.0 are supported). Currently, the following languages/frameworks are supported:

docs/templating.md

@@ -18,7 +18,7 @@ The transform logic needs to implement [CodegenConfig.java](https://github.com/o
 > * Maven Plugin: `templateDirectory`
 > * Gradle Plugin: `templateDir` 
 
-Built-in templates are written in Mustache and processed by [jmustache](https://github.com/samskivert/jmustache). We plan to eventually support Handlebars and user-defined template engines via plugins.
+Built-in templates are written in Mustache and processed by [jmustache](https://github.com/samskivert/jmustache). Beginning with version 4.0.0, we support experimental Handlebars and user-defined template engines via plugins.
 
 OpenAPI Generator supports user-defined templates. This approach is often the easiest when creating a custom template. Our generators implement a combination of language and framework features, and it's fully possible to use an existing generator to implement a custom template for a different framework. Suppose you have internal utilities which you'd like to incorporate into generated code (e.g. logging, monitoring, fault-handling)... this is easy to add via custom templates.
 
@@ -261,6 +261,165 @@ Congratulations! You've now modified one of the built-in templates to meet your
 
 Adding/modifying template logic simply requires a little bit of [mustache](https://mustache.github.io/), for which you can use existing templates as a guide.
 
+### Custom Engines
+
+> Custom template engine support is *experimental*
+
+If Mustache or the experimental Handlebars engines don't suit your needs, you can define an adapter to your templating engine of choice. To do this, you'll need to define a new project which consumes the `openapi-generator-core` artifact, and at a minimum implement `TemplatingEngineAdapter`.
+
+This example:
+
+* creates an adapter providing the fundamental logic to compile [Pebble Templates](https://pebbletemplates.io)
+* will be implemented in Kotlin to demonstrate ServiceLoader configuration specific to Kotlin (Java will be similar)
+* requires Gradle 5.0+
+* provides project setup instructions for IntelliJ
+
+To begin, create a [new Gradle project](https://www.jetbrains.com/help/idea/getting-started-with-gradle.html) with Kotlin support. To do this, go to `File` ➞ `New` ➞ `Project`, choose "Gradle" and "Kotlin". Specify groupId `org.openapitools.examples` and artifactId `pebble-template-adapter`.
+
+Ensure the new project uses Gradle 5.0. Navigate to the newly created directory and execute:
+
+```bash
+gradle wrapper --gradle-version 5.0
+```
+
+In `build.gradle`, we'll add a dependency for OpenAPI Tools core which defines the interface and an abstract helper type for implementing the adapter. We'll also pull in the Pebble artifact. We'll be evaluating this new artifact locally, so we'll also add the Maven plugin for installing to the local maven repository. We'll also create a fatjar using the `shadow` plugin to simplify our classpath.
+
+Modifications to the new project's `build.gradle` should be made in the `plugins` and `dependencies` nodes:
+
+```diff
+ plugins {
+    id 'org.jetbrains.kotlin.jvm' version '1.3.11'
+    id "com.github.johnrengelman.shadow" version "5.0.0"
+ }
+ 
+ dependencies {
+    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
+    compile "org.openapitools:openapi-generator-core:4.0.0-SNAPSHOT"
+    compile "io.pebbletemplates:pebble:3.0.8"
+ }
+```
+
+The above configuration for the `shadow` plugin is strictly optional. It is not needed, for instance, if you plan to publish your adapter and consume it via the Maven or Gradle plugins.
+
+Next, create a new class file called `PebbleTemplateEngineAdapter` under `src/kotlin`. We'll define the template adapter's name as `pebble` and we'll also list this as the only supported file extension. We'll implement the adapter by extending `AbstractTemplatingEngineAdapter`, which includes reusable logic, such as retrieving a list of all possible template names for our provided template extensions(s). 
+
+The class in its simplest form looks like this (with inline comments):
+
+```kotlin
+// Allows specifying engine by class name
+// e.g. -e org.openapitools.examples.templating.PebbleTemplateAdapter
+@file:JvmName("PebbleTemplateAdapter")
+package org.openapitools.examples.templating
+
+// imports
+
+class PebbleTemplateAdapter : AbstractTemplatingEngineAdapter() {
+    // initialize the template compilation engine
+    private val engine: PebbleEngine = PebbleEngine.Builder()
+        .cacheActive(false)
+        .loader(DelegatingLoader(listOf(FileLoader(), ClasspathLoader())))
+        .build()
+
+    // allows targeting engine by id/name: -e pebble
+    override fun getIdentifier(): String = "pebble"
+
+    override fun compileTemplate(
+        generator: TemplatingGenerator?,
+        bundle: MutableMap<String, Any>?,
+        templateFile: String?
+    ): String {
+        // This will convert, for example, model.mustache to model.pebble
+        val modifiedTemplate = this.getModifiedFileLocation(templateFile).first()
+
+        // Uses generator built-in template resolution strategy to find the full template file
+        val filePath = generator?.getFullTemplatePath(modifiedTemplate)
+
+        val writer = StringWriter()
+        // Conditionally writes out the template if found.
+        if (filePath != null) {
+            engine.getTemplate(filePath.toAbsolutePath().toString())?.evaluate(writer, bundle)
+        }
+        return writer.toString()
+    }
+
+    override fun getFileExtensions(): Array<String> = arrayOf("pebble")
+}
+```
+
+Lastly, create a file `resources/META-INF/services/org.openapitools.codegen.api.TemplatingEngineAdapter`, containing the full class path to the above class:
+
+```
+org.openapitools.examples.templating.PebbleTemplateAdapter
+```
+
+This allows the adapter to load via ServiceLoader, and to be referenced via the identifier `pebble`. This is optional; if you don't provide the above file and contents, you'll only be able to load the engine via full class name (explained in a bit).
+
+Now, build the fatjar for this new adapter:
+
+```bash
+./gradlew shadowJar
+```
+
+To test compilation of some templates, we'll need to first create one or more template files. Create a temp directory at `/tmp/pebble-example/templates` and add the following files.
+
+*api.pebble*
+
+```
+package {{packageName}}
+
+import (
+    "net/http"
+{% for item in imports %}
+    "{{item.import}}"
+{% endfor %}
+)
+
+type Generated{{classname}}Servicer 
+
+// etc
+```
+
+*model.pebble*
+
+```
+package {{packageName}}
+
+{% for item in models %}
+{% if item.isEnum %}
+// TODO: enum
+{% else %}
+{% if item.description is not empty %}// {{item.description}}{% endif %}
+type {{item.classname}} struct {
+{% for var in item.model.vars %}
+    {% if var.description is not empty %}// {{var.description}}{% endif %}
+    {{var.name}} {% if var.isNullable %}*{% endif %}{{var.dataType}} `json:"{{var.baseName}}{% if var.required == false %},omitempty{% endif %}"{% if var.withXml == true %} xml:"{{var.baseName}}{% if var.isXmlAttribute %},attr{% endif %}"{% endif %}`
+{% endfor %}
+}
+{% endif %}
+{{model.name}}
+{% endfor %}
+```
+
+> Find object structures passed to templates later in this document's **Structures** section.
+
+Finally, we can compile some code by explicitly defining our classpath and jar entrypoint for CLI (be sure to modify `/your/path` below) 
+
+```bash
+java $JAVA_OPTS -cp /your/path/build/libs/pebble-template-adapter-1.0-SNAPSHOT-all.jar:modules/openapi-generator-cli/target/openapi-generator-cli.jar \
+    org.openapitools.codegen.OpenAPIGenerator \
+    generate \
+    -g go \
+    -i https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/json/petstore-minimal.json \
+    -e pebble \
+    -o /tmp/pebble-example/out \
+    -t /tmp/pebble-example/templates \
+    -Dmodels -DmodelDocs=false -DmodelTests=false -Dapis -DapiTests=false -DapiDocs=false
+```
+
+Notice how we've targeted our custom template engine adapter via `-e pebble`. If you don't include the SPI file under `META-INF/services`, you'll need to specify the exact classpath: `org.openapitools.examples.templating.PebbleTemplateAdapter`. Notice that the target class here matches the Kotlin class name. This is because of the `@file:JvmName` annotation.
+
+Congratulations on creating a custom templating engine adapter!
+
 ## Structures
 
 Aside from transforming an API document, the implementing class gets to decide how to apply the data structure to templates. We can decide which data structure to apply to which template files. You have the following structures at your disposal.

modules/openapi-generator-core/src/main/java/org/openapitools/codegen/api/TemplatingGenerator.java

@@ -16,6 +16,8 @@
 
 package org.openapitools.codegen.api;
 
+import java.nio.file.Path;
+
 /**
  * interface to the full template content
  *   implementers might take into account the -t cli option,
@@ -25,9 +27,19 @@ public interface TemplatingGenerator {
 
   /**
    * returns the template content by name
+   *
    * @param name the template name (e.g. model.mustache)
+   *
    * @return the contents of that template
    */
   String getFullTemplateContents(String name);
 
+  /**
+   * Returns the path of a template, allowing access to the template where consuming literal contents aren't desirable or possible.
+   *
+   * @param name the template name (e.g. model.mustache)
+   *
+   * @return The {@link Path} to the template
+   */
+  Path getFullTemplatePath(String name);
 }

modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultCodegen.java

@@ -1308,6 +1308,7 @@ public String toDefaultValue(Schema schema) {
 
     /**
      * Return property value depending on property type.
+     *
      * @param schema property type
      * @return property value
      */
@@ -1472,7 +1473,8 @@ private static String getPrimitiveType(Schema schema) {
             } else if (ModelUtils.isDoubleSchema(schema)) {
                 return SchemaTypeUtil.DOUBLE_FORMAT;
             } else {
-                LOGGER.warn("Unknown `format` detected for " + schema.getName() + ": " + schema.getFormat());
+                LOGGER.warn("Unknown `format` {} detected for type `number`. Defaulting to `number`", schema.getFormat());
+                return "number";
             }
         } else if (ModelUtils.isIntegerSchema(schema)) {
             if (ModelUtils.isLongSchema(schema)) {
@@ -1913,7 +1915,7 @@ public String getterAndSetterCapitalize(String name) {
      */
     public CodegenProperty fromProperty(String name, Schema p) {
         if (p == null) {
-            LOGGER.error("Undefined property/schema for `{}`. Default to type:string.",  name);
+            LOGGER.error("Undefined property/schema for `{}`. Default to type:string.", name);
             return null;
         }
         LOGGER.debug("debugging fromProperty for " + name + " : " + p);
@@ -2063,7 +2065,7 @@ public CodegenProperty fromProperty(String name, Schema p) {
             if (innerSchema == null) {
                 LOGGER.error("Undefined array inner type for `{}`. Default to String.", p.getName());
                 innerSchema = new StringSchema().description("//TODO automatically added by openapi-generator due to undefined type");
-                ((ArraySchema)p).setItems(innerSchema);
+                ((ArraySchema) p).setItems(innerSchema);
             }
         } else if (ModelUtils.isMapSchema(p)) {
             Schema innerSchema = ModelUtils.unaliasSchema(this.openAPI, ModelUtils.getAdditionalProperties(p));
@@ -2145,7 +2147,7 @@ public CodegenProperty fromProperty(String name, Schema p) {
             if (innerSchema == null) {
                 LOGGER.error("Undefined array inner type for `{}`. Default to String.", p.getName());
                 innerSchema = new StringSchema().description("//TODO automatically added by openapi-generator due to undefined type");
-                ((ArraySchema)p).setItems(innerSchema);
+                ((ArraySchema) p).setItems(innerSchema);
             }
             CodegenProperty cp = fromProperty(itemName, innerSchema);
             updatePropertyForArray(property, cp);
@@ -2537,8 +2539,8 @@ public CodegenOperation fromOperation(String path,
         CodegenParameter bodyParam = null;
         RequestBody requestBody = operation.getRequestBody();
         if (requestBody != null) {
-            if ("application/x-www-form-urlencoded".equalsIgnoreCase(getContentType(requestBody)) ||
-                    "multipart/form-data".equalsIgnoreCase(getContentType(requestBody))) {
+            if (getContentType(requestBody).toLowerCase(Locale.ROOT).startsWith("application/x-www-form-urlencoded") ||
+                    getContentType(requestBody).toLowerCase(Locale.ROOT).startsWith("multipart/form-data")) {
                 // process form parameters
                 formParams = fromRequestBodyToFormParameters(requestBody, imports);
                 for (CodegenParameter cp : formParams) {
@@ -4216,7 +4218,8 @@ public void writePropertyBack(String propertyKey, boolean value) {
 
     protected String getContentType(RequestBody requestBody) {
         if (requestBody == null || requestBody.getContent() == null || requestBody.getContent().isEmpty()) {
-            return null;
+            LOGGER.warn("Cannot determine the content type. Default to UNKNOWN_CONTENT_TYPE.");
+            return "UNKNOWN_CONTENT_TYPE";
         }
         return new ArrayList<>(requestBody.getContent().keySet()).get(0);
     }
@@ -4297,7 +4300,9 @@ public boolean hasFormParameter(OpenAPI openAPI, Operation operation) {
         }
 
         for (String consume : consumesInfo) {
-            if ("application/x-www-form-urlencoded".equalsIgnoreCase(consume) || "multipart/form-data".equalsIgnoreCase(consume)) {
+            if (consume != null &&
+                    consume.toLowerCase(Locale.ROOT).startsWith("application/x-www-form-urlencoded") ||
+                    consume.toLowerCase(Locale.ROOT).startsWith("multipart/form-data")) {
                 return true;
             }
         }
@@ -4871,7 +4876,7 @@ public List<CodegenServerVariable> fromServerVariables(Map<String, ServerVariabl
     }
 
     private void setParameterNullable(CodegenParameter parameter, CodegenProperty property) {
-        if(parameter == null || property == null) {
+        if (parameter == null || property == null) {
             return;
         }
         parameter.isNullable = property.isNullable;
@@ -4920,7 +4925,7 @@ public boolean isEnableMinimalUpdate() {
     /**
      * Set the boolean value indicating the state of the option for updating only changed files
      *
-     * @param enableMinimalUpdate    true to enable minimal update
+     * @param enableMinimalUpdate true to enable minimal update
      */
     @Override
     public void setEnableMinimalUpdate(boolean enableMinimalUpdate) {

modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultGenerator.java

@@ -47,6 +47,7 @@
 
 import java.io.*;
 import java.net.URL;
+import java.nio.file.Path;
 import java.time.ZonedDateTime;
 import java.util.*;
 
@@ -936,6 +937,19 @@ public String getFullTemplateContents(String templateName) {
         return readTemplate(getFullTemplateFile(config, templateName));
     }
 
+    /**
+     * Returns the path of a template, allowing access to the template where consuming literal contents aren't desirable or possible.
+     *
+     * @param name the template name (e.g. model.mustache)
+     *
+     * @return The {@link Path} to the template
+     */
+    @Override
+    public Path getFullTemplatePath(String name) {
+        String fullPath = getFullTemplateFile(config, name);
+        return java.nio.file.Paths.get(fullPath);
+    }
+
     protected File processTemplateToFile(Map<String, Object> templateData, String templateName, String outputFilename) throws IOException {
         String adjustedOutputFilename = outputFilename.replaceAll("//", "/").replace('/', File.separatorChar);
         if (ignoreProcessor.allowsFile(new File(adjustedOutputFilename))) {

modules/openapi-generator/src/main/java/org/openapitools/codegen/config/CodegenConfigurator.java

@@ -541,6 +541,8 @@ public ClientOptInput toClientOptInput() {
         config.languageSpecificPrimitives().addAll(languageSpecificPrimitives);
         config.reservedWordsMappings().putAll(reservedWordMappings);
 
+        config.setStrictSpecBehavior(isStrictSpecBehavior());
+
         checkAndSetAdditionalProperty(apiPackage, CodegenConstants.API_PACKAGE);
         checkAndSetAdditionalProperty(modelPackage, CodegenConstants.MODEL_PACKAGE);
         checkAndSetAdditionalProperty(invokerPackage, CodegenConstants.INVOKER_PACKAGE);

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/AbstractJavaCodegen.java

@@ -607,7 +607,7 @@ public String toVarName(String name) {
         }
 
         // if it's all uppper case, do nothing
-        if (name.matches("^[A-Z_]*$")) {
+        if (name.matches("^[A-Z0-9_]*$")) {
             return name;
         }
 

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/AbstractKotlinCodegen.java

@@ -717,7 +717,7 @@ public String toVarName(String name) {
         }
 
         // if it's all uppper case, do nothing
-        if (name.matches("^[A-Z_]*$")) {
+        if (name.matches("^[A-Z0-9_]*$")) {
             return name;
         }
 

modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/HaskellServantCodegen.java

@@ -301,7 +301,6 @@ public void preprocessOpenAPI(OpenAPI openAPI) {
      */
     private void setGenerateToSchema(CodegenModel model) {
         for (CodegenProperty var : model.vars) {
-            LOGGER.warn(var.dataType);
             if (var.dataType.contentEquals("Value") || var.dataType.contains(" Value")) {
                 additionalProperties.put("generateToSchema", false);
             }
@@ -349,7 +348,7 @@ public String getTypeDeclaration(Schema p) {
     @Override
     public String getSchemaType(Schema p) {
         String schemaType = super.getSchemaType(p);
-        LOGGER.debug("debugging swager type: " + p.getType() + ", " + p.getFormat() + " => " + schemaType);
+        LOGGER.debug("debugging OpenAPI type: " + p.getType() + ", " + p.getFormat() + " => " + schemaType);
         String type = null;
         if (typeMapping.containsKey(schemaType)) {
             type = typeMapping.get(schemaType);