Big thoughts for 4.0

[darrelmiller]

We would like to take an opportunity to discuss what might be some big, radical changes that could be done in a 4.0 version that would address issues with OAS that we have not been able to address with past changes.

Let us know what major changes you think would make a significant improvement to the way you describe your APIs.

[swaldron58]

Bear with me on this one as it is not a change to the OAS per se, but a potential change in the way the spec or what may become specs (or types of extensions) are organized.
On board with the call for “simplification” and usability of the spec with the underlying caveat most have, “as long as the spec does what I need it to do, no matter how complex that is”. I don’t need to repeat that driver for change.

It’s groundhog day for me with this recurring lifecycle of the evolution of specs and languages from simple, to addressing more complex but needed capabilities, to “the spec is too complex and hard to understand, let’s start over”. I started in 360 assembler and 6 bit line protocols (P1024B to be specific) and everything since. Yes, some fundamental changes since then like moving away from SMP and procedural compute models to the current forms of shared nothing which has been huge value. But so many mistakes repeated again and again. I will also note that I view things more from an IT leader managing thousands of developers on hundreds of projects. The real world needs to run a successful and profitable IT shop. The need for velocity and predictability of outcome (quality, function, time to market, etc.). Specs are used by developers to solve a point problem. Standards are enforced by IT leaders when they lead to real business value.

One of those concerns is watching increasing use of schema as a programming language. Embedded business logic in what was intended to be CRUD messages to a discreet business object. This has always happened before hence nothing new, however it leads to a lot of instability, complexity and cost. Note that I am not suggesting we start taking things out of the spec to avoid improper or at least unadvisable use. My suggested change is in what we can do to help the community not shoot themselves in the foot quite so easily.

In programming languages we are all used to the migration to higher level languages. The benefit being providing a format where average developers can express the business functions needed and let a complier produce the runtime. There was always the ability to drop back into a lower level language for edge cases. There are tools out there doing this or at least moving towards the same for schema. I said schema instead of JSON or anything specific as hopefully the tools are not tightly bound to any one instance of a format.

My change is for OAI to overtly support this trend. Attack simplification by helping any tool provider coming up with a better way for developers to describe the 90% of the business needs at the API level and get developers away from manually updating schema. Compliers and similar approaches can do a lot better at leveraging the OAS to meet business needs with a lot fewer mistakes, faster to market, less fragile, hence lower costs. What can we do that makes it easier for a tool supplier providing a simpler interface to produce OAS compliant documentation? A means that negates the need for manually updating and managing JSON schema? This would still be API first, just not at a level so error prone.

As a refence point I recently ran a large development shop using SAFe coupled with model driven development. The business objects and services defined in the model. A complier produced .xsd and OAS 3.0 where there was little need or opportunity for anyone to touch the schema. One service definition in the model could begat numerous (100s even) of resource names (APIs deployed). Didn’t care about the number of APIs as wasn’t trying to manage individual schema or even crazier, snippets of schema. There was no need to make omnibus APIs to save on development work. Who needs multipath statements when I can just define another facet of the model for the unique need of each path? My point being left shifting the development work away from working at the schema level works. A simpler schema is not the answer. It’s how can average developers define business objects together with what actions are to be taken against them. Then let tooling produce the needed artifacts to deploy using a spec such as OAS 3.1.

[SensibleWood]

My change is for OAI to overtly support this trend. Attack simplification by helping any tool provider coming up with a better way for developers to describe the 90% of the business needs at the API level and get developers away from manually updating schema.

I 100% agree with this. The salient API-first approaches still concentrate on the primacy of the OpenAPI document as the vehicle for specification development, which is what many developers rally against from a code-first perspective. Being able to drive this from a common modelling approach, which is still API-first but removes the dependency on the OpenAPI document would be a big boost. The modelling approach would help drive the API specification, which then drives the code. That extra layer of abstraction would be really helpful for lots of folks, and allow for commonality in tooling on how to drive it.

[patricekrakow]

"Merge" with AsyncAPI in order to have one (and only one) specification language (and therefore, hopefully, ecosystem of tools) to define/document both synchronous (request-response) communication patterns and asynchronous (publish-subscribe) communication patterns for APIs.

[patricekrakow]

Remove the servers property in order to have the OAS solely focusing on the definition of the interface, not giving any details about the implementation(s) of the interface. This is a bit "on the edge", but I think the servers property is crossing the boundary between the interface and the implementation.

[reloxx13]

I18n
#274
Also reopen the above issue as its not duplicated as explained multiple times now.
Content I18n != Documentation I18n

[jeremyfiel]

I'm not sure adding a headers map to the Operation object is the right move, but I really want some clarification on why reusable headers are not allowed across requests and responses. This is a very confusing area of the spec. If I have a header used in Request and Response, I expect to be able to $ref my components>headers entry. Why should I need to duplicate a parameter object for my request header?

We have many GET/DELETE operations requiring custom headers in the Request and it's not currently possible to reuse my components/headers in these operations because a headers map doesn't exist on the operation object and we don't allow request bodies on these operations.

[handrews]

• Rework the whole content encoding area. I integrated the JSON Schema content* keywords for 3.1, but it was a bit of a rush job and needs cleanup. I hope to contribute basic cleanup for 3.1.1, and maybe some slight improvements would fit in a 3.2, but really there's now a lot of conceptual overlap and weirdness about how that is handled and it would benefit from a comprehensive re-think for 4.0.

• Add annotations for additional client-server semantics to OAI's JSON Schema extension vocabulary, or perhaps as their own vocabulary. This would be better done through OAI than JSON Schema IMHO. It's a somewhat trickier area than it initially looks primarily due to people wanting things to be required or not depending on message direction, which doesn't fit nicely into the JSON Schema validation model. But there are some ways to work with that to clarify things like "you can't change this and we'll ignore it if you try" vs "you can include it in a PUT if it's unchanged from the state marked by your ETag, but if you change it it's an error" vs "if you include this in a PUT or PATCH at all it's an error", etc. Also create-only fields, write-once fields (create-only is a special case where the write-once has to happen on create), and probably other stuff I'm forgetting.

• Concise representation-oriented syntax. As a system for describing any HTTP API, OpenAPI's flexibility is a strength, but it comes at the cost of a lot of boilerplate if your API uses regular, predictable patterns for interacting with a representation. I should just be able to say "this is my representation schema (using the annotations from the point above), here's the URI template for the item and collection, and GET/PUT/POST/DELETE work as expected, and PATCH works with patch media types X & Y (which allow the PATCH request schema to be derived predictably from the representation schema). There are varying levels to which this can be done, e.g. automatically deriving PATCH schemas is more of a bonus than essential. This does require being able to specify what your "normal" item and collection interactions look like, which is more of a concern for collections.

• Consider reworking parameter description into a single schema rather than per-parameter descriptions, so that interactions among parameters (and possibly also headers? idk) can be described with JSON Schema. This sort of interaction-describing functionality is requested regularly IIRC, and JSON Schema already has all of the necessary features.

[karenetheridge]

I've raised this on slack, so I guess I should raise it here as well:

• rethink/rework the style/explode behaviour. It's a lot more clear how to use these values when serializing a data structure into a request, and much less clear when dealing with a http request/response and trying to deserialize into a data structure that can have a json schema applied to it.
  • the Swagger docs (https://swagger.io/docs/specification/serialization/) imply that the deserializer needs to peek inside the json schema at the 'type' keyword to see if it is desired to deserialize to an array or an object or a number (or an array of numbers or an object of numbers) which gets hairy really quickly and not really doable after the first level of an object/array if the json schema has any sort of complication to it at all (e.g. nested schemas inside allOf, etc).
  • Some headers can be used twice and probably should be parsed into an array for validation (but the spec doesn't talk about that). Some headers aren't parsed using the default style=simple, explode=false mechanism (e.g. Set-Cookie only uses separate header values and individual entries cannot be split; Cookie is split by ;\s*, multiple values are not allowed for Content-Length).
  • Additionally I believe it is already possible to parse all query parameters into one "parameter value" as an object (which is something people have been asking for) using a particular combination of style/explode/schema values, so an example of this should be provided in the spec.
  • use of style/explode shouldn't be allowed in request or response bodies at all - we already have media types for this.

[handrews]

use of style/explode shouldn't be allowed in request or response bodies at all - we already have media types for this.

Which reminds me, we can apply media types to the entire query string. The one that everyone thinks is "standard" (but is not actually part of RFC 3986 at all) is application/x-www-form-urlencoded, which easily maps into the JSON data model and therefore could be described by a single schema rather than partitioned out by parameter.

Furthermore, doing so allows use of the content* keywords to do things like embed JSON values (although i guess that can be done on a per-parameter basis as well if you want to).

To address some of @karenetheridge 's concerns about explode, and also thinking about issues like #1706 ("deep objects"), it seems to me that a better solution than increasingly complex control keywords is to define additional query-string-friendly media types that map into the JSON Schema data model (or potentially other schema types- really, mapping to a data model is the important part, then you can describe model however you want).

This could either be a hybrid thing like using JSON Schema's content* to embed other media types inside application/x-www-form-urlencoded or you could do something more direct with the entire query string. Hmm.. can the necessary URL encoding be considered a content encoding of some sort? OK now I know I'm going to mix up content codings, content ENcodings, transfer codings, etc. so I'll stop rambling. My point being that there are a lot of ways to support complex query parameters and taking a step back and re-thinking it more thoroughly, as I think @karenetheridge is proposing, seems like a good idea to me.

[darrelmiller]

Consider flattening the responses object to an array that uses pattern matching to define the corresponding response object.

  /pet:
    get:
       responses:
         - criteria:
             status: '200'
             mediaType: application/json 
             conditions:
               "$/response/body#/petType": dog
               "$/request.path.format": json            
           description: Success
           schema:
            type: object
            properties:
              name: 
                type: string
              barkVolume:
                type: number

This would significantly reduce the nesting for simple API designs, provide support for more sophisticated interactions. It would allow mapping request values to specific responses, which is a common feature request.