Modifying the libopenapi data model

Add, edit and delete OpenAPI specifications with ease.

Until v0.7.0 the library was optimized for reading. With the introduction of v0.7.0 the porcelain layer is now Renderable. Any changes made to the high-level can now be captured when ‘rendering’ the model to YAML or JSON.

We made a conscious choice to only build this functionality into the OpenAPI 3+ model and NOT the Swagger model. This is because We’re not actively developing the Swagger model, and we don’t want to encourage people to use it.

Swagger should not be used, The library will throw an error if any of the techniques below are used on a Swagger model.

Document ordering

When using YAML, it can be really annoying having to locate something that has jumped in position after editing and saving a document. This happens because most tools convert to and from JSON under the covers. JSON has no ordering (unlike YAML), so this data is all lost.

libopenapi retains the original model line and column numbers, so when re-rendering, it knows how or order the output of everything. New content sinks to the bottom of the object and all the original data is rendered above new content, in order.

Modify and Re-render

The libopenapi data-model works by reading in an OpenAPI spec into a low-level model first, and then creates a high-level model on top of that low-level model with much simpler to use APIs and data structures.

Model lifecycle

model diagram of libopenapi
  • Raw YAML/JSON is read in
  • plumbing/low-level model is created
  • porcelain/high-level model is created
  • OpenAPI model is available to the document
  • The document is rendered into a *yaml.Node tree
  • The *yaml.Node tree is rendered into YAML/JSON
The plumbing/low-level model is immutable and any changes to it, will not be picked up when the model is re-rendered.

Rendering individual objects

All high-level objects in the model are Renderable which means they all implement the MarshalYAML() method. Every high-level object also implements a Render() method that will shortcut the need to call yaml.Marshal()

Rendering something like a Tag is easy.

import "github.com/pb33f/libopenapi/datamodel/high/base"

func PrintTag() {

  // create a tag 
  tag := &base.Tag{
     Name:        "useless",
     Description: "this is a useless tag",
  }

  rendered, _ := tag.Render()
  fmt.Print(string(rendered))
}

Will print:

name: useless description: this is a useless tag

Modifying OpenAPI specifications

The porcelain/high-level model is mutable. It can be edited and modified as much as required. Once ready, the model can be rendered back into YAML/JSON

RenderAndReload()

On the Document object, there is a RenderAndReload() method that will render the model back into bytes, and then re-build the document model back. This is how any changes made to the high level model are propagated back into the low-level model.

The lifecycle of the document only flows one way, so the only effective way to refresh the low-level model, is to render out the spec and reparse it back in.

The signature of the RenderAndReload() method is:

RenderAndReload() ([]byte, Document, *DocumentModel[v3high.Document], []error)
  • The first return value is the bytes of the rendered document
  • The second return value is the new Document
  • The third return value is the new DocumentModel (OpenAPI 3+ Only)
  • The fourth return value is a slice of errors (if any were generated)

An example

In this example, we read in an OpenAPI specification, we create a new Path Item, complete with a new Get operation.

import (
    "fmt"
    "github.com/pb33f/libopenapi"
    v3 "github.com/pb33f/libopenapi/datamodel/high/v3"
    "gopkg.in/yaml.v3"
    "os"
    "strings"
)

func EditOpenAPISpec() {

    // How to read in an OpenAPI 3 Specification, into a Document,
    // modify the document and then re-render it back to YAML bytes.

    // load an OpenAPI 3 specification from bytes
    petstore, _ := os.ReadFile("petstorev3.json")

    // create a new document from specification bytes
    doc, err := libopenapi.NewDocument(petstore)

    // if anything went wrong, an error is thrown
    if err != nil {
        panic(fmt.Sprintf("cannot create new document: %e", err))
    }

    // because we know this is a v3 spec, we can build a ready to go model from it.
    v3Model, errors := doc.BuildV3Model()

    // if anything went wrong when building the v3 model, a slice of errors will be returned
    if len(errors) > 0 {
        for i := range errors {
            fmt.Printf("error: %e\n", errors[i])
        }
        panic(fmt.Sprintf("cannot create v3 model from document: %d errors reported", len(errors)))
    }

    // create a new path item and operation.
    newPath := &v3.PathItem{
        Description: "this is a new path item",
        Get: &v3.Operation{
            Description: "this is a get operation",
            OperationId: "getNewThing",
            RequestBody: &v3.RequestBody{
                Description: "this is a new request body",
            },
        },
    }

    // capture original number of paths
    originalPaths := v3Model.Model.Paths.PathItems.Len()

    // add the path to the document
    v3Model.Model.Paths.PathItems.Set("/new/path", newPath)

    // render out the new path item to YAML

Now we can use the Render() method to render out the newPath Path Item, into YAML 

    renderedPathItem, _ := newPath.Render()

The RenderAndReload() method available on the document, will render the document back to bytes, and reload the model from scratch. This is the only way to ensure that the model is reloaded with the new changes and all the low-level details are refreshed. 

    // render the document back to bytes and reload the model.
    _, _, newModel, errs := doc.RenderAndReload()

    // if anything went wrong when re-rendering the v3 model, a slice of errors will be returned
    if len(errors) > 0 {
        panic(fmt.Sprintf("cannot re-render document: %d errors reported", len(errs)))
    }

    // capture new number of paths after re-rendering
    newPaths := newModel.Model.Paths.PathItems.Len()

    // capture the line number of the operationId of our new path
    newOpIdLine := newModel.Model.Paths.PathItems.GetOrZero("/new/path").Get.GoLow().OperationId.KeyNode.Line

    // print out path counts, and the new line number for our new path.
    fmt.Printf("There were %d original paths. There are now %d paths in the document\n", originalPaths, newPaths)
    fmt.Printf("The new pathItem is on line %d\n---\n", newOpIdLine)
    fmt.Printf("When rendered, looks like this:\n%s", strings.TrimSpace(string(renderedPathItem)))
}

This will print:

There were 13 original paths. There are now 14 paths in the document The new pathItem is on line 588 When rendered, looks like this: --- description: this is a new path item get: description: this is a get operation operationId: getNewThing requestBody: description: this is a new request body The RenderAndReload() method will re-render the document back to bytes, and reload the model from scratch. It also means that the original model is no good and should be replaced with the model returned from the RenderAndReload()

Creating new OpenAPI specifications

Here is an example of how to build a useless, but operational OpenAPI specification from scratch, that is then rendered into YAML.

Creating Schemas

The backbone of any OpenAPI specification is the Schema object. libopenapi wraps all Schema objects in a SchemaProxy to avoid run-away circular references from creating endless loops. The SchemaProxy acts as a buffer that holds an un-composed Schema object until it’s required to be constructed via its Schema() method.

A convenience function exists to quickly create a SchemaProxy from a Schema object:

base.CreateSchemaProxy(schema *Schema) *SchemaProxy

Here is an example on how to use it.

import  "github.com/pb33f/libopenapi/datamodel/high/base"

func CreateUselessSchema() {
    
    // create a new SchemaProxy from a Schema object
    // The Schema also contains a property called 'nothing' that is
    // also Schema wrapped in a SchemaProxy
    
    // create an ordered property map
    propMap := orderedmap.New[string, *base.SchemaProxy]()
    propMap.Set("nothing", base.CreateSchemaProxy(&base.Schema{
        Type:    []string{"string"},
        Example: &yaml.Node{Value: "nothing"},
    }))
    
    // create a schema proxy
    proxy := base.CreateSchemaProxy(&base.Schema{
        Type: []string{"object"},
        Properties: propMap,
    })
}

Build an OpenAPI spec

It’s pretty simple to build up an OpenAPI specification using the high-level model.


import (
    "fmt"
    "github.com/pb33f/libopenapi"
    base "github.com/pb33f/libopenapi/datamodel/high/base"
    v3 "github.com/pb33f/libopenapi/datamodel/high/v3"
    yaml "gopkg.in/yaml.v3"
)

func main() {

  // create an ordered property map
  propMap := orderedmap.New[string, *base.SchemaProxy]()
  propMap.Set("nothing", base.CreateSchemaProxy(&base.Schema{
      Type:    []string{"string"},
      Example: &yaml.Node{Value: "nothing"},
  }))

  // create a sample/empty schema that we can use in our document
  nothingSchema := base.CreateSchemaProxy(&base.Schema{
      Type:       []string{"object"},
      Properties: propMap,
  })

  // create response map
  responseMap := orderedmap.New[string, *v3.Response]()

  // OK response
  okResponseContentMap := orderedmap.New[string, *v3.MediaType]()
  okResponseContentMap.Set("application/json", &v3.MediaType{
      Schema: nothingSchema,
  })

  // add responses
  responseMap.Set("200", &v3.Response{
      Description: "nothing to see here.",
      Content:     okResponseContentMap,
  })

  // map path items
  pathItemMap := orderedmap.New[string, *v3.PathItem]()
  pathItemMap.Set("/nothing", &v3.PathItem{
      Get: &v3.Operation{
          OperationId: "getNothing",
          Description: "literally does nothing",
          Responses: &v3.Responses{
             Codes: responseMap,
          },
      },
  })

  // create a new OpenAPI document.
  doc := &v3.Document{
      Version: "3.1.0",
      Info: &base.Info{
         Title: "Useless API",
          Contact: &base.Contact{
              Name:  "quobix",
              Email: "buckaroo@pb33f.io",
          },
      },
      Paths: &v3.Paths{
          PathItems: pathItemMap,
      },
  }

  // render the doc
  rend, _ := doc.Render()
  fmt.Print(string(rend))

  // render the document to YAML.
  renderedYAMLBytes, _ := doc.Render()

  // create a new OpenAPI document from our rendered YAML.
  newDoc, _ := libopenapi.NewDocument(renderedYAMLBytes)

  // build a new v3 model from the document.
  newModel, _ := newDoc.BuildV3Model()

  // print out the line and column number of our 200 response description value:
  line := newModel.Model.Paths.PathItems.GetOrZero("/nothing").Get.Responses.Codes.GetOrZero("200").GoLow().Description.ValueNode.Line
  col := newModel.Model.Paths.PathItems.GetOrZero("/nothing").Get.Responses.Codes.GetOrZero("200").GoLow().Description.ValueNode.Column
  description := newModel.Model.Paths.PathItems.GetOrZero("/nothing").Get.Responses.Codes.GetOrZero("200").Description

  fmt.Printf("\n'%s' is located on line: %d, col: %d\n", description, line, col)

This will print the following to the terminal:

openapi: 3.1.0 info: title: Useless API contact: name: quobix email: buckaroo@pb33f.io paths: /nothing: get: description: literally does nothing operationId: getNothing responses: "200": description: nothing to see here. content: application/json: schema: type: object properties: nothing: type: string example: nothing 'nothing to see here.' is located on line: 14, col: 34

Using references

libopenapi fully supports the use of references when reading OpenAPI documents. It has a more limited support for references when writing OpenAPI documents. References can only be created with Schema definitions when mutating the document.

References are a great, but they can be abused. One of the hardest parts of any library that parses JSON Schema to get right, is references.

libopenapi currently only allows references to be created when a Schema is used.

Helper methods

Like the CreateSchemaProxy method, there is a sister method for creating references to schemas:

base.CreateSchemaProxyRef(ref string) *SchemaProxy

The idea here is that instead of creating a Schema object, you create a SchemaProxy that points to a reference.

For example:

import  "github.com/pb33f/libopenapi/datamodel/high/base"

func CreateUselessSchemaRef() {
    
    // When we just need to reference a schema, we can use the CreateSchemaProxyRef method
    // to create a SchemaProxy that points to a reference.
    proxy := base.CreateSchemaProxyRef("#/components/schemas/Priority")
}
There is no validation in place to ensure that the reference exists. Make sure that the path to the reference is correct.

References example

Here is an example of how to create a simple OpenAPI specification that uses references.

import (
  "fmt"
  "github.com/pb33f/libopenapi/datamodel/high/base"
  "github.com/pb33f/libopenapi/datamodel/high/v3"
  "github.com/pb33f/libopenapi/orderedmap"
)

func SchemaWithReferences() {
    
  // create an ordered property map
  propMap := orderedmap.New[string, *base.SchemaProxy]()
  propMap.Set("nothing", base.CreateSchemaProxy(&base.Schema{
      Type:    []string{"string"},
      Example: &yaml.Node{Value: "nothing"},
  }))

  // create a sample/empty schema that we can use in our document
  nothingSchema := base.CreateSchemaProxy(&base.Schema{
      Type:       []string{"object"},
      Properties: propMap,
  })

  // create another re-usable status component
  statusMap := orderedmap.New[string, *base.SchemaProxy]()
  statusMap.Set("status", base.CreateSchemaProxy(&base.Schema{
      Type:    []string{"string"},
      Example: &yaml.Node{Value: "done"},
  }))
  statusSchema := base.CreateSchemaProxy(&base.Schema{
      Type:        []string{"object"},
      Description: "this is a generic status schema",
      Properties:  statusMap,
  })

  // create a re-usable status component
  paramSchema := base.CreateSchemaProxy(&base.Schema{
      Type:        []string{"integer"},
      Description: "The higher the number, the higher the priority.",
  })

  // create a new parameter, with a schema that has a reference to a component
  // and a custom vendor extension.
  extMap := orderedmap.New[string, *yaml.Node]()
  extMap.Set("x-somethingInternal", &yaml.Node{Kind: yaml.ScalarNode, Value: "true"})
  priorityParam := &v3.Parameter{
      Name:        "priority",
      In:          "query",
      Description: "the priority of the status",
      Schema:      base.CreateSchemaProxyRef("#/components/schemas/Priority"),
      Extensions:  extMap,
  }

  // create a new OpenAPI document.
  // map schemas
  schemaMap := orderedmap.New[string, *base.SchemaProxy]()
  schemaMap.Set("Nothing", nothingSchema)
  schemaMap.Set("Status", statusSchema)
  schemaMap.Set("Priority", paramSchema)

  // create response map
  responseMap := orderedmap.New[string, *v3.Response]()

  // OK response
  okResponseContentMap := orderedmap.New[string, *v3.MediaType]()
  okResponseContentMap.Set("application/json", &v3.MediaType{
      Schema: base.CreateSchemaProxyRef("#/components/schemas/Nothing"),
  })

  // error response
  errorResponseContentMap := orderedmap.New[string, *v3.MediaType]()
  errorResponseContentMap.Set("application/json", &v3.MediaType{
        Schema: base.CreateSchemaProxyRef("#/components/schemas/Status"),
  })

  // add responses
  responseMap.Set("200", &v3.Response{
      Description: "nothing to see here.",
      Content:     okResponseContentMap,
  })
  responseMap.Set("500", &v3.Response{
      Description: "something went wrong",
      Content:     errorResponseContentMap,
  })

  // map path items
  pathItemMap := orderedmap.New[string, *v3.PathItem]()
  pathItemMap.Set("/nothing", &v3.PathItem{
      Get: &v3.Operation{
          OperationId: "getNothing",
          Description: "literally does nothing",
          Parameters: []*v3.Parameter{
              priorityParam,
          },
          Responses: &v3.Responses{
              Codes: responseMap,
          },
      },
  })

  doc := &v3.Document{
      Version: "3.1.0",
      Info: &base.Info{
          Title: "Useless API",
          Contact: &base.Contact{
              Name:  "quobix",
              Email: "buckaroo@pb33f.io",
          },
      },
      Components: &v3.Components{
          Schemas: schemaMap,
      },
      Paths: &v3.Paths{
          PathItems: pathItemMap,
      },
  }
   
  rend, _ := doc.Render()
  fmt.Print(string(rend))
}

This will generate the following OpenAPI document:

openapi: 3.1.0
info:
  title: Useless API
  contact:
    name: Dave Shanley
    email: buckaroo@pb33f.io
paths:
  /nothing:
    get:
      description: literally does nothing
      operationId: getNothing
      parameters:
        - name: priority
          in: query
          description: the priority of the status
          schema:
            $ref: '#/components/schemas/Priority'
          x-somethingInternal: "true"
      responses:
        "200":
          description: nothing to see here.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Nothing'
        "500":
          description: something went wrong
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Status'
components:
  schemas:
    Status:
      type: object
      properties:
        status:
          type: string
          example: done
      description: this is a generic status schema
    Priority:
      type: integer
      description: The higher the number, the higher the priority.
    Nothing:
      type: object
      properties:
        nothing:
          type: string
          example: nothing