Resolving OpenAPI specification references

If you have ever used OpenAPI with golang, you’re probably aware of using $ref to reference other objects like Schemas or Parameters that we want to re-use across a specification, or across multiple specifications.

liboenapi knows how to look up these references and resolve them.

What does resolving mean?

A $ref is a JSON Pointer that points to a location in the specification. The location could be local to the current document, or it could be in another document on the local file system, or out there in cyberspace somewhere.

Resolving is a two-step process, the first being locating the document (local, file or remote) and then finding the object being pointed to inside that document.

The second process is to then create a new document but with the $ref instances, replaced with the actual object referenced.

How libopenapi resolves references

First of all we create an index of the specification that provides a lookup for everything in the OpenAPI specification.

Local references

The index will locate every $ref and determine what type of lookup to perform, if the reference is local, there is no prefix attached to the reference path, so it looks something like this:

  $ref: "#/components/schemas/Burger"

The # (hash) symbol indicates the root of the document.

The index looks up the component from within the same document and keeps a record of the found object.

File references

If the reference is a file reference, it will look something like this:

  $ref: "some/path/to/openapi-schemas.yaml#/components/schemas/Burger"

The prefix (before the hash) is the path to the file that contains the lookup. libopenapi will attempt to locate the file on the local operating system and then load it into the index. If the path is relative, the resolver will look from where it is being run.

When a file reference is used, a whole new index is created for the entire file, so the rolodex, maintains indexes for every file pulled in.

Remote references

If the reference is a remote reference, it will look something like this:

  $ref: "https://pb33f.io/openapi.yaml#/components/schemas/Burger"

Like file references, the prefix (before the hash) is the URI to the file that contains the lookup. libopenapi will pull down this remote document and then treat it as a local reference internally.

When a remote reference is used, a whole new index is created for the entire remote file (like file references).

Learn more about remote/relative references.

Sticking it all together

So far, we have only talked about the rolodex and index and how they locate references. The resolver is actually responsible for looking at every reference in the document, and then locating the object it references from the index(es) and then replacing the reference with the actual object.

This is not useful_ for humans, but it is useful for machines, because it means that the document is now self-contained.

The resolver exists because it (alongside libopenapi) was born inside vacuum.

vacuum does not use the data model to operate, it works on the low level *yaml.Node structure and needs a fully resolved node tree to work correctly.

Why do I need a resolver?

If we’re just using the data model to work with OpenAPI specifications, we don’t need to resolve specifications.

However, if our application has a use-case like vacuum in which the application needs to recurse through an entire OpenAPI specification and treat everything as in-line, for the sake of validation - then a resolver is what’s need.

How do I use the resolver?

In v0.12 and below The resolver and data model were decoupled from one another, meaning that you could use the resolver without the data model, and vice versa.

In v0.13+ the relationship model between these elements has changed. Now the resolver and the index are all tightly coupled together and are not designed to be used separately.

This is because a new element called the rolodex was introduced that provides much more power over how references are discovered and looked up, before being resolved.

To resolve an OpenAPI specification, it’s best to create a rolodex first, and then use that rolodex to resolve the specification.

package main

import (
  "fmt"
  "github.com/pb33f/libopenapi/index"
  "gopkg.in/yaml.v3"
  "io"
  "net/http"
)

// read the swagger petstore OpenAPI Specification
func readPetstore() []byte {
  res, err := http.Get("https://petstore3.swagger.io/api/v3/openapi.json")
  if err != nil {
      panic(err) 
  }
  defer res.Body.Close()
  data, err := io.ReadAll(res.Body)
  if err != nil {
      panic(err) 
  }
  return data
}

// Resolve the petstore OpenAPI specification
func petstore() {

  // create a root node to unmarshal the spec into.
  var rootNode yaml.Node
  _ = yaml.Unmarshal(readPetstore(), &rootNode)

  // create a new config that does not allow lookups.
  indexConfig := index.CreateClosedAPIIndexConfig()

  // create a new rolodex
  rolodex := index.NewRolodex(indexConfig)

  // set the rolodex root node to the root node of the spec.
  rolodex.SetRootNode(&rootNode)

  // index the rolodex
  indexedErr := rolodex.IndexTheRolodex()
  if indexedErr != nil {
    panic(indexedErr)
  }

  // resolve the petstore
  rolodex.Resolve()

  // extract the resolver from the root index.
  resolver := rolodex.GetRootIndex().GetResolver()

  // print out some interesting information discovered when visiting all the references.
  fmt.Printf("%d errors repored\n", len(rolodex.GetCaughtErrors()))
  fmt.Printf("%d references visited\n", resolver.GetReferenceVisited())
  fmt.Printf("%d journeys taken\n", resolver.GetJourneysTaken())
  fmt.Printf("%d index visits\n", resolver.GetIndexesVisited())
  fmt.Printf("%d relatives seen\n", resolver.GetRelativesSeen())
}

Detecting circular references

Circular references are complex, they are not errors in a traditional sence, but they are something that needs to be handled (or ignored) by the application.

The code to check for circular references is the same, except we call the CheckForCircularReferences() method instead of the Resolve() method.

  // check for circular references
  rolodex.CheckForCircularReferences()

read more about circular references

Error types

When resolving, anything that goes wrong along the way is recorded as a resolver.ResolvingError and returned as a slice of errors.

Anything other than a circular reference should be considered something that went wrong during resolution. The most common error is missing local or remote references, or un-locatable ones.