Circular References in OpenAPI

How libopenapi detects and warns about them.

This documentation only applies to v0.13+ of libopenapi. In all releases before then the index and resolver architecture was significantly different and the behavior and relationship of them was also different.

View the older documentation for v0.12 and below

Circular references are loops in a data structure or Schema that causes loops.

components:
  schemas:
    One:
      properties:
        thing:
          "$ref": "#/components/schemas/Two"
      required:
        - thing
    Two:
      description: "test two"
      properties:
        testThing:
          "$ref": "#/components/schemas/One"

This creates a circular reference. One depends Two which depends on One One –> Two –> One –>… and so on.

The effect of ‘required’

A circular reference isn’t considered something that’s broken, unless required is set to true. When a schema is required AND it’s part of a loop - an unrecoverable infinite loop.

Detecting circular references

Internally, the resolver looks at every single reference found in the document, by using the index to look them all up, and then it follows the dependency graph for each and every other reference found.

There are opportunities to use references everywhere, and in Schemas, it can get quite messy. Polymorphic dependencies can create all kinds of chaos at scale.

libopenapi will find them all, and be able to detect if they are infinte loops.

There are two ways to detect circular references programmatically:

Automatically with a model

When building a data model, A resolver instance is created during the build, and runs the CheckForCircularReferences().

CheckForCircularReferences() is a non-destructive operation on the resolver that follows the same code-path as the resolver (so it can detect all the circles), however it doesn’t actually resolve anything or change the tree in anyway.

If any infinite loops were detected, they are collected up and returned as a slice of resolver.ResolvingError and returned by the second argument of the BuildVXModel().

It’s easy to extract details about the circular reference from the ResolvingError because, there will be a pointer to a index.CircularReferenceResult as a part of the error, available via the CircularReference property of the ResolvingError struct.

For example:

import (
  "errors"
  "fmt"
  "github.com/pb33f/libopenapi"
  "github.com/pb33f/libopenapi/index"
  "os"
)

func circles() {

  // load an OpenAPI 3 specification with circular refs from bytes
  circularBytes, _ := os.ReadFile("test_specs/circular-tests.yaml")

  // create a new document from specification bytes
  document, _ := libopenapi.NewDocument(circularBytes)

  // build a v3 model from the document.
  _, modelErrors := document.BuildV3Model()

  // print out resolving modelErrors:
  // a slice of modelErrors will be returned
  for i := range modelErrors {

    // check if this is a resolving error.
    var err *index.ResolvingError
    if errors.As(modelErrors[i], &err) {

      // check if there is a circular reference attached.
      if err.CircularReference != nil {
        fmt.Printf("Circular reference found: %s\n",
            err.CircularReference.GenerateJourneyPath())
      }
    }
  }
}

Using the rolodex

If your application’s use case does not require a model, you can perform the check independently using the rolodex (this is what the model builder does under the hood)

import (
  "fmt"
  "github.com/pb33f/libopenapi/index"
  "gopkg.in/yaml.v3"
  "os"
)

func checkCirclesFromRolodex() {

  // load an OpenAPI 3 specification with circular refs from bytes
  circularBytes, _ := os.ReadFile("test_specs/circular-tests.yaml")

  // create a root node to unmarshal the spec into.
  var rootNode yaml.Node
  _ = yaml.Unmarshal(circularBytes, &rootNode)

  // create a new config that does not allow lookups.
  indexConfig := index.CreateClosedAPIIndexConfig()

  // create a new rolodex
  rolodex := index.NewRolodex(indexConfig)

  // set the rolodex root node to the root node of the spec.
  rolodex.SetRootNode(&rootNode)

  // index the rolodex
  resolvingErrors := rolodex.IndexTheRolodex()

  // print out circular errors detected during indexing.  
  fmt.Println(resolvingErrors.Error())
}

Circular reference results

When a circular reference is detected, a index.CircularReferenceResult is created that contains all the information about the circular reference, including the path taken to get there.

// CircularReferenceResult contains a circular reference found when traversing the graph.
type CircularReferenceResult struct {
  Journey             []*Reference
  Start               *Reference
  LoopIndex           int
  LoopPoint           *Reference
  IsArrayResult       bool   // if this result comes from an array loop.
  PolymorphicType     string // which type of polymorphic loop is this? (oneOf, anyOf, allOf)
  IsPolymorphicResult bool   // if this result comes from a polymorphic loop.
  IsInfiniteLoop      bool   // if all the definitions in the reference loop are marked as required
}

When a polymorphic or array based circular reference is detected, the IsPolymorphicResult or IsArrayResult properties are set. This is useful for ignoring these types of circular references, as they may be valid use cases for a particular specification design.

The PolymorphicType property is set when the IsPolymorphicResult is set to true, and will be set to either oneOf, anyOf or allOf depending on the type of polymorphic loop detected.

Polymorphic circular references

Using oneOf, anyOf or allOf can create circular references, and they are detected in the same way as any other loop, however these may actually be valid use cases for a particular specification design. In some cases, it may be desirable to loop in common properties.

Here is an example of a polymorphic circular reference:

components:
  schemas:
    One:
      properties:
        thing:
          oneOf:
            - "$ref": "#/components/schemas/Two"
            - "$ref": "#/components/schemas/Three"
      required:
        - thing
    Two:
      description: "test two"
      properties:
        testThing:
          "$ref": "#/components/schemas/One"
    Three:
      description: "test three"
      properties:
        testThing:
          "$ref": "#/components/schemas/One"

This creates a circular reference. One depends Two which depends on One –> Two –> One –>… and so on.

Ignore polymorphic references

To stop libopenapi from complaining about circles found through polymorphic references, you can tell the rolodex to simply ignore them by setting the IgnorePolymorphicCircularReferences to true on the index.SpecIndexConfig configuration when creating the rolodex.

indexConfig := index.CreateClosedAPIIndexConfig()
indexConfig.IgnorePolymorphicCircularReferences = true

// create a new rolodex
rolodex := index.NewRolodex(indexConfig)

If you’re building an OpenAPI document, there is a configuration property named IgnorePolymorphicCircularReferences on the datamodel.DocucmentConfiguration struct to ignore polymorphic circular references when building a model at a high level.

var spec = `openapi: 3.1.0
...`

docConfig := datamodel.NewClosedDocumentConfiguration()

// ignore polymorphic circular references
docConfig.IgnorePolymorphicCircularReferences = true

document, err := NewDocumentWithConfiguration([]byte(spec), config)
...

Array based circular references

It’s common to see circular references in arrays, and they are detected in the same way as any other loop, however these may actually be valid use cases for a particular specification design. In some cases, it may be desirable to loop in common properties.

Here is an example of an array based circular reference

components:
  schemas:
    ProductCategory:
      type: "object"
      properties:
        name:
          type: "string"
        children:
          type: "array"
          items:
            $ref: "#/components/schemas/ProductCategory"
          description: "Array of sub-categories in the same format."
      required:
        - "name"
        - "children"

Ignore array references

To stop libopenapi from complaining about circles found through array based references, you can tell the rolodex to simply ignore them by setting the IgnoreArrayCircularReferences to true on the index.SpecIndexConfig configuration when creating the rolodex.

indexConfig := index.CreateClosedAPIIndexConfig()
indexConfig.IgnoreArrayCircularReferences = true

// create a new rolodex
rolodex := index.NewRolodex(indexConfig)

If you’re building an OpenAPI document, there is a configuration property named IgnoreArrayCircularReferences on the datamodel.DocucmentConfiguration struct to ignore array based circular references when building a model at a high level.

var spec = `openapi: 3.1.0
...`

docConfig := datamodel.NewClosedDocumentConfiguration()

// ignore array circular references
docConfig.IgnoreArrayCircularReferences = true

document, err := NewDocumentWithConfiguration([]byte(spec), config)
...

Capturing ignored circular references

Don’t want the references to trigger errors, but still want to capture them and use them for analysis or something else?

The rolodex has a GetIgnoredCircularReferences() method available that will return a slice ([]*index.CircularReferenceResult) that contains all the ignored circular references found during indexing.

This can be further broken down into types of circular references via the resolver

// extract the resolver from the root index.
resolver := rolodex.GetRootIndex().GetResolver()

// infinte / unrecoverable circular references
infiniteRefs := resolver.GetInfiniteCircularReferences()

// ignored polymorphic circular references
ignoredPoly := resolver.GetIgnoredCircularPolyReferences()

// ignored array circular references
ignoredArray := resolver.GetIgnoredCircularArrayReferences()