Bundle OpenAPI references into a single file

Lots of OpenAPI specifications are exploded out across lots of different files. This is great for maintainability, but not so great for portability.

Lots of OpenAPI tools do not support exploded / multi-file OpenAPI specifications. Loading and locating references can be tricky, particularly when they are spread across local or remote file systems, so lots of tools just give up on trying to locate references and just ask for a single monolithic file.

Bundling modes

There are two ways to bundle specs in libopenapi.

• Composed
• Inlined

Composed bundling

Composed bundling will re-structure and re-build the exploded / distributed spec into a new single spec. This new spec will maintain all the references that exist today, except they will all be made local to the single document.

All the references will be detected (as best possible) and then recomposed into the components section of the bundled specification.

TLDR; Everything will be plucked from where it lives and recomposed into a single doc.

Composed bundling was added in v0.22.0

Inlined bundling

Inlined bundling will perform the role of a ‘resolver’ and will extract every reference and replace every $ref with the concrete data referenced. This means every reference will have the same copy of data inlined where the reference once stood.

Inlining means tools that do not support OpenAPI specs with references or multiple files will be able to work.

Inline bundling will create large file sizes.

Composed bundling of raw bytes

// BundleBytesComposed will take a byte slice of an OpenAPI specification and return a composed bundled version of it.
// this is the same as BundleBytes, but it will compose the bundling instead of inline it.
func BundleBytesComposed(bytes []byte, configuration *datamodel.DocumentConfiguration,
	compositionConfig *BundleCompositionConfig) ([]byte, error) {

The bytes are the OpenAPI specification bytes, the a DocumentConfiguration pointer sets up any optional configuration options, and the BundleCompositionConfig looks like this:

// BundleCompositionConfig is used to configure the composition of OpenAPI documents when using BundleDocumentComposed.
type BundleCompositionConfig struct {
	Delimiter string // Delimiter is used to separate clashing names. Defaults to `__`.
}

The Demimiter is used for handling name collisions.

To counter collisions, libopenapi will compose a unique name, and will use the Delimiter as a word separator.

for example: AddUser__corporate__data

The default delimiter is __

Composed bundling of a document

// BundleDocumentComposed will take a v3.Document and return a composed bundled version of it. Composed means
// that every external file will have references lifted out and added to the `components` section of the document.
// Names will be preserved where possible, conflicts will be appended with a number. If the type of the reference cannot
// be determined, it will be added to the `components` section as a `Schema` type, a warning will be logged.
// The document model will be mutated permanently.
//
// Circular references will not be resolved and will be skipped.
func BundleDocumentComposed(model *v3.Document, compositionConfig *BundleCompositionConfig) ([]byte, error) {
	return compose(model, compositionConfig)
}

Inline bundling raw bytes

func BundleBytes(bytes []byte, configuration *datamodel.DocumentConfiguration) ([]byte, error)

This method takes a slice of bytes, and a DocumentConfiguration pointer and returns a slice of bytes and an error if something went wrong.

The returned bytes is a single, fully bundled OpenAPI specification, all external references are resolved and inlined.

Local (references inside the root document) references are not bundled, They stay in place.

Inline bundling a document

The BundleDocument function from the bundler package is a convenience function that takes a v3.Document pointer and returns a slice of bytes and an error if something went wrong:

func BundleDocument(model *v3.Document) ([]byte, error)

Here is an example of checking out the mother of all exploded specifications the Digital Ocean OpenAPI specification and bundling it into a single file:

import (
  "bytes"
  "github.com/pb33f/libopenapi"
  "github.com/pb33f/libopenapi/bundler"
  "github.com/pb33f/libopenapi/datamodel"
  "github.com/stretchr/testify/assert"
  "log"
  "log/slog"
  "os"
  "os/exec"
  "path/filepath"
  "runtime"
  "strings"
)

func main() {

  // check out the mother of all exploded specifications
  tmp, _ := os.MkdirTemp("", "openapi")
  cmd := exec.Command("git", "clone", "https://github.com/digitalocean/openapi", tmp)
  defer os.RemoveAll(filepath.Join(tmp, "openapi"))

  err := cmd.Run()
  if err != nil {
      log.Fatalf("cmd.Run() failed with %s\n", err)
  }

  spec, _ := filepath.Abs(filepath.Join(tmp+"/specification", "DigitalOcean-public.v2.yaml"))
  specBytes, _ := os.ReadFile(spec)

  doc, err := libopenapi.NewDocumentWithConfiguration([]byte(specBytes), &datamodel.DocumentConfiguration{
      BasePath:                tmp + "/specification",
      ExtractRefsSequentially: true,
      Logger: slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
          Level: slog.LevelWarn,
      })),
  }) 
  if err != nil {
      panic(err) 
  }

  v3Doc, errs := doc.BuildV3Model()
  if len(errs) > 0 {
      panic(errs)
  }

  bytes, e := bundler.BundleDocument(&v3Doc.Model)

 //... do something with the bytes
}

And now we have a single, bundled OpenAPI specification that we can use with any tool that doesn’t support exploded specs.