ApiBlaze: Parsing and Searching Elements in OpenApi Specs

ApiBlaze is a tool to explore API specifications: Search for a keyword, filter for objects, properties, or endpoints, and immediately see descriptions and code examples. ApiBlaze helps you to answer a specific question about an API lightning fast. You can try it here: apiblaze.admantium.com.

The format of an OpenAPI spec is a verbose, metadata rich JSON structure. Searching in this structure is not ideal, especially when a long list of object properties need to be traversed, or when the response or request objects of API endpoints need to be searched. Therefore, ApiBlaze generates an internal data structure that makes it easier for parsing.

In this blog post, I cover the details of building this structure, fulfilling this core requirement of ApiBlaze:

  • SEL01 — Distinguish objects, properties and endpoints
  • SEL02 — Search for API Elements by keywords

This article originally appeared at my blog.

Challenges when Parsing OpenAPI Spec

The information that is relevant for ApiBlaze are the paths (API endpoints) and the definitions (data structures for requests and responses). And here, OpenAPI allows the definition and usage of references. For example, you have a standard response object for all API endpoints. Instead of defining this data structure at all endpoints, you define it once in the definitions section, and then include a reference from the endpoint to this definition.

Lets consider an example from the Kubernetes spec for a pod. A pod has the spec property, which is defined as a $ref to io.k8s.api.core.v1.PodSpec. And this object has the property affinity, which is again a Sref to io.k8s.api.core.v1.Affinity. And so on. These references need to be resolved.

"io.k8s.api.core.v1.Pod": {
"properties": {
"spec": {
"$ref": "#/definitions/io.k8s.api.core.v1.PodSpec",
"description": "Specification of the desired behavior of the pod. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status"
},
// ...
}
}
"io.k8s.api.core.v1.PodSpec": {
"description": "PodSpec is a description of a pod.",
"properties": {
"affinity": {
"$ref": "#/definitions/io.k8s.api.core.v1.Affinity",
"description": "If specified, the pod's scheduling constraints"
},
// ...
}
}
"io.k8s.api.core.v1.Affinity": {
"description": "Affinity is a group of affinity scheduling rules.",
"properties": {
"nodeAffinity": {
"$ref": "#/definitions/io.k8s.api.core.v1.NodeAffinity",
"description": "Describes node affinity scheduling rules for the pod."
},
// ...
}
}

The second challenge are nested data structures, Once the references are resolved, we will have a nested object. If we want to access the Pod Affinity from the example above, we would need the path ["io.k8s.api.core.v1.Pod"].properties.spec.properties.affinity.properties.nodeAffinity. This should be simplified to ["io.k8s.api.core.v1.Pod"].spec.affinity.nodeAffinity.

Let’s see how to solve these two challenges.

Resolving References

async function loadApiModel (specFile) {
const dereferencedSpec = await SwaggerParser.dereference(specFile)
return {
definitions: dereferencedSpec.definitions,
endpoints: dereferencedSpec.paths
}
}
const apiSpec = await loadApiModel('./k8s.json')

This function returns the object { definitions: {}, endpoints: {}}, on which we continue with searching.

Searching

  1. Check the passed keyword(s)
  2. Create a regular expression
  3. Find matching occurrences of this keywords in objects, properties and endpoints
  4. Sort all results by their score

Here is an excerpt of the search function:

search(keyword) {
keyword.match(/ /) ? keyword = keyword.split(' ').map(tag => `(${tag})`).join('|') : ''
return [
...this._search(this.indexedObjects, keyword),
...this._search(this.indexedProperties, keyword),
...this._search(this.indexedEndpoints, keyword)
].sort((a,b) => b.score - a.score)
}
_search(list, keyword) {
return [
...list
.filter(i => JSON.stringify(i).match(new RegExp(keyword), 'ig'))
.map(i => Object.assign(i, {score: JSON.stringify(i).match(new RegExp(keyword, 'ig')).length}))
]
}

Resolving nested Structures

Transforming the nested structure to a simpler one is handled by the drillDown function. It recursively calls itself to resolve all the properties key and replace them with their children.

_drillDownObject(item) {
item.properties = this._drillDownProperties({}, this.definitions[item.containingObject].properties)
return item
}
_drillDownProperties (stack, properties) {
if (!properties || !Object.entries(properties)) {
return stack;
}
for (let [key, values] of Object.entries(properties)) {
switch (values.type) {
case 'string':
stack[key] = { _type: 'string', _description: values.description};
break;
case 'object':
stack[key] = { _type: 'object', _description: values.description, ...this._drillDownProperties({}, values.properties) };
break;
case 'array':
stack[key] = { _type: 'array', _description: values.description, ...this._drillDownProperties({}, values.items.properties) };
break;
}
}
return stack;
}

This results in this structure:

{
name: "Pod",
containingObject: "io.k8s.api.core.v1.Pod"
properties: {
affinity: {
nodeAffinity: {
type: "string",
description: "Describes node affinity scheduling rules for the pod."
}
}
}
}

Review: ApiBlaze Project Requirements

Searching for APIS

  • ✅ SEA01 — Search for APIs by Keyword
  • ✅ SEA02 — Show search results in a popup
  • ✅ SEA03 — Select a search results with arrow keys, enter and mouse click

Searching API Elements

  • ✅ SEL01 — Distinguish objects, properties and endpoints
  • ✅ SEL02 — Search for API Elements by keywords

Framework

  • ✅ FRAME01 — Controller & Routing
  • ✅ FRAME02 — Stateful Pages & Components
  • ✅ FRAME03 — Actions
  • ✅ FRAME04 — Optimized Bundling

Technologies

  • ✅ TECH01 — Use PlainJS & Custom Framework
  • ✅ TECH02 — Use SAAS for CSS
  • ✅ TECH03 — Use WebSockets to Connect Frontend and Backend

Conclusion

IT Project Manager & Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store