Extending API documentation

When developing new services for Totara's GraphQL APIs, it is important to consider how your services will be documented, as this ensures that they can be easily used by other developers. This page outlines some technical considerations to keep in mind, including information on how to document API services, as well as the associated tooling for generating browsable documentation pages.

Authoring API reference documentation

Authoring guidelines for new documentation

For information on what to write in your API documentation, see our authoring guidelines.

Technical details on writing documentation

Schema comments

GraphQL has native support for exposing schema comments via introspection, meaning the comments on the schema files themselves can be extracted and converted into user reference documentation. This has the advantage that it keeps the explanation close to the schema it references, making it easier to keep up to date.

To document a schema item, include a string surrounded by three sets of double-quotes, either on a single line or across multiple lines like this:

""" This is a single line comment on the foo scalar """
scalar foo

"""
This is a multi-line comment on the `bar` scalar.

You can put as much information as you need in here:
* Markdown
* Is supported
* For basic
* Formatting
"""
scalar bar

Please note that basic markdown is supported, but don't use more complex formatting such as HTML or the special interpolation syntax provided by our documentation tool, as it may not be displayed correctly in all clients.

One additional feature we do support is that we'll automatically convert the string:

https://YOUR-SITE-URL

into the actual URL of the site, so you can use this string within your schema comments to create valid URLs within your site.

When commenting on a schema, it should be possible to put comments on all parts of the schema. In some cases you may need to break up arguments or fields over more lines to make space for the comments. For example:

""" Comment on 'car' type """
type car {
  """ Comment on 'colour' field """
  colour: String!
  """ Comment on 'top_speed' field """
  top_speed(
    """ Comment on top_speed 'format' argument """
    format: speed_options = KPH
  ): Int!
}


""" Comment on 'speed_options' enum """
enum speed_options {
  """ Comment on 'KPH' enum value """
  KPH
  """ Comment on 'MPH' enum value """ 
  MPH
}

extend type Query {
  """ Comment on 'get_cars' query """
  get_cars(
    """ Comment on 'limit' query argument """
    limit: Int = 10
  ): [car!]!
}

Note how the query return type doesn't have a comment - that is obtained from the comment on the type, rather than within the query definition.

metadata.json files

Unfortunately, the GraphQL specification does not provide a mechanism for documenting everything that we want to specify about our schema. To supplement the existing information, the documentation tool we are using to generate reference documentation (SpectaQL) supports providing additional metadata, which is used during the build process. See Building API reference documentation below for more details.

Currently the tool offers support for two additional features, by allowing you to specify a metadata.json file containing the extra information:

1. Marking parts of the schema as 'undocumented' so they won't appear in the reference docs
2. Providing example values for schema objects, which are included in the documentation to show how it can be used.

Location of metadata.json files

The tool requires a single file with all the metadata, however, Totara is an extensible system with the need for plugins to be able to extend core schema. To solve this, we support individual metadata.json files within each plugin, and our documentation build script provides support for merging individual files into a single output file, similar to how .graphqls schema files are combined to generate the overall schema.

Like .graphqls schema files and .graphql persisted query files, metadata.json files can be defined either for all schema (by putting in the webapi/ folder of a particular component) or for a specific endpoint type (by putting in a subfolder such as webapi/ajax/). Some example valid file locations include:

server/mod/perform/webapi/metadata.json				# will be included when building metadata for all schemas.
server/mod/perform/webapi/external/metadata.json	# will only be included when building metadata for external schema.
lib/webapi/metadata.json							# core metadata.json file for all schemas.
lib/webapi/ajax/metadata.json						# core metadata.json file for ajax schema only.
server/local/customplugin/webapi/metadata.json		# metadata.json file in a third party plugin.

Building of single metadata.json file

As part of the documentation build script (see the section below), we merge all the individual metadata.json files for a specified endpoint type to generate a final file, which is used to build the reference documentation.

This is done by merging JSON keys recursively to create a new output JSON object that can be saved to a file. Components must avoid naming collisions to avoid generating an invalid output structure, however, collisions are not expected if individual components only provide metadata for their own schema definition, given the component-based namespacing used within our schema.

Structure of metadata.json files

The metadata file is in JSON format, and should be structured as follows (omit any sections that are not required):

{
	"OBJECT": {
		"Mutation": {
			"fields": {
				"undocumented_mutation": {
					"documentation": {
						"undocumented": true
					}
				}
			}
		},
		"Query": {
			"fields": {
				"undocumented_query": {
					"documentation": {
						"undocumented": true
					}
				},
				"my_query": {
					"args": {
 						"undocumented_arg": {
							"documentation": { "undocumented": true }
						},
						"my_query_arg": {
							"documentation": { "example": "example_for_my_query_arg" }
						}
					}
				}
			}
		},
		"undocumented_object_type": {
			"documentation": {
				"undocumented": true
			}
		},
		"my_object_type": {
			"fields": {
				"my_object_type_field": {
					"documentation": {
						"example": "example_for_my_object_type_field"
					}
				}
			}
		}
	},
	"SCALAR": {
		"undocumented_scalar": {
			"documentation": {
				"undocumented": true
			}
		},
		"my_scalar": {
			"documentation": {
				"example": "example_for_my_scalar"
			}
		}
	},
	"INPUT_OBJECT": {
		"my_input": {
			"inputFields": {
				"undocumented_input_field": {
					"documentation": {
						"undocumented": true
					}
				},
				"my_input_field": {
					"documentation": {
						"example": "example_for_my_input_field"
					}
				}
			}
		}
	},
	"ENUM": {
		"my_enum": {
			"enumValues": {
				"undocumented_enum_value": {
					"documentation": {
						"undocumented": true
					}
				},
				"my_enum_value": {
					"documentation": {
						"example": "example_for_my_enum_value"
					}
				}
			}
		}
	},
	"INTERFACE": {
		"my_interface": {
			"fields": {
				"undocumented_interface_field": {
					"documentation": {
						"undocumented": true
					}
				},
				"my_interface_field": {
					"documentation": {
						"example": "example_for_my_interface_field"
					}
				}
			}
		}
	}
}

See the SpectaQL documentation and example metadata.json file for more details.

Getting the structure of the metadata file so the data is taken into account can be tricky. Here are some tips to get it working:

• Make sure your file is valid JSON, including double quoting all keys and correct use of commas (no missing or unwanted trailing commas). It can be useful to validate your JSON with your IDE or an online JSON validator to pick up any issues.
• Be careful when copying and pasting sections - note that there are inconsistencies in key names (for example: "inputFields" for input types vs. "fields" for object types).
• Ensure you put your references within the correct top-level category (for example input types go in "INPUT_OBJECT" but result types go in "OBJECT", and interfaces are separate too).
• Note that queries and mutations go as a sub-type of the "OBJECT" key, and have a different structure to other types (OBJECT.Query.fields.query_name as opposed to OBJECT.type_name).
• The examples above are all strings, but they can also be JSON objects themselves.
• Note that to document a query you actually need to provide example data for its input and result types, not the query itself.

It can be helpful to start from the example structure above and modify it, rather than trying to generate it from scratch.

Markdown guides

SpectaQL also supports text-based documentation in the form of markdown files that can be displayed in a table of contents in the Introduction section. In Totara, those files are stored in the directory extensions/api_docs/spectaql/guides/, and should be saved with the .md file extension. They are included in the navigation via configuration in extensions/api_docs/spectaql/config.yml.

Other build configuration, including the Totara theme and navigation generation code, can be found in extensions/api_docs/spectaql/.

Building API reference documentation

The in-product API reference documentation for Totara was developed using a third-party tool called SpectaQL. SpectaQL is a NodeJS package which accepts the GraphQL schema (plus some optional metadata) and generates the HTML, CSS and JavaScript needed to display the API reference documentation.

In Totara's implementation, we insert the generated code within a Shadow DOM element within a Totara page in the product, which allows us to keep the SpectaQL and Totara styles isolated from each other to avoid stylesheet conflicts. We implement Totara-specific styling via a SpectaQL theme, and also make use of SpectaQL's data arrangement functionality.

We also mediate the output to allow us to perform permission checks before displaying it, and to allow us to dynamically replace the https://YOUR-SITE-URL URL with the actual Totara site URL within the documentation.

We ship the generated API reference documentation with our tagged releases, so for uncustomised sites on tagged versions the API reference documentation will automatically be available, and will reflect the API services available in that version.

For sites with code customisations that modify the API schema, developers will need to complete a build step to make the API documentation reflect their API changes. The API documentation page will automatically detect if the site schema differs from the released schema and display a warning if it is out of date.

The documentation build process

In order to build the API documentation you will need:

• A working php command with access to the site's dataroot folder
• A working node command with access to the site's dataroot folder

In many cases, you may have a single environment that meets both of these requirements, however, our Docker setup used by many developers keeps PHP and node in separate containers, so it is also possible they won't be available at the same time.

Two-step process (separate PHP and node environments)

Step 1: PHP build

The first step is to run a PHP build step, which completes the following tasks:

• Build the schema for each endpoint type and store it in a separate file (api/schema.{$endpoint_type}.graphqls) in the site's dataroot
• Build the metadata for each endpoint type and store it in a separate file (api/metadata.{$endpoint_type}.json) in the site's dataroot
• Build the site's component navigation structure (the name and title of all installed components) and store it in a file (api/nav.js) in the site's dataroot

The script is executed as follows (starting from the code root folder):

% php server/totara/api/cli/prep_api_docs.php
Schema definitions and metadata saved in /path/to/dataroot/api

If called with the -q flag it will run as before, but only output the name of the directory containing the files without the informational string:

% php server/totara/api/cli/prep_api_docs.php -q
/path/to/dataroot/api

The output files should look something like this:

ls -1 /path/to/dataroot/api
metadata.ajax.json
metadata.dev.json
metadata.external.json
metadata.mobile.json
nav.js
schema.ajax.graphqls
schema.dev.graphqls
schema.external.graphqls
schema.mobile.graphqls

Step 2: Node JS build

Once step 1 is complete, it should be possible to trigger the Node JS build step, which makes use of the files generated in step 1 to build the documentation files.

This can be executed as follows (starting from the code root folder):

cd extensions/api_docs
npm ci
npm run build /path/to/dataroot/api

This will install the SpectaQL dependencies and execute a script which calls the command and generates the documentation build files in the appropriate code folder (extensions/api_docs/build/spectaql/). Once complete, you should find two files in that folder: out.html, which is a single file containing the HTML, CSS and JavaScript for the page, and out.json, which is a JSON file containing some metadata about the build step (used to detect schema changes).

The API reference documentation should now be available on your site via Quick access menu > Development > API > API Documentation.

Single-step process (combined PHP and node environments)

If PHP and node are both available in a single environment, you should be able to generate the documentation with a single command that passes the location of output from the PHP step to the node step as follows (starting from the code root folder):

cd extensions/api_docs; npm ci; php ../../server/totara/api/cli/prep_api_docs.php -q | xargs node build