Json validator library
Totara 15.0 introduces a Json validator into core. It provides a generic API that can do cleaning and validating of input json. The term "input json" here means json data that we get from an external response, or it could be the json data that produced by one of our internal tools which is generated due to interactions from the end users, for example: Weka editor.
This document outlines how to use the json validator library in code. The json validator is an adapter for the third party json validator libraries to be used within our system. Currently we are using OPIS json schema library as the implementation of the validator.
Structure
The json schema structure can be found here https://json-schema.org/. The structure is just a json that describe the schema of the json text that we are about to validate. There are two ways to validate the json text with the structure.
Define structure on the go
With this approach, it is very easy to bring something up quickly, the downside is that the schema structure defined on the go would be harder to reuse as a sub schema in some other places.
<?php
define('CLI_SCRIPT', true);
require_once(__DIR__ . '/server/config.php');
use core\json\validation_adapter;
use core\json\structure\structure;
use core\json\type;
$structure = json_encode([
'type' => type::OBJ,
'properties' => [
'id' => [
'type' => type::INT,
],
'username' => [
'type' => type::STR
]
]
]);
$validator = validation_adapter::create_default();
$result = $validator->validate_by_json_structure(
json_encode([
'id' => 15,
'name' => 'bruce_wayne'
]),
$structure
);
echo $result->get_error_message();
Define structure within the specific namespace class
With this approach the system can index the structure by its name, and use this structure when external caller requires it.
<?php
// File totara/contentmarketplaces/linkedin/classes/core_json/structure/learning_asset.php
namespace contentmarketplace_linkedin\core_json\structure {
use core\json\structure\structure;
use core\json\type;
class learning_asset extends structure {
public static function get_definition(): string {
return json_encode([
'type' => type::OBJ,
'properties' => [
'urn' => [
'type' => type::STR,
],
'title' => [
'type' => type::OBJ,
'properties' => [
'value' => [
'type' => type::STR
],
'locale' => [
'type' => type::OBJ,
'properties' => [
'language' => [
'type' => type::STR,
'maximum' => 2,
'minimum' => 2,
],
'country' => [
'type' => type::STR,
'maximum' => 2,
'minimum' => 2
]
],
'required' => ['language']
],
],
'required' => ['value', 'locale']
]
],
'required' => ['urn', 'title']
]);
}
}
}
And to invoke the structure
<?php
define('CLI_SCRIPT', true);
require_once(__DIR__ . '/server/config.php');
use core\json\validation_adapter;
$json_data = [
'urn' => 'urn:li:lyndaCourse:252',
'title' => [
'value' => 'Python crash course 101',
'locale' => [
'language' => 'en',
'country' => 'US'
]
]
];
$validator = validation_adapter::create_default();
$validator->validate_by_structure_name($json_data, 'learning_asset', 'contentmarketplace_linkedin');
The advantages are:
- The json schema that had been defined in the class structure can be reused as a sub schema in other schema.
- Easier to generate developer document with the json schema.
- Ability to fetch the json schema from the external json schema storage. This can be easily done as the apprach above (define structure on the go), however, with this encapsulation in place, we can do all sorts of logics around caching the external schema and so on.
Note
To define a structure that can be indexed by the system, the structure class must live within a special namespace, in our case it should be under 'core_json\structure' within the plugin.
Advances structure
The json validator also supports several edge cases that we had identified before with the first proposal such as:
- Validate the collection of mixed objects
- Validate the collection of mixed scalar
The examples above are more likely about define the schema structure for the common cases, which we know exactly what the object's properties and such would look like.
For the edge case like validate the collection of mixed objects, the structure would look like the example below:
AnyOf
There are different operators when it comes to validate the collection of mixed objects, anyOf is one of those operators. With anyOf operator, at least one object is valid by validate against one of the sub schema defined in the list, then the result is valid.
For more reference you can look up to OPIS dev doc https://opis.io/json-schema/1.x/multiple-subschemas.html#anyof
// Collection of mixed objects.
$structure = json_encode([
'type' => type::ARRAY,
structure::ANY_OF => [
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'id' => [
'type' => type::INT,
'minimum' => 1
],
'name' => [
'type' => type::STR,
]
],
'required' => ['id', 'name']
]
],
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'uuid' => [
'type' => type::STR,
'maxLength' => 100
],
'uu_name' => [
'type' => type::STR,
],
],
'required' => ['uuid', 'uu_name']
]
]
]
]);
And the validation
<?php
define('CLI_SCRIPT', true);
require_once(__DIR__ . '/server/config.php');
use core\json\validation_adapter;
$validator = validation_adapter::create_default();
$result = $validator->validate_by_json_structure(
json_encode([
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
],
[
'x' => 'y',
'z' => 15
]
]),
$structure
);
echo $result->get_error_message(); // Empty -> this is happening because there are two objects had been matched with the schema.
$result = $validator->validate_by_json_structure(
json_encode([
[
'x' => 'y',
'z' => 15
]
]),
$structure
);
echo $result->get_error_message(); // Error - due to the object does not match with one of the either.
Note
OneOf
With oneOf operator, the collection of mixed object will only be valid if it is a collection of one type or the other, must be exactly one
$structure = json_encode([
'type' => type::ARRAY,
structure::ONE_OF => [
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'id' => [
'type' => type::INT,
'minimum' => 1
],
'name' => [
'type' => type::STR,
]
],
'required' => ['id', 'name']
]
],
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'uuid' => [
'type' => type::STR,
],
'uu_name' => [
'type' => type::STR,
]
],
'required' => ['uuid', 'uu_name']
]
],
]
]);
And the validation
<?php
define('CLI_SCRIPT', true);
require_once(__DIR__ . '/server/config.php');
use core\json\validation_adapter;
$validator = validation_adapter::create_default();
$result = $validator->validate_by_json_structure(
json_encode([
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Error message, because we can only accept one or the other.
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
],
[
'x' => 'y',
'z' => 15
]
]),
$structure
);
echo $result->get_error_message(); // Error message
AllOf
The json is valid against this operator only if all of the collection has all of of the objects that matches with sub schema. Meaning that if we are having two different types of object's schema within the collection, then there must be two two of them appears within the collection in order to pass the validation.
$structure = json_encode([
'type' => type::ARRAY,
structure::ALL_OF => [
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'id' => [
'type' => type::INT,
'minimum' => 1
],
'name' => [
'type' => type::STR,
]
],
'required' => ['id', 'name']
]
],
[
'contains' => [
'type' => type::OBJ,
'properties' => [
'uuid' => [
'type' => type::STR,
],
'uu_name' => [
'type' => type::STR,
]
],
'required' => ['uuid', 'uu_name']
]
],
]
]);
And the validation
<?php
define('CLI_SCRIPT', true);
require_once(__DIR__ . '/server/config.php');
use core\json\validation_adapter;
$validator = validation_adapter::create_default();
$result = $schema->validate_by_json_structure(
json_encode([
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Error, due to missing the second object
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
]
]),
$structure
);
echo $validator->get_error_message(); // Error, due to missing the first object
// Valid case
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
]
]),
$structure
);
echo $result->get_error_message(); // Empty
$result = $validator->validate_by_json_structure(
json_encode([
[
'uuid' => '15',
'uu_name' => 'Hello world'
],
[
'id' => 15,
'name' => 'Hello world'
],
[
'x' => 'y',
'z' => 15
]
]),
$structure
);
echo $result->get_error_message(); // Error message, due to one unexpected object.
Data format
The data format term is the custom validator that we can add to the schema which allow us to define what the format of the value will looks like.
For example: PARAM_ALPHA is a sort of data format which we can only have the a-z characters without any other characters.
To define a custom data format classes.
<?php
namespace contentmarketplace_linkedin\core_json\data_format;
use core\json\type;
class urn extends data_format {
/**
* @return string
*/
public function get_for_type(): string {
return type::STR;
}
/**
* @param string $value
* @return bool
*/
public function validate($value): bool {
if (!is_string($value)) {
// Prevent array, or object to validate by the clean param.
return false;
}
return false !== strpos($value, ':');
}
}
Note
The data format class must live in a special namespace within a plugin in order to allow system wire it up. The special namespace is 'core_json\\data_format'
Important
The name of the format to use within the json schema is a concatenated string of plugin/component name and name of the class, with hyphen as a delimiter
For those data formats that sit in core, the name would just be the class name only.
And the structure that require the data format will look like below:
$structure = json_encode([
'type' => type::OBJ,
'properties' => [
'urn' => [
'type' => type::STR,
'format' => 'contentmarketplace_linkedin-urn' // Please note that the name of the data_format that lives in different plugin
// is always a concatenated string of the plugin component name and the actualname of the class.
],
],
]);