Application dashboard visibility layer

Owned by Chris Snyder
Mar 23, 2023
4 min read

The problem

Determining the collection of applications that a user can see is not a trivial task, because several interrelated capabilities determine who can see applications in various states.

For example, by default, a user who is the manager of other users and is also a workflow approver (for a different set of users) can see:

  • Their own draft applications
  • Submitted applications where they are the subject
  • Submitted applications from others that are pending their approval (the application is at an approval level where they are an approver)
  • Submitted applications from others that are on an assignment where they are an approver at some level (this overlaps with the previous point)
  • Applications submitted by their reports

When loading the applications dashboard view, we need to find the union of all of these visible applications, across all of the workflow assignments in the system. This needs to be done in a single query so that it can be properly sorted and paged.

We originally used has_capability_sql() to determine at query time whether a user had the specific capabilities that needed to be checked for each application condition, but performance testing showed that this approach does not scale as the number of contexts in the system increases.

We needed to pre-compute who could see what, but to do that for every user on a site with many thousands of users would be a daunting task.

Role-capability maps

Our primary solution to the problem is to periodically go through the roles defined in the system, discover which ones have the capabilities we are interested in, and record the contexts where those capabilities are assigned to the roles. We save this map in the role_capability_map table.

There is a scheduled task called Regenerate role capability maps to optimise application dashboard loading that runs every hour to update the map.

The map is also generated the first time the applications dashboard is loaded, if it doesn't already exist. 

Using the role-capability map improved performance, but the joins required to resolve all of the user's role assignments in the contexts we're concerned with were also not easy to optimise. On very large sites they can take a few seconds to generate, making the applications dashboard too slow. 

User-role-capability maps

This led to the creation of a set of user-specific visibility maps that could be computed on demand when necessary (such as the first time the user loads the applications dashboard) and in the background the rest of the time.

There is a map table for each dashboard application visibility capability. Each maps a user to all of the contexts where they have application visibility via the table's capability. So for example, the approval_dashboard_application_any table maps users to all of the mod_approval contexts where they are allowed the mod/approval:view_dashboard_application_any capability.

In the case of view_dashboard_application_pending, the map table also records the approval level where the user is assigned as an approver.

These maps make the applications dashboard queries fast enough to work in real time, as the page is rendering, even for very large sites.

How it works, and how it doesn't

The role capability maps for a user are first generated on demand, and then persist from request to request. Currently they persist for the life of the user's session, plus one request. That is, the first applications dashboard request after a user logs in to the site triggers an ad hoc task to regenerate the user-role-capability maps. Up to a minute later, the maps are replaced by the ad hoc task and the user sees an up-to-date dashboard on their next request.

There is an obvious flaw with this approach, in that a busy dashboard can get very out-of-date before the maps are regenerated, and further solutions have been proposed but not actioned, leaving maps regeneration frequency as an exercise for the developer.

A session cache (capability_map) is used to store the capability map generation status for the user. The cache key is maps_reset, if it is set to 1 that means maps were regenerated this session. If maps_reset is cleared or set to 0, the maps will be regenerated again via ad hoc task on the next request.

It is also possible to regenerate the maps via PHP API: 

$user = \core\entity\user::logged_in();
\mod_approval\data_provider\application\capability_map\capability_map_controller::regenerate_all_maps($user->id);

Troubleshooting performance issues

There is a PHP script in dev/approval/others_applications-performance-test.php which can be used to help troubleshoot applications dashboard performance or visibility issues for a given user:

Write sanitised configuration and applications dashboard performance information to a file.

Usage:
    php others_applications-performance-test.php --output=</path/to/file.txt> --approver=<user id>

Required parameters:
    -o, --output       Full path to writeable file for collecting data
    -a, --approver     User ID of approver for dashboard performance check

Options:
    -h, --help            Print out this help

Example:
  $ php others_applications-performance-test.php -o=/home/vagrant/sf182-performance.txt -a=5