Create audit of export processes

[zoecrosby]

When an export is requested from an audit and user perspective it would be helpful to see the details of the export. If the export is for a back office system they could then check to make sure the export ran ok without having to contact IT, and they would know the value of the export to use that total to compare against what is imported into the back office system to ensure the right file has been imported accordingly.

[bmbc-dev-detec]

This requires some discussion.

Background:
We've introduced an Imports section within the IMS and some widgets to show on the dashboard, which show the latest imports into the system.

Suggestion:
Can we implement something similar for exports - so people know when they ran?

Discussion:

The core IMS system should be kept as simple as possible. Any code or functionality that doesn't directly affect the ability to manage income management should not be introduced into the system. Whilst exports sound similar to imports (it's data moving in or out of the system) a key difference between them is that imports generate data within the IMS. That data needs to be processed - we have to check that account references are validated for example. It seems reasonable to store the results of that processing within the IMS.

To enable exporting data the IMS API provides endpoints to search for various types of data.
The intention is that export integrations plug into this API to extract the required data.
The export integration will then remodel the IMS API data into an appropriate format before sending it on to it's final destination (e.g. a SAP system).

However, data could be read from this API for non export purposes, and right now we don't know the reason an API is being made.

I'm not sure it's in the interest of the IMS itself to know that a request to retrieve data has been actioned:

1. What income management system function does this directly support?
2. It could create an enormous amount of data, which may impact on general system performance
3. How do we record just 'exports' and not general queries for data via the API?

Perhaps it's the responsibility of the export integration itself to report, or expose a mechanism for checking, the status of it's current exports?

There is value in people being able aware that an export worked or not, but does this sit outside of the IMS?
If each export integration stored information about each export that was attempted, then we could use that data to create an export dashboard - a dashboard that could look at all export integrations. We could potentially make that data retrievable via an IMS widget on the dashboard page - meaning the IMS knows nothing directly about exports, but it can display information from an external system on the dashboard.

What are your thoughts @zoecrosby?

[zoecrosby]

I think it would be good if the exports could store information about each export attempt so it can be shown on the dashboard