Implement a clinical safety process

[PBRichardson]

I'm proposing a process is written and adopted that addresses clinical safety explicitly rather than just through good practice. In the UK healthcare software products are required to have such a process wrapped around the development process. It functions in much the same way as with vulnerabilities; the aim being to:

• adopt an approach and processes that minimise the likely introduction of a clinical safety issue
• permit 3rd parties to report any suspected incidents of clinical safety issues (i.e. both internally via Github and publically via a form/email)
• ensure that GH issues are tagged if they are deemed to have a potential CS impact
• have a mechanism for communicating that a CS issue has been identified
• have an accelerated response to such issues

Again, not just looking to create work, but this is commercially important as well as important for clinical safety, and so I'm happy to put together some ideas with my team and some CS experts for others to comment on. The aim will be for light touch processes, that are not tangential to the standard dev and release pipeline.

[PBRichardson]

For an example of an issue that we might want to tag as have potential CS impact, there is this.

[mgaffigan]

@PBRichardson, I'm all for ensuring process results in quality, and agree that software can have a critical role in good and bad outcomes. That said, I'm not sure I understand what specific changes you are proposing.

My thoughts around improving quality (which should be proximate to clinical safety) would be to:

• Ensure there is a communicated "SLA" for the product, primarily to identify the expected failure circumstances.
• Ensure there is a notification process to communicate any defect which can result in deviation from the SLA.
• Use software development best practice (code control, testing, CI/CD, etc...) to identify and prevent defects before release.
• Include in box those tools required for channel developers to use change management, testing, RBAC, and transport security

For a proposed SLA:

• The system may:
  • Fail to process, refuse, lose, or indefinitely delay any message, fractionally or in a whole, with or without generating errors.
  • Store data in unencrypted form on any server involved in the operation of the engine, including indefinitely
  • Store or Transmit data to any third party explicitly configured in a secure or unsecure form, as may be configured
  • Run any supplied by an authorized user without limits
• It represents a broken guarantee of the system to:
  • Lose a message which has been ack-ed on a channel appropriately configured
  • Disclose information to any third party not explicitly configured
  • Allow remote API access except by an authenticated user of the system
  • Corrupt or mis-parse a validly formatted message

I'm sure I'm missing a few, but given the "building-block" nature of Mirth, and the incredible ecosystem of backends, plugins, and systems to which it communicates, I think it will be challenging to form a stronger set of guarantees. You surely have a greater experience in clinical safety than I do, so I'm interested in any techniques that have worked well for other interface engines or other "ball-of-clay" style projects. I can't find any single interface engine listed as a SaMD in the US, so it's hard to find any baselines.

A team operating the engine can form a much stronger set of guarantees, since they have a narrower use case and also control the environment. They can set monitoring to identify deviation from expectation. They also are responsible for ensuring that channels are developed with sufficient care and attention to failure cases.

I'm not sure I see the relation in #93 to being a clinical safety issue. It is an error, but except in the most expansive definition I don't see how that makes it a clinical safety issue. Safe systems must operate safely in the presence of errors. I'm all for requiring tests / excess design documentation / risk mitigation on PR's that can break the SLA - but in the same way that most bugs can be called security bugs, I think most bugs can be called clinical safety issues.

[PBRichardson]

Hi Mitch, thank for your thoughts. I think that we are largely on the same page actually.

I think that the SLA bullet points you list are excellent for expectation setting. It's quite possible that being as upfront as that will probably lead to some pushback from potential deployers/users, but that's fine, it will inform our roadmap.

I think I would add 'unauthorised access to the Mirth Connect Administrator' to the list, to mirror the API access, though I guess you could argue that the latter prevents the former, so on reflection that may be unnecessary.

I agree that for an integration engine, there is at least an equal set of responsibilities that falls on the the downstream parties, i.e. whoever builds the plugin, channel, code template, and ultimately deploys OIE. They will have to answer to their own clinical safety stipulations.

I agree with your list of mechanisms for improving/ensuring quality in general, though I would still argue for increasing awareness of clinical safety among all contributors, and those on the SC who hold the torch for OIE. It may go without being said, but e.g. I would argue that it is worth adding a paragraph to the contributor guidelines to emphasis the point that, given the very nature of OIE, bugs have the potential to cause clinical harm and that contributors should bear that in mind, and specifically consider whether test cases could be written to help avoid issues that might cause clinical harm. A list of examples could be given, e.g. mixing up patient identifiers, losing important clinical context, and 'silently' failing in a way that means an issue with messaging isn't investigated.

I think that OIE's responsibilities in relation to CS lie mainly in the realm of:

• encouraging an enhanced focus on clinical safety issues, especially for new contributors
• putting in place the processes that I listed in my original post

Re: whether this issue has potential CS impact or not, I'm not qualified to judge, but I was highlighting how the process might work, i.e. someone with CS responsibility, who may well be a lay person in relation to understanding the code, would read the description, see that the description suggests that different threads are interfering and thus may have a CS impact, so adds a 'For CS review' tag, which is then picked up for analysis by those more familiar with the code, and only if they agree that there is a CS impact under some stated circumstances, would the process kick off by which a remedy is accelerated, and those who have deployed the version with the bug are notified - just in the same way a security vulnerability is handled.

Does this help to explain the CS perspective. I'm a business owner with a software engineering background, but I'm coming at this from the perspective of having had many discussions with Clinical Safety Officers whose job it is to prepare clinical safety cases demonstrating that best practice has been adopted by those who develop products and those who deploy them. Tbh open source health informatics has probably had a free ride to date purely because it hasn't gone through a procurement process; those processes in the UK will always have standard clauses that are aimed at CS.