Skip to content

Custom mocha conformance test reporter#1769

Merged
kriswest merged 4 commits intomainfrom
conformance-runner
Mar 3, 2026
Merged

Custom mocha conformance test reporter#1769
kriswest merged 4 commits intomainfrom
conformance-runner

Conversation

@Roaders
Copy link
Contributor

@Roaders Roaders commented Feb 24, 2026

Describe your change

Implement a custom mocha test reporter. When I started to run the conformance tests I found the runner not very helpful. Nothing would happen for long periods of time and I would think that the process had crashed. It turns out that the runner was just waiting for a test with a long timeout to complete but there was nothing to show that we were waiting for a current test to complete.

This PR creates a custom mocha test runner. The benefits are:

  • We can see what test is currently executing. Hopefully if the test has a long timeout the test name will make this clear so the person running the tests will know that there will be a wait
  • when running the manual tests you can see the instructions as to what to do BEFORE the test completes:
image

Drawbacks:

  • There are nearly 200 lines of code that are new and will need to be maintained. If (probably when) Mocha changes in the future there might be issues updating this code.

I am hoping that this code won't need to change much or at all but future updates to Mocha could be an issue.

I am very open for this PR to be rejected if we don't want to take on the burden of maintaining this. It didn't take long to write this (for our AI overlords) so there is not much lost time at all. On the other hand if this had been in place when I started running the conformance test it would have helped my understanding of what was going on quite a bit.

Contributor License Agreement

  • I acknowledge that a contributor license agreement is required and that I have one in place or will seek to put one in place ASAP.

Review Checklist

  • Issue: If a change was made to the FDC3 Standard, was an issue linked above?
  • CHANGELOG: Is a CHANGELOG.md entry included?
  • API changes: Does this PR include changes to any of the FDC3 APIs (DesktopAgent, Channel, PrivateChannel, Listener, Bridging)?
    • Docs & Sources: If yes, were both documentation (/docs) and sources updated?

      JSDoc comments on interfaces and types should be matched to the main documentation in /docs
    • Conformance tests: If yes, are conformance test definitions (/toolbox/fdc3-conformance) still correct and complete?

      Conformance test definitions should cover all required aspects of an FDC3 Desktop Agent implementation, which are usually marked with a MUST keyword, and optional features (SHOULD or MAY) where the format of those features is defined
    • Schemas: If yes, were changes applied to the Bridging and FDC3 for Web protocol schemas?

      The Web Connection protocol and Desktop Agent Communication Protocol schemas must be able to support all necessary aspects of the Desktop Agent API, while Bridging must support those aspects necessary for Desktop Agents to communicate with each other
      • If yes, was code generation (npm run build) run and the results checked in?

        Generated code will be found at /src/api/BrowserTypes.ts and/or /src/bridging/BridgingTypes.ts
  • Context types: Were new Context type schemas created or modified in this PR?
    • Were the field type conventions adhered to?
    • Was the BaseContext schema applied via allOf (as it is in existing types)?
    • Was a title and description provided for all properties defined in the schema?
    • Was at least one example provided?
    • Was code generation (npm run build) run and the results checked in?

      Generated code will be found at /src/context/ContextTypes.ts
  • Intents: Were new Intents created in this PR?

THIS SOFTWARE IS CONTRIBUTED SUBJECT TO THE TERMS OF THE FINOS CORPORATE CONTRIBUTOR LICENSE AGREEMENT.

THIS SOFTWARE IS LICENSED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OF NON-INFRINGEMENT, ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THIS SOFTWARE MAY BE REDISTRIBUTED TO OTHERS ONLY BY EFFECTIVELY USING THIS OR ANOTHER EQUIVALENT DISCLAIMER IN ADDITION TO ANY OTHER REQUIRED LICENSE TERMS.

@Roaders Roaders requested a review from a team as a code owner February 24, 2026 16:23
@netlify
Copy link

netlify bot commented Feb 24, 2026

Deploy Preview for fdc3 ready!

Name Link
🔨 Latest commit 697140b
🔍 Latest deploy log https://app.netlify.com/projects/fdc3/deploys/69a5f1dae6f54900088707b0
😎 Deploy Preview https://deploy-preview-1769.preview-fdc3.finos.org
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

554 passed

@github-actions
Copy link

github-actions bot commented Feb 24, 2026

Coverage Report

Commit: 697140b
Base: main@e198b0f

Type Base This PR
Total Statements Coverage  96.57%  96.57% (+0%)
Total Branches Coverage  85.75%  85.75% (+0%)
Total Functions Coverage  95.32%  95.32% (+0%)
Total Lines Coverage  96.65%  96.65% (+0%)
Details (changed files)
FileStatementsBranchesFunctionsLines
Details (all files)
FileStatementsBranchesFunctionsLines
packages/fdc3-agent-proxy/src/DesktopAgentProxy.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/index.ts 100% 100% 62.5% 100%
packages/fdc3-agent-proxy/src/apps/DefaultAppSupport.ts 88% 50% 100% 88%
packages/fdc3-agent-proxy/src/channels/DefaultChannel.ts 78.94% 77.77% 71.42% 78.94%
packages/fdc3-agent-proxy/src/channels/DefaultChannelSupport.ts 98.94% 93.1% 100% 98.9%
packages/fdc3-agent-proxy/src/channels/DefaultPrivateChannel.ts 97.5% 66.66% 100% 97.5%
packages/fdc3-agent-proxy/src/heartbeat/DefaultHeartbeatSupport.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/intents/DefaultIntentResolution.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/intents/DefaultIntentSupport.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/listeners/AbstractListener.ts 100% 60% 100% 100%
packages/fdc3-agent-proxy/src/listeners/DefaultContextListener.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/listeners/DefaultIntentListener.ts 100% 77.77% 100% 100%
packages/fdc3-agent-proxy/src/listeners/DesktopAgentEventListener.ts 90.47% 85.71% 100% 90.47%
packages/fdc3-agent-proxy/src/listeners/EventListener.ts 10% 0% 0% 10%
packages/fdc3-agent-proxy/src/listeners/HeartbeatListener.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/listeners/PrivateChannelEventListener.ts 93.33% 72.72% 100% 93.33%
packages/fdc3-agent-proxy/src/messaging/AbstractMessaging.ts 94.59% 100% 80% 94.59%
packages/fdc3-agent-proxy/src/util/AbstractFDC3Logger.ts 100% 94.11% 100% 100%
packages/fdc3-agent-proxy/src/util/Logger.ts 100% 100% 100% 100%
packages/fdc3-agent-proxy/src/util/throwIfUndefined.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/index.ts 100% 100% 28.57% 100%
packages/fdc3-get-agent/src/messaging/MessagePortMessaging.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/messaging/message-port.ts 97.43% 86.66% 100% 97.43%
packages/fdc3-get-agent/src/sessionStorage/DesktopAgentDetails.ts 97.36% 89.47% 100% 97.36%
packages/fdc3-get-agent/src/strategies/DesktopAgentPreloadLoader.ts 100% 77.77% 100% 100%
packages/fdc3-get-agent/src/strategies/FailoverHandler.ts 100% 76.47% 100% 100%
packages/fdc3-get-agent/src/strategies/HelloHandler.ts 94% 81.25% 100% 94%
packages/fdc3-get-agent/src/strategies/IdentityValidationHandler.ts 95.65% 73.33% 100% 95.65%
packages/fdc3-get-agent/src/strategies/PostMessageLoader.ts 98.48% 86.95% 100% 98.46%
packages/fdc3-get-agent/src/strategies/Timeouts.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/strategies/getAgent.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/ui/AbstractUIComponent.ts 97.14% 71.42% 100% 97.01%
packages/fdc3-get-agent/src/ui/DefaultDesktopAgentChannelSelector.ts 100% 75% 100% 100%
packages/fdc3-get-agent/src/ui/DefaultDesktopAgentIntentResolver.ts 100% 90% 100% 100%
packages/fdc3-get-agent/src/ui/NullChannelSelector.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/ui/NullIntentResolver.ts 100% 100% 66.66% 100%
packages/fdc3-get-agent/src/util/Logger.ts 100% 100% 100% 100%
packages/fdc3-get-agent/src/util/Uuid.ts 100% 100% 100% 100%
packages/fdc3-standard/src/index.ts 91.3% 70.83% 60% 95%
packages/fdc3-standard/src/api/AppIdentifier.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/AppIntent.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/AppMetadata.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Channel.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/ContextMetadata.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/DesktopAgent.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/DisplayMetadata.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Errors.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Events.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/GetAgent.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Icon.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Image.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/ImplementationMetadata.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/IntentMetadata.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/IntentResolution.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Listener.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Methods.ts 94.18% 84.28% 96.29% 95.23%
packages/fdc3-standard/src/api/PrivateChannel.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/RecommendedChannels.ts 100% 100% 100% 100%
packages/fdc3-standard/src/api/Types.ts 100% 100% 100% 100%
packages/fdc3-standard/src/context/ContextType.ts 100% 100% 100% 100%
packages/fdc3-standard/src/intents/Intents.ts 100% 100% 100% 100%
packages/fdc3-standard/src/internal/contextConfiguration.ts 100% 100% 100% 100%
packages/fdc3-standard/src/internal/intentConfiguration.ts 100% 100% 100% 100%
packages/fdc3-standard/src/internal/typeHelpers.ts 100% 100% 100% 100%
packages/fdc3-standard/src/ui/ChannelSelector.ts 100% 100% 100% 100%
packages/fdc3-standard/src/ui/Connectable.ts 100% 100% 100% 100%
packages/fdc3-standard/src/ui/IntentResolver.ts 100% 100% 100% 100%
toolbox/fdc3-for-web/fdc3-web-impl/src/BasicFDC3Server.ts 100% 100% 100% 100%
toolbox/fdc3-for-web/fdc3-web-impl/src/ServerContext.ts 100% 100% 100% 100%
toolbox/fdc3-for-web/fdc3-web-impl/src/directory/BasicDirectory.ts 96.87% 84.21% 100% 96.55%
toolbox/fdc3-for-web/fdc3-web-impl/src/handlers/BroadcastHandler.ts 96.21% 86.17% 98.24% 95.97%
toolbox/fdc3-for-web/fdc3-web-impl/src/handlers/HeartbeatHandler.ts 88.23% 71.87% 86.66% 90%
toolbox/fdc3-for-web/fdc3-web-impl/src/handlers/IntentHandler.ts 97.36% 90.47% 100% 96.98%
toolbox/fdc3-for-web/fdc3-web-impl/src/handlers/OpenHandler.ts 97.14% 86.84% 100% 97.14%
toolbox/fdc3-for-web/fdc3-web-impl/src/handlers/support.ts 100% 100% 100% 100%

@github-actions
Copy link

554 passed

@github-actions
Copy link

554 passed

@kriswest kriswest self-requested a review February 26, 2026 15:49
Copy link
Contributor

@kriswest kriswest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Roaders could you add a changelog entry and then I think this is good to merge

@github-actions
Copy link

554 passed

@kriswest kriswest self-requested a review March 2, 2026 12:35
Copy link
Contributor

@kriswest kriswest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to check changes to BrowserTypes and add PR number to Changelog, then good to go - another great little contribution

@@ -1,15 +1,7 @@
// To parse this data:
//
// import { Convert, WebConnectionProtocol1Hello, WebConnectionProtocol2LoadURL, WebConnectionProtocol3Handshake, WebConnectionProtocol4ValidateAppIdentity, WebConnectionProtocol5ValidateAppIdentityFailedResponse, WebConnectionProtocol5ValidateAppIdentitySuccessResponse, WebConnectionProtocol6Goodbye, WebConnectionProtocolMessage, AddContextListenerRequest, AddContextListenerResponse, AddEventListenerRequest, AddEventListenerResponse, AddIntentListenerRequest, AddIntentListenerResponse, AgentEventMessage, AgentResponseMessage, AppRequestMessage, BroadcastEvent, BroadcastRequest, BroadcastResponse, ChannelChangedEvent, ClearContextRequest, ClearContextResponse, ContextClearedEvent, ContextListenerUnsubscribeRequest, ContextListenerUnsubscribeResponse, CreatePrivateChannelRequest, CreatePrivateChannelResponse, EventListenerUnsubscribeRequest, EventListenerUnsubscribeResponse, Fdc3UserInterfaceChannelSelected, Fdc3UserInterfaceChannels, Fdc3UserInterfaceDrag, Fdc3UserInterfaceHandshake, Fdc3UserInterfaceHello, Fdc3UserInterfaceMessage, Fdc3UserInterfaceResolve, Fdc3UserInterfaceResolveAction, Fdc3UserInterfaceRestyle, FindInstancesRequest, FindInstancesResponse, FindIntentRequest, FindIntentResponse, FindIntentsByContextRequest, FindIntentsByContextResponse, GetAppMetadataRequest, GetAppMetadataResponse, GetCurrentChannelRequest, GetCurrentChannelResponse, GetCurrentContextRequest, GetCurrentContextResponse, GetInfoRequest, GetInfoResponse, GetOrCreateChannelRequest, GetOrCreateChannelResponse, GetUserChannelsRequest, GetUserChannelsResponse, HeartbeatAcknowledgementRequest, HeartbeatEvent, IntentEvent, IntentListenerUnsubscribeRequest, IntentListenerUnsubscribeResponse, IntentResultRequest, IntentResultResponse, JoinUserChannelRequest, JoinUserChannelResponse, LeaveCurrentChannelRequest, LeaveCurrentChannelResponse, OpenRequest, OpenResponse, PrivateChannelAddEventListenerRequest, PrivateChannelAddEventListenerResponse, PrivateChannelDisconnectRequest, PrivateChannelDisconnectResponse, PrivateChannelOnAddContextListenerEvent, PrivateChannelOnDisconnectEvent, PrivateChannelOnUnsubscribeEvent, PrivateChannelUnsubscribeEventListenerRequest, PrivateChannelUnsubscribeEventListenerResponse, RaiseIntentForContextRequest, RaiseIntentForContextResponse, RaiseIntentRequest, RaiseIntentResponse, RaiseIntentResultResponse } from "./file";
// import { Convert, AddContextListenerRequest, AddContextListenerResponse, AddEventListenerRequest, AddEventListenerResponse, AddIntentListenerRequest, AddIntentListenerResponse, AgentEventMessage, AgentResponseMessage, AppRequestMessage, BroadcastEvent, BroadcastRequest, BroadcastResponse, ChannelChangedEvent, ClearContextRequest, ClearContextResponse, ContextClearedEvent, ContextListenerUnsubscribeRequest, ContextListenerUnsubscribeResponse, CreatePrivateChannelRequest, CreatePrivateChannelResponse, EventListenerUnsubscribeRequest, EventListenerUnsubscribeResponse, Fdc3UserInterfaceChannels, Fdc3UserInterfaceChannelSelected, Fdc3UserInterfaceDrag, Fdc3UserInterfaceHandshake, Fdc3UserInterfaceHello, Fdc3UserInterfaceMessage, Fdc3UserInterfaceResolve, Fdc3UserInterfaceResolveAction, Fdc3UserInterfaceRestyle, FindInstancesRequest, FindInstancesResponse, FindIntentRequest, FindIntentResponse, FindIntentsByContextRequest, FindIntentsByContextResponse, GetAppMetadataRequest, GetAppMetadataResponse, GetCurrentChannelRequest, GetCurrentChannelResponse, GetCurrentContextRequest, GetCurrentContextResponse, GetInfoRequest, GetInfoResponse, GetOrCreateChannelRequest, GetOrCreateChannelResponse, GetUserChannelsRequest, GetUserChannelsResponse, HeartbeatAcknowledgementRequest, HeartbeatEvent, IntentEvent, IntentListenerUnsubscribeRequest, IntentListenerUnsubscribeResponse, IntentResultRequest, IntentResultResponse, JoinUserChannelRequest, JoinUserChannelResponse, LeaveCurrentChannelRequest, LeaveCurrentChannelResponse, OpenRequest, OpenResponse, PrivateChannelAddEventListenerRequest, PrivateChannelAddEventListenerResponse, PrivateChannelDisconnectRequest, PrivateChannelDisconnectResponse, PrivateChannelOnAddContextListenerEvent, PrivateChannelOnDisconnectEvent, PrivateChannelOnUnsubscribeEvent, PrivateChannelUnsubscribeEventListenerRequest, PrivateChannelUnsubscribeEventListenerResponse, RaiseIntentForContextRequest, RaiseIntentForContextResponse, RaiseIntentRequest, RaiseIntentResponse, RaiseIntentResultResponse, WebConnectionProtocol1Hello, WebConnectionProtocol2LoadURL, WebConnectionProtocol3Handshake, WebConnectionProtocol4ValidateAppIdentity, WebConnectionProtocol5ValidateAppIdentityFailedResponse, WebConnectionProtocol5ValidateAppIdentitySuccessResponse, WebConnectionProtocol6Goodbye, WebConnectionProtocolMessage } from "./file";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Roaders as there are a load of changes in this file, could you check if its the main branch thats wrong or yours? Somehow they are out of sync with each other...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why but whenver I build the repo locally I ALWAYS get changes to BrowserTypes. I usually remember to not include it. I will remove.

CHANGELOG.md Outdated
* Ported FDC3 Conformance Project as-is into the FDC3 Monorepo, just including minimal fixes for typescript compilation. ([#1576](https://github.com/finos/FDC3/pull/1576))
* Added `clearContext` function and associated `contextClearedEvent` to the `Channel` API, to be able to clear specific or all context types from the channel. ([#1379](https://github.com/finos/FDC3/pull/1379))
* Added Conformance tests for FDC3 2.2 ([#1586](https://github.com/finos/FDC3/pull/1586))
* Added custom mocha test runner for conformance tests to better display test progress
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why I can't leave a suggestion, but could you update this to:

* Added custom mocha test runner for conformance tests to better display test progress. ([#1769](https://github.com/finos/FDC3/pull/1769))

i.e. add the PR number

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated changelog and removed changes to BrowserTypes

@Roaders Roaders force-pushed the conformance-runner branch from 9152f5c to 697140b Compare March 2, 2026 20:23
@github-actions
Copy link

github-actions bot commented Mar 2, 2026

554 passed

Copy link
Contributor

@kriswest kriswest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kriswest kriswest merged commit 410b1e7 into main Mar 3, 2026
7 checks passed
@kriswest kriswest deleted the conformance-runner branch March 3, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants