Gracefully handle file upload persistence errors by clearing state

docs/features/code-based/COMPONENTS.md

@@ -0,0 +1,144 @@
+---
+layout: default
+title: Components
+parent: Code-based Features
+grand_parent: Features
+nav_order: 6
+---
+
+# Components
+
+This guide covers key concepts for developing components.
+
+## Overview
+
+Components are the building blocks of forms in the engine. Components usually extend from two base classes:
+
+- ComponentBase: general components that display content on the page, such as Markdown or HTML components.
+
+- FormComponents: these components are specialised components that take user input. They represent individual form fields and controls (text inputs, file uploads, radios, checkboxes, etc.) and handle their own validation, state (data) and rendering.
+
+> [!NOTE]
+> Custom components are currently not supported when registering the plugin. Whilst you can develop custom pages as a plugin consumer, custom components must be built into the core engine.
+
+## Post-processing component state on form submission
+
+### Overview
+
+The `InvalidComponentStateError` is a specialised exception thrown by form components during submission when their external state has become invalid or inconsistent with their internal state. This mechanism provides a graceful recovery path for components that interact with external systems, allowing the form engine to reset the component's state and prompt the user to re-enter data rather than failing the entire submission.
+
+### Why it's necessary
+
+#### Internal vs external state
+
+The forms engine automatically validates **internal state** - data stored in the user's session that represents their answers to form fields. However, some components maintain **external state** in addition to internal state:
+
+- **Internal State**: Data stored in the user's session (e.g., file upload IDs, reference numbers)
+- **External State**: Data stored in external systems (e.g., files in S3, payment records in a payment provider)
+
+#### The problem
+
+External state can become invalid between the time a user enters data and when they submit the form. For example:
+
+- **File Uploads**: A user uploads files, receiving file IDs representing an external resource. Those files are stored in S3 with a retrieval key. Later, when submitting the form:
+  - The files may have expired (TTL exceeded)
+  - The retrieval key may have become invalid
+  - The files may have been deleted from S3
+- **Payments**: A user initiates a payment and returns to the form. When submitting:
+  - The payment session may have expired
+  - The payment may have been cancelled externally
+
+#### The solution
+
+Rather than failing the entire submission with an unrecoverable error, components can throw `InvalidComponentStateError` to trigger a controlled recovery process:
+
+1. The component throws the error with a user-friendly message
+2. The form engine catches the error
+3. The component's internal state is cleared from the session
+4. The user is redirected back to the component's page
+5. An error message is displayed to the user
+6. The user can re-enter the data and continue
+
+### How it works
+
+Components throw the exception in their `onSubmit()` method when they detect invalid external state. The component does not handle state clearing - it only throws the error.
+
+#### Submission flow
+
+```
+1. User clicks "Submit" on summary page
+  ↓
+2. SummaryPageController.handleFormSubmit()
+  ↓
+3. finaliseComponents() - validates external state for each component
+  ↓
+4. component.onSubmit() called for each component
+  ↓
+5. Component validates external state
+  ↓
+6a. [Success Path]                    6b. [Invalid State Path]
+  External state valid                  External state invalid
+  ↓                                     ↓
+  Continue submission                   throw InvalidComponentStateError
+                                        ↓
+                                      7. SummaryPageController catches error
+                                        ↓
+                                      8. Flash error message to user
+                                        ↓
+                                      9. CacheService.resetComponentStates()
+                                        - Clears component data from session
+                                        ↓
+                                      10. Redirect to component's page
+                                        ↓
+                                      11. User sees error and can re-enter data
+```
+
+#### Catching the exception (controller level)
+
+The exception is caught and handled by the **SummaryPageController** during form submission.
+
+**Location**: `src/server/plugins/engine/pageControllers/SummaryPageController.ts`
+
+The controller:
+
+1. Catches `InvalidComponentStateError` during `finaliseComponents()`
+2. Creates a GOV.UK error message object
+3. Flashes the error to the user's session
+4. Calls `cacheService.resetComponentStates()` to clear the component's state
+5. Redirects the user back to the component's page
+
+### How to implement it for a component
+
+Override the `onSubmit()` method in your component class. This method is called during form submission to finalise any external state.
+
+```typescript
+import { InvalidComponentStateError } from '~/src/server/plugins/engine/pageControllers/errors.js'
+
+async onSubmit(
+  request: FormRequestPayload,
+  metadata: FormMetadata,
+  context: FormContext
+) {
+  const value = this.getData(context.state)
+
+  if (!value) {
+    return // No data to validate
+  }
+
+  try {
+    // Attempt to validate/finalise external state
+    await this.validateExternalState(value)
+  } catch (error) {
+    // Check if this is a recoverable error
+    if (this.isRecoverableError(error)) {
+      throw new InvalidComponentStateError(
+        this,
+        'There was a problem with your [data type]. Please [action] before submitting the form again.'
+      )
+    }
+
+    // Non-recoverable errors should be re-thrown
+    throw error
+  }
+}
+```

src/server/constants.js

@@ -2,3 +2,4 @@ export const PREVIEW_PATH_PREFIX = '/preview'
 export const FORM_PREFIX = ''
 export const EXTERNAL_STATE_PAYLOAD = 'EXTERNAL_STATE_PAYLOAD'
 export const EXTERNAL_STATE_APPENDAGE = 'EXTERNAL_STATE_APPENDAGE'
+export const COMPONENT_STATE_ERROR = 'COMPONENT_STATE_ERROR'

src/server/forms/register-as-a-unicorn-breeder.yaml

@@ -306,5 +306,4 @@ lists:
         value: Aquatic
       - text: Rainbow
         value: Rainbow
-outputEmail: defraforms@defra.gov.uk
 startPage: '/whats-your-name'

src/server/forms/simple-form.yaml

@@ -0,0 +1,64 @@
+---
+name: Page events
+engine: V2
+schema: 2
+startPage: '/summary'
+pages:
+  - title: Your name
+    path: '/your-name'
+    components:
+      - type: TextField
+        title: What is your first name?
+        name: applicantFirstName
+        shortDescription: Your first name
+        hint: ''
+        options:
+          required: true
+        schema: {}
+        id: 1fb8e182-c709-4792-8f83-e01d8b1fee1a
+      - type: TextField
+        title: What is your last name?
+        name: applicantLastName
+        shortDescription: Your last name
+        hint: ''
+        options:
+          required: true
+        schema: {}
+        id: b68df7f1-d4f4-4c17-83c8-402f584906c9
+    next: []
+    id: 622a35ec-3795-418a-81f3-a45746959045
+  - title: Upload a copy of your passport
+    controller: FileUploadPageController
+    path: '/upload-passport'
+    components:
+      - type: FileUploadField
+        title: Please upload a copy of your passport
+        name: passportUpload
+        shortDescription: Upload passport
+        hint: ''
+        options:
+          required: true
+        schema: {}
+        id: 987c1234-56d7-89e0-1234-56789abcdef0
+    id: 23456789-0abc-def1-2345-67890abcdef1
+  - title: ''
+    path: '/date-of-birth'
+    components:
+      - type: DatePartsField
+        title: When is {{ applicantFirstName }} {{ applicantLastName }}'s birthday?
+        name: dateOfBirth
+        shortDescription: Your birthday
+        hint: ''
+        options:
+          required: true
+        schema: {}
+        id: '00738799-3489-4ab2-a57b-542eecb31bfa'
+    next: []
+    id: da0fbdb4-a2de-4650-be16-9ba552af135f
+  - id: 449a45f6-4541-4a46-91bd-8b8931b07b50
+    title: ''
+    path: '/summary'
+    controller: SummaryPageController
+conditions: []
+sections: []
+lists: []

src/server/plugins/engine/beta/form-context.test.ts

@@ -209,11 +209,12 @@ describe('getFormModel helper', () => {
 
 describe('resolveFormModel helper', () => {
   const slug = 'tb-origin'
-  const definition = { pages: [], outputEmail: 'fallback@example.com' }
+  const definition = { pages: [] }
   const metadata = {
     id: 'metadata-123',
     live: { updatedAt: new Date('2024-10-15T10:00:00Z') },
-    versions: [{ versionNumber: 9 }]
+    versions: [{ versionNumber: 9 }],
+    notificationEmail: 'enrique.chase@defra.gov.uk'
   }
   let server: Request['server']
   let formModelInstance: { id: string }
@@ -265,7 +266,7 @@ describe('resolveFormModel helper', () => {
     expect(FormModel).toHaveBeenCalledTimes(2)
     expect(mockFormsService.getFormDefinition).toHaveBeenCalledTimes(2)
     expect(mockCheckEmailAddressForLiveFormSubmission).toHaveBeenCalledWith(
-      definition.outputEmail,
+      undefined,
       true
     )
     expect(FormModel).toHaveBeenCalledWith(

src/server/plugins/engine/beta/form-context.ts

@@ -174,9 +174,10 @@ export async function resolveFormModel(
       )
     }
 
-    const emailAddress = metadata.notificationEmail ?? definition.outputEmail
-
-    checkEmailAddressForLiveFormSubmission(emailAddress, isPreview)
+    checkEmailAddressForLiveFormSubmission(
+      metadata.notificationEmail,
+      isPreview
+    )
 
     const routePrefix =
       options.routePrefix ?? server.realm.modifiers.route.prefix