Features-ui: Add download audit modals, ZIP bundle generation, and PDF proxy endpoints by nando-bingani · Pull Request #14 · SAEON/odp-ui

nando-bingani · 2026-02-04T13:24:21Z

Implement client-side download audit system with user metadata collection, ZIP bundle generation, and PDF proxy endpoints for CORS-free architecture.

Key Features:

Download Audit Modals - Collect user details (name, email, organisation) before downloads
Client-Side ZIP Generation - JSZip library creates bundles with metadata PDFs + data files
PDF Proxy Endpoint - Flask proxy to ODP API for CORS-free PDF generation
Form Caching - LocalStorage saves user details (30-day expiry)
Email Validation - Client-side validation with regex pattern
Download Audit Logging - Posts audit data to /catalog/download-audit proxy

New Features:

Download Modal (#download-popup) - Single file downloads from detail page
Bundle Modal (#download-audit-modal) - Multi-record ZIP downloads
ZIP Generation - Client-side bundle creation with JSZip
PDF Proxy - /catalog/format/metadata.pdf endpoint
Audit Proxy - /catalog/download-audit endpoint forwards to ODP API

Files Modified:

odp/ui/base/views/catalog.py - Proxy endpoints (PDF + audit)
odp/ui/base/static/scripts/catalog.js - Download handlers, ZIP generation, form caching
odp/ui/base/templates/catalog_index.html - Search results with checkboxes
odp/ui/base/templates/catalog_record.html - Detail page download button
odp/ui/base/macros/catalog.j2 - Download modals and buttons

Bug Fixes:

Fixed binary PDF response handling (requests.post() instead of cli.post())
Fixed download-audit proxy (removed .json() on already-parsed response)
Removed unused variables and debug console.log statements
Removed commented-out code blocks

Production Ready:

All debug code removed
Clean code with no print statements
Proper error handling
Email validation enforced

Browser Compatibility:

Chrome, Firefox, Safari, Edge (all modern browsers)
LocalStorage for form caching
JSZip 3.10.1 for ZIP generation

…set in catalog.j2 and a few css customs.

…cen.gov

…ngle record

…set in catalog.j2 and a few css customs.

…cen.gov

dylanpivo · 2026-02-05T11:50:35Z

odp/ui/base/macros/catalog.j2

+                                   onclick="toggleSelectAll(this)">
+                            <label class="form-check-label" for="select_all">Select All</label>
+                        </div>
+{#                        <div class="form-check">#}


Don't leave commented out code. Remove it if it's not needed

dylanpivo · 2026-02-05T11:54:38Z

odp/ui/base/macros/catalog.j2

+{% macro result_list(records,app_name=None) %}
+
+    <form id="recordForm">
+        {% if app_name == 'MIMS' %}


Maybe move the code in this if into it's own macro and call that macro here.

I think it would be better if this file didn't know about things like MIMS. The ides is for these to be re-usable chunks of code. So maybe pass in a variable called something like: 'show_bulk_download_options'. You can also set that to false by default which means you will only have to pass in true when you're calling it from MIMS and you don't have to change anything else from other areas.

dylanpivo · 2026-02-05T12:03:12Z

odp/ui/base/macros/catalog.j2

+                                   onclick="toggleSelectAll(this)">
+                            <label class="form-check-label" for="select_all">Select All</label>
+                        </div>
+{#                        <div class="form-check">#}


If this code is not needed remove it rather than commenting it out

dylanpivo · 2026-02-05T12:06:20Z

odp/ui/base/macros/catalog.j2

+{% macro result_list(records,app_name=None) %}
+
+    <form id="recordForm">
+        {% if app_name == 'MIMS' %}


Maybe move the code that lies in this if to it's own macro. This will help declutter the current macro.

This macro probably shouldn't know about app names like MIMS and such. It might be better to pass in a variable like 'show_download_options' that defaults to false.

dylanpivo · 2026-02-05T12:06:27Z

odp/ui/base/macros/catalog.j2

+        </span>
+    </button>
+
+{#                                    <button#}


Commented code

dylanpivo · 2026-02-05T12:18:04Z

odp/ui/base/macros/catalog.j2

+                                    <button class="btn btn-link p-0 no-outline"
+            type="button"
+            onclick="openSingleDownloadModal('{{ download_url }}', '{{ record.doi }}', '{{ record.id }}')">
+        <img src="{{ url_for('static', filename='images/download-icon.png') }}"


Please have a look at the formatting of this file. Try keep the indenting right so that it's more readable.

dylanpivo · 2026-02-05T12:31:51Z

odp/ui/base/macros/catalog.j2

-                                {%- if not loop.last %}, {% endif %}
-                            </span>
+                        {% set download_urls = [] %}
+                        {% for record in records %}


Lets look into this flow.

dylanpivo · 2026-02-05T12:32:38Z

odp/ui/base/macros/catalog.j2

+
+    </form>
+
+    <div class="modal fade" id="download-popup" tabindex="-1">


A lot of this looks like a repetition of the above?

dylanpivo · 2026-02-06T07:26:19Z

odp/ui/base/macros/catalog.j2

-                            (No DOI)
-                        {% endif %}
-                    </span>
+{% macro result_list(records,app_name=None) %}


Maybe change the variable to something like "show_download_options" so that this file doesn't have to know about MIMS and such.

dylanpivo · 2026-02-06T07:27:48Z

odp/ui/base/macros/catalog.j2

+                                   onclick="toggleSelectAll(this)">
+                            <label class="form-check-label" for="select_all">Select All</label>
+                        </div>
+{#                        <div class="form-check">#}


Commented out code should be removed if it's not being used

dylanpivo · 2026-02-06T07:39:42Z

odp/ui/base/macros/catalog.j2

+{% macro result_list(records,app_name=None) %}
+
+    <form id="recordForm">
+        {% if app_name == 'MIMS' %}


It would maybe make this more readable if the code within this 'if' was moved to it's own macro and the macro was called here

dylanpivo · 2026-02-06T07:40:15Z

odp/ui/base/macros/catalog.j2

+        </span>
+    </button>
+
+{#                                    <button#}


Commented code

dylanpivo · 2026-02-06T07:43:58Z

odp/ui/base/macros/catalog.j2

-                                {%- if not loop.last %}, {% endif %}
-                            </span>
+                        {% set download_urls = [] %}
+                        {% for record in records %}


I'm not sure this should be stored in the DOM. Lets discuss

dylanpivo · 2026-02-06T07:46:15Z

odp/ui/base/macros/catalog.j2

+
+    </form>
+
+    <div class="modal fade" id="download-popup" tabindex="-1">


This looks very similar to the modal above

dylanpivo · 2026-02-06T07:50:09Z

odp/ui/base/macros/catalog.j2

-                {% elif prop == 'Keywords' %}
-                    {{ record.keywords | sort | join(', ') }}
+            {% elif prop == 'Keywords' %}
+                {% for keyword in record.keywords | sort %}


Will this maybe be better in a server side function that gets the keywords data in the right state before it's delivered to the template

dylanpivo · 2026-02-06T07:51:23Z

odp/ui/base/macros/catalog.j2

+                                <input class="form-check-input" type="checkbox" id="accept-terms-of-use-popup">
+                                <label class="form-check-label" for="accept-terms-of-use-popup">
+                                    <small class="text-muted">
+                                        <strong>Data Usage Acknowledgment:</strong> These data are made available with the express understanding that any such use will properly acknowledge the originator(s) and publisher and cite the accession numbers and/or associated Digital Object Identifiers. Anyone wishing to use these data should properly cite and attribute the data providers listed as authors in the metadata provided with each dataset. It is expected that all the conditions of the data license will be strictly honoured. Use of any material herein should be properly cited using the dataset's persistent identifiers, such as accession numbers and DOIs.


This has been repeated 3 times

dylanpivo · 2026-02-06T07:52:04Z

odp/ui/base/macros/submissions.j2

I'm not sure this should be here?

dylanpivo · 2026-02-06T08:00:12Z

odp/ui/base/static/scripts/catalog.js

+    }
+}
+
+function clearDownloadCache() {


This isn't being used apparently

dylanpivo · 2026-02-06T08:04:26Z

odp/ui/base/static/scripts/catalog.js

+    window.location.href = redirectUrl;
+}
+
+function selectedRecordListLink(event) {


Also not in use apparently

dylanpivo · 2026-02-06T08:06:29Z

odp/ui/base/static/scripts/catalog.js

+    updateButtonStates();
+}
+
+function toggleUnSelectAll() {


dylanpivo · 2026-02-06T08:10:41Z

odp/ui/base/static/scripts/catalog.js

+    if (cb?.type === 'checkbox') cb.checked = true;
+}
+
+function downloadSelectedRecords(event, buttonEl, record_id) {


Small issue, but pay attention to variable naming conventions. snake_case in python and camelCase in js. Then the actual name as well.

dylanpivo · 2026-02-06T08:30:21Z

odp/ui/base/static/scripts/catalog.js

+    const disclaimerCheckbox = document.getElementById('disclaimer-acknowledgment');
+
+    // Validate all required fields
+    if (!nameInput.value.trim()) {


This approach of validating and handling the requests in JS works, but it's not traditionally how the systems handles this sort of thing. We would generally have a form within the modal with a submit target. The validation would be handled by wtforms and the submission logic would be handled in a view

dylanpivo · 2026-02-06T08:35:54Z

odp/ui/base/templates/catalog_subset.html

+
+                    </div>
+                            {{ pagination }}
+{#                    <div class="d-flex justify-content-end p-3">#}


commented code

dylanpivo · 2026-02-06T08:42:10Z

odp/ui/base/views/catalog.py

+
+
+@bp.route('/format/metadata.pdf', methods=['POST'])
+def format_metadata_pdf():


This is an unusual code flow. You are taking the metadata from the front-end, sending it to the back-end for pdf generation and then sending it back? Should it not just send a list of record id's back which will build the pdf by getting the metadata from the db and then sending it back, or something along those lines?

dylanpivo · 2026-02-06T08:45:36Z

odp/ui/base/views/catalog.py

+        # The cli.post() method tries to parse JSON, but PDF is binary
+        # So we use requests directly to get the binary content
+        api_url = config.ODP.API_URL
+        response = requests.post(


We use either the api or cli objects so that communication between the client and API is authorised.

dylanpivo · 2026-02-06T08:46:56Z

odp/ui/base/static/scripts/catalog.js

+    event.preventDefault();
+    const targetInput = document.getElementById('record-subset-link');
+    if (!targetInput) {
+        console.error("Missing target input 'record-subset-link'");


please look out for console logs

dylanpivo · 2026-02-06T08:49:03Z

odp/ui/base/views/catalog.py

-from flask import Blueprint, abort, current_app, make_response, redirect, render_template, request, url_for
+from flask import Blueprint, abort, current_app, make_response, redirect, render_template, request, url_for, Response, \
+    jsonify, send_file
+from io import BytesIO


Format inputs. Check shortcuts. Mine is Ctrl+Alt+o

dylanpivo · 2026-02-06T08:50:59Z

odp/ui/base/macros/catalog.j2

        </div>
-    {% endfor %}
+
+        <div class="modal fade" id="download-audit-modal" tabindex="-1" aria-labelledby="downloadModalLabel" aria-hidden="true">


Please check out the formatting, Especially with HTML. Use the shortcuts to help. Mine is Ctrl+Alt+l

dylanpivo · 2026-02-06T09:15:40Z

odp/ui/base/views/catalog.py

+    page = 1  # request.args.getlist('page')[0]
+
+    size = 5  # request.args.getlist('size')[0]
+    catalog_record_list = cli.get(f'/catalog/{catalog_id}/subset?{record_ids_query}&page={page}&size={size}')


the convention is to pass the page and size in as parameters rather than on the url string. Have a look at index()

dylanpivo

If we can sit and talk about the flow that would be great

- Replaced client-side JSZip with server-side ZIP generation - Simplified _performZipDownload() to use new /catalog/generate-zip-bundle endpoint - Removed manual audit logging from frontend (server handles it automatically) - Reduced frontend code: 73 lines of JSZip → 40 lines of clean server call - Added UI proxy endpoint for CORS handling (UI port 8080 → API port 2020) - Fixed modal field ID mismatch by passing userData as parameter Benefits: - fewer HTTP requests (2N+1 → 1) - Lower browser resource usage - atomic audit logging - No CORS errors Author: nando-bingani <n.bingani@saeon.nrf.ac.za>

- Clean up: Removed commented-out code and unused JavaScript functions in `catalog.j2`, `catalog_subset.html`, and `catalog.js`. - Refactor (Jinja): - Replaced 'app_name' check with 'show_bulk_download_options' boolean in `result_list` macro. - Extracted download audit logic into its own reusable macro `download_audit_modal`. - Improved HTML indentation and formatting across template files. - Refactor (Python/Views): - Updated `subset_record_list` to pass 'page' and 'size' as parameters to `cli.get` rather than string formatting the URL. - Standardized facet formatting and removed internal-only facets ('Keyword', 'SDG Variables') in a centralized helper. - Fixed imports and removed redundant Response/jsonify definitions. - JavaScript: - Renamed variables to use camelCase (e.g., `recordId`) to match JS conventions. - Removed production console logs. - Download Flow: - Migrated download validation to use a formal WTForm (`DownloadAuditForm`) and server-side ZIP generation via an API proxy.

nando-bingani added 30 commits September 10, 2025 10:49

added catalog.html, created new route subset in catalog. rendered sub…

2098d19

…set in catalog.j2 and a few css customs.

added size and page on the url, added a share link on the records

c5ab0ae

fixed list to get the [0] argument

29d865b

modified the share button

4a25ce7

subset view

644b0be

subset view

7c4f893

added download button on each catalog

6a03e71

Allowing multiple downloads, and only download urls from repository.o…

665f4c0

…cen.gov

multiple downloads

9c86f36

downloading metata files in cross orign

4c084c5

download subset fix

715147b

added pdf on metadata creation

be9a782

pdf template update more readable

03ca0b9

fixed share select issue, auto select share record

e17ba46

fixed the download button to download record metadata and data for si…

4c82ada

…ngle record

added a download all button with out a function

08ec646

added catalog.html, created new route subset in catalog. rendered sub…

ff3a5ed

…set in catalog.j2 and a few css customs.

added size and page on the url, added a share link on the records

df1a071

fixed list to get the [0] argument

0cc0320

modified the share button

9f8c206

subset view

a10cf01

subset view

6ad2e82

added download button on each catalog

8adc581

Allowing multiple downloads, and only download urls from repository.o…

bd304c1

…cen.gov

multiple downloads

2d9b1a3

downloading metata files in cross orign

88d2f64

download subset fix

7df2e4d

added pdf on metadata creation

8a8183d

pdf template update more readable

d0fa841

fixed share select issue, auto select share record

935a645

nando-bingani requested review from dylanpivo and gracezhou-tech February 4, 2026 13:24

dylanpivo reviewed Feb 6, 2026

View reviewed changes

odp/ui/base/macros/catalog.j2 Outdated

</span>

</button>

{# <button#}

Copy link
Copy Markdown

Contributor

dylanpivo Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented code

dylanpivo reviewed Feb 6, 2026

View reviewed changes

odp/ui/base/macros/submissions.j2 Outdated

Copy link
Copy Markdown

Contributor

dylanpivo Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this should be here?

dylanpivo reviewed Feb 6, 2026

View reviewed changes

odp/ui/base/static/scripts/catalog.js Outdated

updateButtonStates();

}

function toggleUnSelectAll() {

Copy link
Copy Markdown

Contributor

dylanpivo Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used

dylanpivo reviewed Feb 6, 2026

View reviewed changes

dylanpivo requested changes Feb 6, 2026

View reviewed changes

nando-bingani force-pushed the v2-prod-protologue branch from 88a9851 to 1096888 Compare February 10, 2026 16:48

nando-bingani force-pushed the v2-prod-protologue branch from 1096888 to 1e6d42e Compare February 10, 2026 16:56

nando-bingani added 3 commits February 11, 2026 10:08

current stable ui version needs revisions

4edd324

used a fix for checkboxes, removed unused functions

26ceae0


		</form>

		<div class="modal fade" id="download-popup" tabindex="-1">



		@bp.route('/format/metadata.pdf', methods=['POST'])
		def format_metadata_pdf():

Conversation

nando-bingani commented Feb 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dylanpivo Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dylanpivo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dylanpivo Feb 6, 2026 •

edited

Loading