Added robots.txt endpoint

[emalinowski]

New Features

Added robots.txt

Breaking Changes

Bug Fixes

Improvements

Dependency updates

Deployment changes

[github-actions]

filepath	passed	error	skipped	SUBTOTAL
tests/test_oauth2.py	15	0	0	15
tests/test_dicom_viewer.py	4	0	0	4
tests/test_centralized_auth.py	16	0	0	16
tests/test_graph_submit_and_query.py	13	0	1	14
tests/test_audit_service.py	3	0	3	6
tests/test_pfb_export.py	1	0	0	1
tests/test_data_upload.py	8	0	1	9
tests/test_guppy_service.py	8	0	0	8
tests/test_presigned_url.py	7	0	0	7
tests/test_dbgap.py	4	0	1	5
tests/test_requestor.py	5	1	0	6
tests/test_drs_endpoint.py	4	0	0	4
tests/test_study_viewer.py	3	0	0	3
tests/test_user_token.py	5	0	0	5
tests/test_user_data_library.py	2	0	1	3
tests/test_oidc_client.py	2	0	0	2
tests/test_gen3client.py	1	0	0	1
tests/test_manifest_service.py	1	0	0	1
tests/test_login_page.py	2	0	0	2
tests/test_register_user.py	2	0	0	2
tests/test_discoverypage.py	1	0	0	1
tests/test_aggregate_mds.py	1	0	0	1
tests/test_workspace.py	1	0	0	1
tests/test_client_credentials.py	1	0	0	1
tests/test_google_data_access.py	1	0	0	1
tests/test_homepage.py	1	0	0	1
tests/test_gen3_workflow.py	0	0	11	11
tests/test_ras_authn.py	0	0	3	3
tests/test_gen3ff_landing_page.py	0	0	2	2
tests/test_env_sanity.py	0	0	1	1
tests/test_etl.py	0	0	1	1
TOTAL	112	1	25	138

Please find the detailed integration test report here

Please find the Github Action logs here