Skip to content

Feat: Added AWS Athena data loader #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Chenglong-MS merged 7 commits into from
Jan 30, 2026
+570 −2
Merged

Feat: Added AWS Athena data loader #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
eebe31a
[deploy] Merge pull request #226 from microsoft/dev
Chenglong-MS Jan 27, 2026
c3992eb
Merge pull request #228 from microsoft/dev
Chenglong-MS Jan 27, 2026
80ed0ca
Feat: Added AWS Athena data loader
azwanzuharimi Jan 29, 2026
4c9428a
Ensure compatibility with Python >=3.9+ and validation of s3 path
azwanzuharimi Jan 29, 2026
51f3f88
Update data loader README
azwanzuharimi Jan 29, 2026
a3a2b75
Update py-src/data_formulator/data_loader/athena_data_loader.py
Chenglong-MS Jan 29, 2026
0fed94c
Update py-src/data_formulator/data_loader/athena_data_loader.py
Chenglong-MS Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions py-src/data_formulator/data_loader/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,15 @@ The UI automatically provide the query completion option to help user generate q

### Example Implementations

- `AthenaDataLoader`: AWS Athena integration (SQL queries on S3 data lakes)
- `BigQueryDataLoader`: Google BigQuery integration
- `KustoDataLoader`: Azure Data Explorer (Kusto) integration
- `MySQLDataLoader`: MySQL database integration
- `PostgreSQLDataLoader`: PostgreSQL database integration
- `MSSQLDataLoader`: Microsoft SQL Server integration
- `S3DataLoader`: Amazon S3 file integration (CSV, Parquet, JSON)
- `AzureBlobDataLoader`: Azure Blob Storage integration
- `MongoDBDataLoader`: MongoDB integration

### Testing

Expand Down
6 changes: 4 additions & 2 deletions py-src/data_formulator/data_loader/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from data_formulator.data_loader.postgresql_data_loader import PostgreSQLDataLoader
from data_formulator.data_loader.mongodb_data_loader import MongoDBDataLoader
from data_formulator.data_loader.bigquery_data_loader import BigQueryDataLoader
from data_formulator.data_loader.athena_data_loader import AthenaDataLoader

DATA_LOADERS = {
"mysql": MySQLDataLoader,
Expand All @@ -16,7 +17,8 @@
"azure_blob": AzureBlobDataLoader,
"postgresql": PostgreSQLDataLoader,
"mongodb": MongoDBDataLoader,
"bigquery": BigQueryDataLoader
"bigquery": BigQueryDataLoader,
"athena": AthenaDataLoader
}

__all__ = ["ExternalDataLoader", "MySQLDataLoader", "MSSQLDataLoader", "KustoDataLoader", "S3DataLoader", "AzureBlobDataLoader","PostgreSQLDataLoader", "MongoDBDataLoader", "BigQueryDataLoader", "DATA_LOADERS"]
__all__ = ["ExternalDataLoader", "MySQLDataLoader", "MSSQLDataLoader", "KustoDataLoader", "S3DataLoader", "AzureBlobDataLoader", "PostgreSQLDataLoader", "MongoDBDataLoader", "BigQueryDataLoader", "AthenaDataLoader", "DATA_LOADERS"]
Loading