-
Notifications
You must be signed in to change notification settings - Fork 0
Add automatic database-first SQL generation with two-project pattern #52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Added RunSqlPackage task to extract database schema using sqlpackage - Added GenerateSqlProj task to create .sqlproj/.csproj/.fsproj files - Added MSBuild properties for SqlProj generation configuration - Registered new tasks in targets file - Added pipeline targets for SqlProj generation workflow Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Simplified approach: extract DACPAC directly from database - Updated RunSqlPackage to output the extracted DACPAC path - Modified targets to properly pass extracted DACPAC to EF Core generation - Added EfcptGenerateSqlProjFile property for optional project file generation - Fixed syntax error in RunSqlPackage.cs Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Created comprehensive sample demonstrating Database → DACPAC → EF Core Models workflow - Added detailed README explaining the new database-first approach - Sample includes EntityFrameworkCoreProject with EfcptGenerateSqlProj enabled - Documented configuration options and comparison with traditional workflow Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Added EfcptQueryDatabaseSchemaForSqlProj target to compute schema fingerprint before extraction - Schema fingerprint enables incremental builds - DACPAC only extracted when database schema changes - Set _EfcptUseConnectionString flag to integrate with existing fingerprinting logic - Ensures deterministic, fast builds for database-first workflow Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Added Database-First SqlProj Generation section highlighting the new capability - Updated Key Features list to include schema extraction - Added new sample to the samples list - Documented workflow: Live Database → DACPAC → EF Core Models - Highlighted benefits and use cases Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Improved target framework parsing to handle edge cases (netstandard, netcoreapp, etc.) - Fixed process timeout handling to kill hanging processes and prevent resource leaks - Avoided modifying input parameter OutputPath by using local variable - Enhanced robustness and clarity of implementation Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Use sqlpackage /Action:Extract /p:ExtractTarget=Flat to extract individual SQL scripts - Generate properly structured SQL projects with organized folder hierarchy - Add auto-generation warning headers to all SQL files - Implement lifecycle hooks: BeforeSqlProjGeneration, AfterSqlProjGeneration, BeforeEfcptGeneration, AfterEfcptGeneration - Build generated SQL project to DACPAC for EF Core model generation - Workflow: Database → SQL Scripts → SqlProj → DACPAC → EF Core Models - SQL project serves as human-readable artifact that can be extended - .NET Framework 4.7.2 compatibility for relative path calculation Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
|
@copilot We need to ensure our samples are on the latest version of EF Core (10.0.1) |
- Updated aspnet-core-appsettings sample - Updated connection-string-mssql sample - Updated custom-renaming sample - Updated dacpac-zero-config sample - Updated database-first-sqlproj-generation sample - Updated microsoft-build-sql-zero-config sample - Updated schema-organization sample - Updated sdk-zero-config sample All samples now use Microsoft.EntityFrameworkCore 10.0.1 Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
|
@copilot This implementation is not quite what we're after. This should behave a bit more similarly to the Spit-Model generation approach. We want there to be 2 user-created projects: DataAccess and Database. We want both the DataAccess and the Database projects to reference JD.Efcpt.Build. It will use the aforementioned lifecycle hooks to ensure the EF/DataAccess project waits for the SQL Project generation to complete before continuing. The SQL Project generation will only occur in one project, and it's optional. If the SQL Project in the solution does not reference JD.Efcpt.Build, it will not emit the generation events, and the DataAccess project will only have to await its normal compilation. This is largely moot as the existing setup should require core compilation to complete in the Database project before generation starts in the DataAccess project. |
I understand. I'll refactor to:
Working on this now.
Refactored to match the Split-Model pattern in commits c1235f6 and b96abb3. Key changes:
The implementation now follows the same pattern as the split-outputs feature. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
BREAKING CHANGE: Removed EfcptGenerateSqlProj property in favor of automatic detection - Auto-detect SQL projects using SDK properties (SqlServerVersion, DSP) - Generate SQL scripts directly in the SQL project directory - Two-project pattern: DatabaseProject (SQL) + DataAccessProject (EF Core) - SQL project builds to DACPAC, DataAccess project references it - Removed separate SQL project generation and build logic - Simplified lifecycle: SQL generation doesn't build a separate project - New sample demonstrating the two-project pattern - Updated properties: EfcptSqlScriptsDir replaces EfcptSqlProjOutputDir Migration guide: - Remove <EfcptGenerateSqlProj>true</EfcptGenerateSqlProj> - Create separate SQL project with MSBuild.Sdk.SqlProj or Microsoft.Build.Sql SDK - Add JD.Efcpt.Build package reference to SQL project - Set connection string in SQL project - DataAccess project references SQL project via ProjectReference Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Removed database-first-sqlproj-generation sample (old approach) - Updated README with new two-project pattern examples - Updated sample references throughout documentation - Clarified workflow: Database → SQL Scripts → DACPAC → EF Core Models Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
JerrettDavis
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ensure consistent implementation with our existing codebase. Ensure all new code is E2E tested using the same strategies we've employed for other similar operations. Ensure all samples build properly. We need an integration test to validate this scenario is correct.
…tilities - Created DotNetToolUtilities for shared dnx detection and TFM parsing - Removed GenerateSqlProj task (not needed - only generate SQL content) - Updated RunSqlPackage to use shared utilities - Removed test hooks from RunSqlPackage (use E2E tests instead) - Consistent implementation across codebase - .NET Framework 4.7.2 compatibility maintained Addresses review comments: - GenerateSqlProj.cs removed (only SQL content generation needed) - Test hooks removed in favor of E2E testing - Shared logic abstracted to DotNetToolUtilities Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
- Added SqlGenerationIntegrationTests.cs with 8 test scenarios - Tests are currently skipped (require database setup) - Document expected test coverage: * SQL project detection * SQL script generation with proper structure * Auto-generation warnings * Two-project pattern (Database + DataAccess) * Schema fingerprinting / incremental builds * Lifecycle hooks * dnx usage for .NET 10+ * Global tool usage for .NET 8/9 - Added SQL Generation Tests collection definition - Tests follow existing integration test patterns Ready for full E2E implementation with database fixtures. Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
…ype and improve file handling
…platform path issues
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #52 +/- ##
==========================================
+ Coverage 84.20% 87.49% +3.28%
==========================================
Files 47 50 +3
Lines 3007 3350 +343
Branches 399 393 -6
==========================================
+ Hits 2532 2931 +399
- Misses 295 419 +124
+ Partials 180 0 -180
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…configuration properties
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated 24 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
samples/schema-organization/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj
Show resolved
Hide resolved
samples/custom-renaming/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj
Show resolved
Hide resolved
samples/connection-string-mssql/EntityFrameworkCoreProject/EntityFrameworkCoreProject.csproj
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…rameworkCoreProject.csproj Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…workCoreProject.csproj Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ityFrameworkCoreProject.csproj Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…sts.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Code Coverage |
Summary
This PR implements automatic database-first SQL generation for JD.Efcpt.Build using a two-project pattern with consistent implementation practices and comprehensive E2E test coverage.
Code Review Changes Implemented
1. Removed Unnecessary Code:
GenerateSqlProj.cstask deleted - only SQL content generationRunSqlPackage.cs2. Shared Utilities:
DotNetToolUtilitiesclass with shared dnx/TFM detection3. E2E Test Coverage:
Test Coverage Implemented
_EfcptIsSqlProjectBuild Status
✅ Solution builds successfully (Release configuration)
✅ All E2E tests compile successfully
✅ Testcontainers integration matches existing test patterns
✅ .NET Framework 4.7.2 compatibility maintained
See the database-first-sql-generation sample for complete example.
Original prompt
This section details on the original issue you should resolve
<issue_title>[Feature] Database-First SQL Project Generation (SqlProj / MSBuild.Sdk.SqlProj)</issue_title>
<issue_description>### Summary
Add a new reverse-engineering mode to JD.Efcpt.Build that can materialize a SQL Server database into a SQL project—either:
.sqlproj, or.csproj/.fsprojThis new mode should integrate seamlessly with the existing Efcpt lifecycle, run before EF Core model reverse engineering, and participate fully in our fingerprinting, snapshotting, and determinism guarantees.
The result is a first-class Database-First pipeline:
With automatic downstream regeneration whenever the database schema changes.
Problem Statement
JD.Efcpt.Build already provides strong, deterministic reverse-engineering into EF Core models.
However, many organizations treat SQL projects as the canonical schema artifact:
Currently, users must choose between:
…or maintain custom, brittle glue code between tools.
This feature closes that gap by making SQL project generation a first-class Efcpt capability.
Proposed Solution
Introduce a new Efcpt reverse-engineering mode that:
Supported Output Targets
The new mode should support generating:
Microsoft.Build.Sql
.sqlprojMSBuild.Sdk.SqlProj
.csproj.fsprojThis mirrors the real-world ecosystems used by SQL Server teams today.
Tooling & Dependencies
Implementation should leverage:
sqlpackageUsed to extract schema from SQL Server
Examples already exist in:
dotnet-sqlpackage/ DNXReference:
MSBuild Integration Model
The experience should be “automagic”, consistent with existing JD.Efcpt.Build flows.
Example Usage
A user should be able to:
…and have everything “just work”.
No custom targets. No manual wiring.
Configuration & Connectivity
Database connectivity should reuse existing JD.Efcpt.Build patterns, including:
This is already solved today for EF Core reverse engineering and should be reused.
Lifecycle & Ordering
The new SqlProj generation mode must integrate into the existing lifecycle:
Key points:
Fingerprinting & Determinism
The SqlProj generation must:
Participate in existing fingerprinting
Snapshot remote schema state
Skip regeneration when:
This ensures:
All required primitives already exist in JD.Efcpt.Build and should be extended—not reinvented.
New Hooks & Extensibility
This feature will require:
New lifecycle hooks for SQL project generation
Coordination between:
Extension of existing fingerprint providers to understand SQL artifacts
No breaking changes to existing flows should be required.
Outcome & Benefits
This enables a fully automated Database-First workflow:
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.