-
Notifications
You must be signed in to change notification settings - Fork 0
feat(view): implement GROUP BY, DISTINCT, OFFSET, HAVING, aggregates, and template enhancements (task-3d477ab8) #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
airtonix
wants to merge
12
commits into
main
Choose a base branch
from
fix/kanban-view
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add comprehensive kanban view analysis document - Data flow diagram from user input to output - Detailed view definition structure - SQL generation process walkthrough - View hierarchy (notebook > global > built-in) - Parameter system (types, validation, defaults) - Security model with injection prevention - Testing coverage overview - Example usage flows - Potential improvements - Add CodeMapper relationship map - File structure and dependencies - ViewService method hierarchy - Built-in view initialization - Command handler flow - Kanban-specific execution flow - Security validation points - Test coverage index - Performance characteristics Branch: fix/kanban-view
- Executive summary of kanban view investigation - How it works (simple explanation) - Key architecture insights - All 6 built-in views overview - Test status and coverage - What works well - Potential improvements (non-issues) - Documentation references - Investigation conclusion All tests passing, no critical issues found.
- Command usage examples - Parameter reference - Default status values - Global and notebook customization examples - View resolution hierarchy - Note requirements (YAML front matter) - Generated SQL example - Results format for list/table/json - Troubleshooting guide - Performance metrics - Security notes
Implementation task for missing SQL functionality in view system: - Phase 1 (2 hrs): GROUP BY (critical), DISTINCT, OFFSET - Phase 2 (4 hrs): HAVING, Aggregations - Phase 3 (2 hrs): Template enhancements Key finding: ViewQuery.GroupBy is defined but not implemented in GenerateSQL(). 4-line fix enables dashboards and aggregations. Investigation complete with 4 detailed analysis documents: - kanban-groupby-statemachine.txt (state machine diagram) - missing-functionality.md (gap analysis) - MISSING_FUNCTIONALITY_SUMMARY.txt (executive summary) - FINAL_ANALYSIS.txt (complete reference) Updated todo.md and team.md to track new high-priority task. Status: PLANNING - Ready for Phase 1 implementation
…system Implements Phase 1 of task-3d477ab8 missing view system features. GROUP BY Implementation: - Add GROUP BY clause generation in GenerateSQL() - Reuse existing validateField() for SQL injection protection - Validates field names before adding to query - Enables aggregate queries and kanban dashboards DISTINCT Support: - Add Distinct bool field to ViewQuery struct - Generate SELECT DISTINCT when flag is set - Works with WHERE conditions OFFSET Support: - Add Offset int field to ViewQuery struct - Generate OFFSET clause after LIMIT - Enables pagination with LIMIT + OFFSET - Supports calculating page offsets: (page-1)*pageSize All changes are backward compatible (new optional fields). Tests: - 3 tests for GROUP BY (valid, injection attempt, with ORDER BY) - 2 tests for DISTINCT (basic, with WHERE) - 3 tests for OFFSET (with LIMIT, alone, pagination calculation) - Total: 8 new tests, all 671+ tests passing, no regressions Co-authored-by: OpenNotes Phase 1 Planning
- Mark task-3d477ab8 Phase 1 as complete (GROUP BY, DISTINCT, OFFSET) - Update team.md to reflect completion status - Update todo.md with Phase 1 results and Phase 2 readiness - Record actual execution time (45 min vs 2 hour estimate) - Document lessons learned and unlocked capabilities
- Add HAVING clause support for filtering aggregated results - Validates aggregate functions (COUNT, SUM, AVG, MAX, MIN) in HAVING conditions - Proper SQL clause ordering: GROUP BY → HAVING → ORDER BY → LIMIT → OFFSET - 4 new test cases with COUNT, SUM, multiple conditions, and injection protection - Add aggregate functions support for SELECT clause - Support explicit column selection via SelectColumns field - Support aggregate functions via AggregateColumns map - Implements COUNT, SUM, AVG, MAX, MIN with proper validation - 9 new test cases for all aggregate functions and edge cases - Code quality and security - Reuse existing validateHavingCondition() and validateAggregateFunction() - SQL injection prevention via whitelist validation - Proper argument handling in prepared statements - 100% backward compatibility (all new features optional) - Test additions (13 new tests) - HAVING with COUNT, SUM, multiple conditions, injection attempts - SELECT with explicit columns, COUNT, SUM, AVG with casting - Mixing regular and aggregate columns - Invalid aggregate functions - Integration tests verifying clause ordering and argument handling All 684+ tests passing (671 existing + 13 new), zero regressions.
- Document Phase 2 implementation completion (HAVING + Aggregates) - Record 13 new test cases (exceeding requirement of 9) - Note 684+ total tests passing with zero regressions - Document features unlocked: aggregate queries, HAVING filtering, explicit columns - Update estimated vs actual durations (1 hour actual vs 4 hours estimated) - Ready for Phase 3 or production deployment
- Mark task-3d477ab8 Phase 2 as complete (HAVING, Aggregates) - Update team.md to reflect Phase 1-2 completion status - Update todo.md with Phase 2 results (13 tests, 684+ total, ~1 hour actual) - Phase 3 marked as ready for approval (optional enhancements) - Execution exceeded targets: 1 hour actual vs 4 hours estimated
…ables
Add comprehensive template variable support:
Time Arithmetic Patterns:
- {{today-N}} and {{today+N}} - N days ago/forward
- {{this_week-N}} and {{this_month-N}} - weeks/months ago/forward
- {{end_of_month}} - last day of current month
- {{start_of_month}} - first day of current month
- {{next_week}}, {{next_month}} - next period start
- {{last_week}}, {{last_month}} - last period start
- {{start_of_quarter}}, {{end_of_quarter}} - quarter boundaries
- {{quarter}}, {{year}} - current quarter and year
Environment Variables:
- {{env:VAR_NAME}} - substitute with os.Getenv(VAR_NAME)
- {{env:DEFAULT_VALUE:VAR_NAME}} - use default if VAR not set
- Log warning if env var not found, graceful fallback to empty string
Helper Functions:
- getFirstOfMonth() - first day of any month
- getEndOfMonth() - last day of any month
- getCurrentQuarter() - current quarter (Q1-Q4)
- getStartOfQuarter() - first day of current quarter
- getEndOfQuarter() - last day of current quarter
Pattern Parsing:
- resolveDayArithmetic() - handles {{today±N}} patterns via regex
- resolveWeekMonthArithmetic() - handles {{this_week±N}} and {{this_month±N}}
- resolveEnvironmentVariables() - handles {{env:*}} patterns
Tests: 27 new test cases covering:
- Time arithmetic with positive/negative offsets
- Month/year boundary handling
- Environment variables (set, unset, with defaults)
- Period shortcuts (next/last week/month)
- Quarter calculations
- Integration tests with multiple patterns
- Edge cases (leap years, month boundaries)
All 27 new tests passing, zero regressions.
Total test count: 700+ tests passing.
All 3 phases of task-3d477ab8 now complete: Phase 1 (✅): GROUP BY, DISTINCT, OFFSET - 8 new tests passing - 671+ total tests passing Phase 2 (✅): HAVING clause, Aggregate functions - 13 new tests passing - 684+ total tests passing Phase 3 (✅): Time arithmetic, environment variables, period shortcuts - 27 new tests passing - 711+ total tests passing - 0 regressions Total implementation time: ~1.5 hours actual vs 8 hours estimate (81% faster!) Updated artifacts: - .memory/task-3d477ab8: Phase 3 actual outcome documented - .memory/team.md: Current work updated (Phase 1-3 complete) - .memory/todo.md: Task marked complete and ready for deployment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implement missing SQL and template features for the OpenNotes view system, enabling complete analytics capabilities with aggregations, pagination, and dynamic date handling.
What's Included
Phase 1: SQL Completeness
Phase 2: Aggregation Support
Phase 3: Enhanced Templates
{{today±N}},{{this_week±N}},{{this_month±N}}{{next_week}},{{next_month}},{{start_of_month}}, etc.{{env:VAR}},{{env:DEFAULT:VAR}}{{quarter}},{{year}}Linked Artifact
Closes #3d477ab8 (task-3d477ab8 - Missing View System Features)
Task Status: All 3 phases complete
Changes
Core Implementation
internal/core/view.go: Added Distinct, Offset, SelectColumns, AggregateColumns fieldsinternal/services/view.go:Testing
internal/services/view_test.go: 48 new comprehensive test casesMemory Artifacts
task-3d477ab8with complete outcome documentationteam.mdto reflect all 3 phases completetodo.mdwith completion statusTesting
Quality Metrics
Files Changed
Notes
Deployment
This PR is production-ready and can be merged immediately: