Skip to content

Refactor professor course and section pipelines into a pipeline builder function #296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ruba0s wants to merge 14 commits into
base: develop
Choose a base branch
from
Open

Refactor professor course and section pipelines into a pipeline builder function #296

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2c22c80
Refactor professor course and section pipelines into a pipeline build…
ruba0s Sep 30, 2025
acaac1d
Merge branch 'develop' into develop
ruba0s Oct 3, 2025
15c56f4
Merge branch 'develop' into develop
ruba0s Oct 3, 2025
c22c940
Update professor.go pipeline builder function
ruba0s Oct 10, 2025
3388ccb
Update professor.go pipeline builder function
ruba0s Oct 10, 2025
f59c0be
Merge branch 'develop' into develop
mikehquan19 Oct 11, 2025
9555b84
Update professor.go pipeline course pagination stages
ruba0s Oct 17, 2025
95d7de4
Merge branch 'develop' of https://github.com/ruba0s/nebula-api into d…
ruba0s Oct 17, 2025
a74fc7f
Update professor.go pipeline course pagination stages
ruba0s Oct 18, 2025
f860119
Merge branch 'develop' into develop
mikehquan19 Oct 18, 2025
8b48386
Merge branch 'develop' into develop
mikehquan19 Oct 18, 2025
6fa43e6
Merge branch 'develop' into develop
mikehquan19 Oct 23, 2025
f767ba3
Merge branch 'UTDNebula:develop' into develop
ruba0s Oct 30, 2025
53aa1e9
Merge branch 'develop' into develop
mikehquan19 Nov 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 78 additions & 78 deletions api/controllers/professor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,82 @@ func ProfessorCourseById() gin.HandlerFunc {
}
}

// Pipeline builder for professor aggregate endpoints
func professorPipeline(endpoint string, professorQuery bson.M, paginateMap map[string]int64) mongo.Pipeline {
// common stages
baseStages := mongo.Pipeline{
// filter the professors
bson.D{{Key: "$match", Value: professorQuery}},

// paginate the professors before pulling the courses from those professor
bson.D{{Key: "$skip", Value: paginateMap["former_offset"]}}, // skip to the specified offset
bson.D{{Key: "$limit", Value: paginateMap["limit"]}}, // limit to the specified number of professors

// lookup the array of sections from sections collection
bson.D{{Key: "$lookup", Value: bson.D{
{Key: "from", Value: "sections"},
{Key: "localField", Value: "sections"},
{Key: "foreignField", Value: "_id"},
{Key: "as", Value: "sections"},
}}},

// keep order deterministic between calls
bson.D{{Key: "$sort", Value: bson.D{{Key: "_id", Value: 1}}}},
Copy link
Contributor

@mikehquan19 mikehquan19 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think $sort could belong to the pagination stages since they are supposed to be sorted right before we paginate the second time

}

// course pagination stages
paginationStages := mongo.Pipeline{
bson.D{{Key: "$skip", Value: paginateMap["latter_offset"]}},
bson.D{{Key: "$limit", Value: paginateMap["limit"]}},
}

if endpoint == "courses" {
courseStages := mongo.Pipeline{
// project the courses referenced by each section in the array
bson.D{{Key: "$project", Value: bson.D{{Key: "courses", Value: "$sections.course_reference"}}}},

// lookup the array of courses from coures collection
bson.D{{Key: "$lookup", Value: bson.D{
{Key: "from", Value: "courses"},
{Key: "localField", Value: "courses"},
{Key: "foreignField", Value: "_id"},
{Key: "as", Value: "courses"},
}}},

// unwind the courses
bson.D{{Key: "$unwind", Value: bson.D{
{Key: "path", Value: "$courses"},
{Key: "preserveNullAndEmptyArrays", Value: false}, // to avoid the professor documents that can't be replaced
}}},

// replace the combination of ids and courses with the courses entirely
bson.D{{Key: "$replaceWith", Value: "$courses"}},
Copy link
Contributor

@mikehquan19 mikehquan19 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see here $unwind and $replaceWith are kind of repeated between 2 branches. Can you come up with the way to add them to pagination stages too and just replace the value with the endpoint parameters?

}

return append(append(baseStages, courseStages...), paginationStages...)
}

if endpoint == "sections" {
sectionStages := mongo.Pipeline{
// project the sections
bson.D{{Key: "$project", Value: bson.D{{Key: "sections", Value: "$sections"}}}},

// unwind the sections
bson.D{{Key: "$unwind", Value: bson.D{
{Key: "path", Value: "$sections"},
{Key: "preserveNullAndEmptyArrays", Value: false}, // to avoid the professor documents that can't be replaced
}}},

// replace the combination of ids and sections with the sections entirely
bson.D{{Key: "$replaceWith", Value: "$sections"}},
}

return append(append(baseStages, sectionStages...), paginationStages...)
}

return append(baseStages, paginationStages...) // fallback (shouldn't happen because we call with either courses or sections)
Copy link
Contributor

@mikehquan19 mikehquan19 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here we don't need a fallback, if the endpoint is invalid we will just panic :)

}

// Get all of the courses of the professors depending on the type of flag
func professorCourse(flag string, c *gin.Context) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
Expand All @@ -222,49 +298,7 @@ func professorCourse(flag string, c *gin.Context) {
}

// Pipeline to query the courses from the filtered professors (or a single professor)
professorCoursePipeline := mongo.Pipeline{
// filter the professors
bson.D{{Key: "$match", Value: professorQuery}},

// paginate the professors before pulling the courses from those professor
bson.D{{Key: "$skip", Value: paginateMap["former_offset"]}}, // skip to the specified offset
bson.D{{Key: "$limit", Value: paginateMap["limit"]}}, // limit to the specified number of professors

// lookup the array of sections from sections collection
bson.D{{Key: "$lookup", Value: bson.D{
{Key: "from", Value: "sections"},
{Key: "localField", Value: "sections"},
{Key: "foreignField", Value: "_id"},
{Key: "as", Value: "sections"},
}}},

// project the courses referenced by each section in the array
bson.D{{Key: "$project", Value: bson.D{{Key: "courses", Value: "$sections.course_reference"}}}},

// lookup the array of courses from coures collection
bson.D{{Key: "$lookup", Value: bson.D{
{Key: "from", Value: "courses"},
{Key: "localField", Value: "courses"},
{Key: "foreignField", Value: "_id"},
{Key: "as", Value: "courses"},
}}},

// unwind the courses
bson.D{{Key: "$unwind", Value: bson.D{
{Key: "path", Value: "$courses"},
{Key: "preserveNullAndEmptyArrays", Value: false}, // to avoid the professor documents that can't be replaced
}}},

// replace the combination of ids and courses with the courses entirely
bson.D{{Key: "$replaceWith", Value: "$courses"}},

// keep order deterministic between calls
bson.D{{Key: "$sort", Value: bson.D{{Key: "_id", Value: 1}}}},

// paginate the courses
bson.D{{Key: "$skip", Value: paginateMap["latter_offset"]}},
bson.D{{Key: "$limit", Value: paginateMap["limit"]}},
}
professorCoursePipeline := professorPipeline("courses", professorQuery, paginateMap)

// Perform aggreration on the pipeline
cursor, err := professorCollection.Aggregate(ctx, professorCoursePipeline)
Expand Down Expand Up @@ -354,41 +388,7 @@ func professorSection(flag string, c *gin.Context) {
}

// Pipeline to query the courses from the filtered professors (or a single professor)
professorSectionPipeline := mongo.Pipeline{
// filter the professors
bson.D{{Key: "$match", Value: professorQuery}},

// paginate the professors before pulling the courses from those professor
bson.D{{Key: "$skip", Value: paginateMap["former_offset"]}}, // skip to the specified offset
bson.D{{Key: "$limit", Value: paginateMap["limit"]}}, // limit to the specified number of professors

// lookup the array of sections from sections collection
bson.D{{Key: "$lookup", Value: bson.D{
{Key: "from", Value: "sections"},
{Key: "localField", Value: "sections"},
{Key: "foreignField", Value: "_id"},
{Key: "as", Value: "sections"},
}}},

// project the sections
bson.D{{Key: "$project", Value: bson.D{{Key: "sections", Value: "$sections"}}}},

// unwind the sections
bson.D{{Key: "$unwind", Value: bson.D{
{Key: "path", Value: "$sections"},
{Key: "preserveNullAndEmptyArrays", Value: false}, // to avoid the professor documents that can't be replaced
}}},

// replace the combination of ids and sections with the sections entirely
bson.D{{Key: "$replaceWith", Value: "$sections"}},

// keep order deterministic between calls
bson.D{{Key: "$sort", Value: bson.D{{Key: "_id", Value: 1}}}},

// paginate the sections
bson.D{{Key: "$skip", Value: paginateMap["latter_offset"]}},
bson.D{{Key: "$limit", Value: paginateMap["limit"]}},
}
professorSectionPipeline := professorPipeline("sections", professorQuery, paginateMap)

// Perform aggreration on the pipeline
cursor, err := professorCollection.Aggregate(ctx, professorSectionPipeline)
Expand Down
4 changes: 2 additions & 2 deletions api/controllers/section.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ func SectionById(c *gin.Context) {
// @Tags Sections
// @Description "Returns paginated list of courses of all the sections matching the query's string-typed key-value pairs. See former_offset and latter_offset for pagination details."
// @Produce json
// @Param former_offset query number false "The starting position of the current page of sections (e.g. For starting at the 16th section, former_offset=16)."
// @Param latter_offset query number false "The starting position of the current page of courses (e.g. For starting at the 16th course, latter_offset=16)."
// @Param former_offset query number false "The starting position of the current page of professors (e.g. For starting at the 17th professor, former_offset=16)."
// @Param latter_offset query number false "The starting position of the current page of sections (e.g. For starting at the 17th professor, offset=16)."
// @Param section_number query string false "The section's official number"
// @Param academic_session.name query string false "The name of the academic session of the section"
// @Param academic_session.start_date query string false "The date of classes starting for the section"
Expand Down
2 changes: 1 addition & 1 deletion api/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ require (
github.com/swaggo/files v1.0.1
github.com/swaggo/gin-swagger v1.6.1
github.com/swaggo/swag v1.16.6
go.mongodb.org/mongo-driver v1.17.3
go.mongodb.org/mongo-driver v1.17.4
google.golang.org/api v0.224.0
)

Expand Down
2 changes: 2 additions & 0 deletions api/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,8 @@ github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfS
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.mongodb.org/mongo-driver v1.17.3 h1:TQyXhnsWfWtgAhMtOgtYHMTkZIfBTpMTsMnd9ZBeHxQ=
go.mongodb.org/mongo-driver v1.17.3/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ=
go.mongodb.org/mongo-driver v1.17.4 h1:jUorfmVzljjr0FLzYQsGP8cgN/qzzxlY9Vh0C9KFXVw=
go.mongodb.org/mongo-driver v1.17.4/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/detectors/gcp v1.34.0 h1:JRxssobiPg23otYU5SbWtQC//snGVIM3Tx6QRzlQBao=
Expand Down