[management] Legacy to embedded IdP migration tool#5586
Conversation
…ation tool Move ListUsers/UpdateUserID out of store.Store and activity.Store into migration-specific interfaces (MigrationStore, MigrationEventStore) so migration code can be cleanly removed later. Add tools/idp-migrate CLI that migrates user IDs and generates EmbeddedIdP management.json config
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds an IdP migration feature: Dex connector mapping and JWKS support, migration library and interfaces, SQL store migration implementations and tests, a CLI tool (tools/idp-migrate) with goreleaser target, and wiring to use embedded Dex keys for JWT validation and auth manager key fetching. Changes
Sequence DiagramsequenceDiagram
participant CLI as IDP Migrate CLI
participant MainStore as Main Store
participant EventStore as Activity/Event Store
participant IdP as External IdP Manager
participant Dex as Dex Connector/Provider
CLI->>MainStore: CheckSchema()
MainStore-->>CLI: schema OK
CLI->>MainStore: ListUsers()
MainStore-->>CLI: users[]
Note over CLI,IdP: Optional: populate missing user info
CLI->>IdP: Fetch user metadata
IdP-->>CLI: email/name
CLI->>MainStore: UpdateUserInfo(user)
CLI->>Dex: DecodeConnector(seed)
Dex-->>CLI: connector
loop per user to migrate
CLI->>MainStore: UpdateUserID(old→new)
MainStore->>MainStore: transactional updates (user, related tables)
MainStore-->>CLI: OK
CLI->>EventStore: UpdateUserID(old→new)
EventStore->>EventStore: update events / deleted_users
EventStore-->>CLI: OK
end
CLI->>MainStore: GenerateConfig() (optional)
MainStore-->>CLI: new management.json (or backup)
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…complexity, fix code smells
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (3)
tools/idp-migrate/DEVELOPMENT.md (1)
12-31: Add language specifier to fenced code blocks for markdownlint compliance.The directory structure code blocks at lines 12 and 37 lack language specifiers, triggering MD040 warnings. Consider adding
textorplaintextas the language.📝 Suggested fix
-``` +```text tools/idp-migrate/ ├── main.go # CLI entry point, connector resolution, config generationAnd similarly for line 37:
-``` +```text netbird-idp-migrate_<version>_linux_<arch>.tar.gz🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/DEVELOPMENT.md` around lines 12 - 31, The fenced directory/listing code blocks in DEVELOPMENT.md trigger markdownlint MD040 because they lack a language specifier; update the two blocks shown (the tools/idp-migrate/ tree block and the tarball name block) to use a plain text specifier (e.g., change the opening "```" to "```text") so markdownlint recognizes them as plaintext listings and the MD040 warnings are resolved.management/server/activity/store/sql_store_idp_migration_test.go (1)
17-26: Consider handling GenerateKey error in test helper.Line 19 ignores the error from
crypt.GenerateKey(). While unlikely to fail, handling it would make test failures more diagnosable.📝 Suggested fix
newStore := func(t *testing.T) *Store { t.Helper() - key, _ := crypt.GenerateKey() + key, err := crypt.GenerateKey() + if err != nil { + t.Fatal(err) + } s, err := NewSqlStore(ctx, t.TempDir(), key)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/activity/store/sql_store_idp_migration_test.go` around lines 17 - 26, The test helper newStore currently ignores the error returned by crypt.GenerateKey(); change it to capture the error (key, err := crypt.GenerateKey()) and fail the test immediately on error (e.g., t.Fatalf or t.Fatal with the error) before passing key into NewSqlStore. Update the newStore function so it checks err after GenerateKey and calls t.Fatalf("crypt.GenerateKey: %v", err) (or similar) to make failures diagnosable while keeping the existing t.Helper and t.Cleanup usage.tools/idp-migrate/MIGRATION_GUIDE.md (1)
233-242: Add language specifier to output code blocks for consistency.The code blocks showing expected output (lines 233, 266, 294, 381) lack language specifiers. Consider adding
textorconsolefor consistency with markdownlint MD040.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/MIGRATION_GUIDE.md` around lines 233 - 242, Update the fenced code blocks that show expected CLI output in the migration guide so they include a language specifier (e.g., ```text or ```console) instead of bare triple backticks; specifically modify each example output block (the ones that begin with lines like "INFO resolved connector: type=oidc, id=auth0, name=auth0" and the subsequent "[DRY RUN]" summaries) to start with ```text (or ```console) and end with ``` to satisfy markdownlint MD040 and ensure consistent rendering.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.goreleaser.yaml:
- Around line 173-174: The ldflags list references -X main.commit, -X main.date
and -X main.builtBy which are not declared; either remove those three -X entries
from ldflags or add matching package-level variable declarations in the
idp-migrate main package (e.g., declare commit, date, builtBy or follow the
existing pattern Version/Commit/BuildDate) so the linker substitutions have
corresponding symbols (look for the ldflags line and the tools/idp-migrate main
package entry points such as main.go to add the vars).
In `@management/server/idp/migration/migration.go`:
- Around line 164-169: The current flattening of allAccounts into idpUsers
(map[string]*idp.UserData) loses account scoping so duplicate external IDs
across accounts can overwrite each other; change idpUsers to be account-scoped
(e.g., map[string]map[string]*idp.UserData or similar keyed by accountID then
userID) and update the subsequent logic that uses idpUsers (the code around the
current lines 193-198) to look up users by accountID then userID instead of a
global lookup; also add a regression test that constructs two accounts
containing the same external ID and verifies each account's user retains its own
email/name after migration.
- Around line 214-225: The current migration logs plaintext PII (email and name)
in both dry-run and live paths; update the log calls around the dryRun branch
and after s.Store().UpdateUserInfo to avoid printing email/name and instead log
only the user identifier and which fields changed (e.g., "email" and/or "name")
and whether it was a dry run. Specifically, change the log.Infof calls that
reference user.Id, email, name to something like a dry-run message and a
live-update message that include user.Id and a concise list of changed fields
(not their values), keep updatedCount increments and the
s.Store().UpdateUserInfo call and error handling for user.Id as-is.
In `@tools/idp-migrate/main_test.go`:
- Around line 345-368: The test fixture uses a string that matches
secret-scanner patterns ("test-key-1234567890123456"); update the
DataStoreEncryptionKey value inside the configJSON fixture to a clearly
fake/non-secret placeholder (e.g. "example-encryption-key-0000") and update the
corresponding assertion that checks cfg.DataStoreEncryptionKey to expect the new
placeholder; locate the configJSON variable in the test and the assertions that
read cfg.DataStoreEncryptionKey (after loadConfig) to make these paired changes.
In `@tools/idp-migrate/main.go`:
- Around line 140-149: validateSchema currently only checks the main store
schema (migStore.CheckSchema) and misses the activity/event store; add a
readiness check for the activity store by calling the event store's schema
validator (e.g., via migStore.EventStore().CheckSchema(...) using the same
migration contract that EventStore().UpdateUserID expects) and if it returns
errors, return an error immediately (same style as the existing migStore check)
so the preflight fails early when an incompatible activity DB is present.
- Around line 335-339: Remove the debug print that can leak secrets: delete the
fmt.Println(conn, err) call after migration.SeedConnectorFromEnv() so the
decoded connector (including client secret) is not written to stdout; if you
need diagnostics, log only non-sensitive status or errors (keep the existing
error check that uses migration.ErrNoSeedInfo and the fmt.Errorf wrap in that
block).
In `@tools/idp-migrate/MIGRATION_GUIDE.md`:
- Line 22: Replace every occurrence of the literal placeholder
"<INSERT_VERSION>" in this document (notably the table entry on line 22 and the
other occurrence around line 458) with the actual release version string (e.g.,
"v0.60.0"); ensure both instances of "<INSERT_VERSION>" are updated consistently
so the table header and any other references show the real version before
publishing.
---
Nitpick comments:
In `@management/server/activity/store/sql_store_idp_migration_test.go`:
- Around line 17-26: The test helper newStore currently ignores the error
returned by crypt.GenerateKey(); change it to capture the error (key, err :=
crypt.GenerateKey()) and fail the test immediately on error (e.g., t.Fatalf or
t.Fatal with the error) before passing key into NewSqlStore. Update the newStore
function so it checks err after GenerateKey and calls
t.Fatalf("crypt.GenerateKey: %v", err) (or similar) to make failures diagnosable
while keeping the existing t.Helper and t.Cleanup usage.
In `@tools/idp-migrate/DEVELOPMENT.md`:
- Around line 12-31: The fenced directory/listing code blocks in DEVELOPMENT.md
trigger markdownlint MD040 because they lack a language specifier; update the
two blocks shown (the tools/idp-migrate/ tree block and the tarball name block)
to use a plain text specifier (e.g., change the opening "```" to "```text") so
markdownlint recognizes them as plaintext listings and the MD040 warnings are
resolved.
In `@tools/idp-migrate/MIGRATION_GUIDE.md`:
- Around line 233-242: Update the fenced code blocks that show expected CLI
output in the migration guide so they include a language specifier (e.g.,
```text or ```console) instead of bare triple backticks; specifically modify
each example output block (the ones that begin with lines like "INFO resolved
connector: type=oidc, id=auth0, name=auth0" and the subsequent "[DRY RUN]"
summaries) to start with ```text (or ```console) and end with ``` to satisfy
markdownlint MD040 and ensure consistent rendering.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: bf9329df-ea4c-400e-ab52-e4bba7033a25
📒 Files selected for processing (15)
.goreleaser.yamlidp/dex/config.goidp/dex/provider_test.gomanagement/internals/server/modules.gomanagement/server/activity/store/sql_store_idp_migration.gomanagement/server/activity/store/sql_store_idp_migration_test.gomanagement/server/idp/embedded.gomanagement/server/idp/migration/migration.gomanagement/server/idp/migration/migration_test.gomanagement/server/idp/migration/store.gomanagement/server/store/sql_store_idp_migration.gotools/idp-migrate/DEVELOPMENT.mdtools/idp-migrate/MIGRATION_GUIDE.mdtools/idp-migrate/main.gotools/idp-migrate/main_test.go
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tools/idp-migrate/main.go (1)
306-313: Consider usingfmt.Printfor the interactive prompt.
log.Infofmay include timestamps and log-level prefixes depending on the logger configuration, which can make the interactive prompt less clear. For user-facing prompts that expect input,fmt.Printprovides cleaner output.Suggested fix
func confirmPrompt(pending int) bool { - log.Infof("About to migrate %d users. This cannot be easily undone. Continue? [y/N] ", pending) + fmt.Printf("About to migrate %d users. This cannot be easily undone. Continue? [y/N] ", pending) reader := bufio.NewReader(os.Stdin) answer, _ := reader.ReadString('\n') answer = strings.TrimSpace(strings.ToLower(answer)) return answer == "y" || answer == "yes" }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/main.go` around lines 306 - 313, The interactive prompt in function confirmPrompt should use fmt.Print or fmt.Printf instead of log.Infof so the prompt is shown without timestamps or log prefixes; update confirmPrompt to write the prompt with fmt.Printf("About to migrate %d users. This cannot be easily undone. Continue? [y/N] ", pending), keep the same bufio.NewReader/os.Stdin read/trim logic, and add the fmt import if missing.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tools/idp-migrate/main.go`:
- Around line 626-631: The current fallback reads
cfg.IdpManagerConfig.ClientConfig.Issuer and returns its host via hostFromURL,
which is the external IdP domain not the management server domain; update the
block in main.go (the conditional referencing cfg.IdpManagerConfig,
ClientConfig.Issuer and hostFromURL) to stop using Issuer as a fallback and
instead return a clear error prompting the operator to set
HttpConfig.LetsEncryptDomain (or remove the fallback entirely). Concretely,
replace the branch that returns
hostFromURL(cfg.IdpManagerConfig.ClientConfig.Issuer) with code that returns an
error message instructing the user to explicitly provide
HttpConfig.LetsEncryptDomain for configuring EmbeddedIdP.Issuer (referencing
EmbeddedIdP.Issuer and HttpConfig.LetsEncryptDomain to locate where the value is
needed).
---
Nitpick comments:
In `@tools/idp-migrate/main.go`:
- Around line 306-313: The interactive prompt in function confirmPrompt should
use fmt.Print or fmt.Printf instead of log.Infof so the prompt is shown without
timestamps or log prefixes; update confirmPrompt to write the prompt with
fmt.Printf("About to migrate %d users. This cannot be easily undone. Continue?
[y/N] ", pending), keep the same bufio.NewReader/os.Stdin read/trim logic, and
add the fmt import if missing.
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (3)
tools/idp-migrate/main.go (1)
593-600:⚠️ Potential issue | 🟠 MajorDo not derive management domain from external IdP issuer.
Falling back to
IdpManagerConfig.ClientConfig.Issuercan produce an embedded issuer under the external IdP domain, which is incorrect for combined-server routing.✂️ Suggested fix
- // Priority 4: parse from IdpManagerConfig.ClientConfig.Issuer - if cfg.IdpManagerConfig != nil && cfg.IdpManagerConfig.ClientConfig != nil && cfg.IdpManagerConfig.ClientConfig.Issuer != "" { - if host := hostFromURL(cfg.IdpManagerConfig.ClientConfig.Issuer); host != "" { - return host, nil - } - } - return "", fmt.Errorf("could not determine domain: set HttpConfig.LetsEncryptDomain, HttpConfig.AuthIssuer, or HttpConfig.OIDCConfigEndpoint in management.json")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/main.go` around lines 593 - 600, The code currently falls back to parsing cfg.IdpManagerConfig.ClientConfig.Issuer via hostFromURL when determining the management domain; remove this fallback so the domain is NOT derived from an external IdP issuer. Specifically, eliminate or skip the block that checks cfg.IdpManagerConfig != nil && cfg.IdpManagerConfig.ClientConfig != nil && cfg.IdpManagerConfig.ClientConfig.Issuer and any call to hostFromURL on that Issuer so the function only uses HttpConfig.LetsEncryptDomain, HttpConfig.AuthIssuer, or HttpConfig.OIDCConfigEndpoint to determine the domain and returns the existing error if none are set.management/server/idp/migration/migration.go (1)
214-225:⚠️ Potential issue | 🟠 MajorAvoid logging plaintext email/name during migration.
These logs emit profile PII in both dry-run and live paths; log changed fields only.
🔒 Safer logging pattern
+ changedEmail := email != user.Email + changedName := name != user.Name if dryRun { - log.Infof("[DRY RUN] would update user %s: email=%q, name=%q", user.Id, email, name) + log.Infof("[DRY RUN] would update user %s (email_changed=%t, name_changed=%t)", user.Id, changedEmail, changedName) updatedCount++ continue } @@ - log.Infof("updated user %s: email=%q, name=%q", user.Id, email, name) + log.Infof("updated user %s (email_changed=%t, name_changed=%t)", user.Id, changedEmail, changedName)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/idp/migration/migration.go` around lines 214 - 225, The logs in the migration loop (see variables/functions dryRun, user.Id, email, name and the log.Infof calls) currently print plaintext PII; change both the dry-run and live log statements to avoid emitting email/name values — instead log user.Id and which fields would/was changed (e.g. "email changed", "name changed") or log masked values (e.g. redacted or hashed) or simply "updated fields: [email,name]". Keep the UpdateUserInfo call and updatedCount logic as-is, only replace the log.Infof format strings to omit raw PII.tools/idp-migrate/MIGRATION_GUIDE.md (1)
22-22:⚠️ Potential issue | 🟠 MajorReplace unresolved release-version placeholders.
<INSERT_VERSION>is still present in user-facing instructions, which makes the migration prerequisites and troubleshooting guidance ambiguous at publish time.Also applies to: 458-458
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/MIGRATION_GUIDE.md` at line 22, Replace the unresolved placeholder "<INSERT_VERSION>" in MIGRATION_GUIDE.md with the actual release version (or a documented template variable that the docs build will substitute) so user-facing instructions are unambiguous; search for all occurrences of "<INSERT_VERSION>" (including the table cell under "NetBird version" and the duplicate at line ~458) and update them to the concrete version string (e.g., vX.Y.Z) or a validated variable like "{{RELEASE_VERSION}}" and ensure any build/process that generates the published docs will replace that variable.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@management/server/activity/store/sql_store_idp_migration.go`:
- Around line 39-60: The transaction currently uses store.db.Transaction without
the provided ctx, so cancellation/deadline signals aren't propagated; change the
call to store.db.WithContext(ctx).Transaction(...) so the context is applied to
the entire transaction and all DB calls (the inner tx stays the same; e.g.
tx.Model(&activity.Event{})... and tx.Exec(...)) ensuring the transaction
respects ctx cancellation/deadlines.
In `@management/server/idp/migration/migration.go`:
- Around line 62-63: The function MigrateUsersToStaticConnectors currently
dereferences conn.ID and will panic if conn is nil; add a nil-check at the start
of MigrateUsersToStaticConnectors (and the other occurrence that encodes
connector IDs) to return a clear error when conn == nil before attempting to
access conn.ID or calling any encode/ID helper, e.g., validate the connector
argument and fail fast with an informative error rather than allowing a
nil-pointer panic.
In `@tools/idp-migrate/DEVELOPMENT.md`:
- Line 12: Update the triple-backtick code fences in DEVELOPMENT.md to include
explicit fence languages (e.g., ```text or ```bash) so the Markdown linter MD040
is satisfied; specifically edit the unnamed code blocks shown around the current
snippet (and the other block referenced at line 37) to add the appropriate
language token for each fence.
In `@tools/idp-migrate/main.go`:
- Around line 494-543: generateConfig currently only injects EmbeddedIdP and
never mutates the existing HttpConfig/PKCE settings as promised; update the
function so after unmarshalling configMap you locate or create the "HttpConfig"
object in configMap and set its management-related fields (e.g., issuer/base
URLs, callback paths, DashboardRedirectURIs) to use the derived domain, ensure
PKCE is enabled/set appropriately, and update any client settings or auth
endpoint URLs that still point to the legacy external IdP to point at the new
management server; reference the existing conn.Config→connConfig, the derived
domain variable, and the EmbeddedIdP insert so you reuse the same URL formatting
and connector redirectURI when modifying HttpConfig and client PKCE fields.
- Around line 549-552: The dry-run branch currently logs full newJSON (which
contains sensitive fields like clientSecret) causing secret leakage; update the
dry-run behavior to sanitize before printing by parsing newJSON, removing or
replacing sensitive keys (e.g., "clientSecret", "client_secret", any credential
fields) with a redaction token, then marshal and log the sanitized JSON instead
of raw newJSON; ensure this change touches the dryRun branch where newJSON is
used so only the sanitized output is printed during dry runs.
In `@tools/idp-migrate/MIGRATION_GUIDE.md`:
- Line 233: Several fenced code blocks use plain triple backticks (```) without
a language identifier, triggering markdownlint rule MD040; locate the fenced
code blocks (the ``` markers) and add appropriate language identifiers (e.g.,
```bash, ```json, ```yaml, etc.) for each block so markdownlint recognizes the
language—update every occurrence of ``` in the document that currently lacks a
language tag (including the blocks noted in the review) to include the correct
language specifier.
- Around line 385-392: The command reference table in MIGRATION_GUIDE.md omits
the --skip-populate-user-info flag; add an entry in the options list (near the
other flags shown like --skip-config and --log-level) documenting the flag name
and a short description such as "Skip populating user info from the IdP during
migration (useful for restricted/air-gapped environments)"; ensure formatting
matches the surrounding --flag string Description style and include any
default behavior or note that it is a boolean/dry-run compatible flag.
---
Duplicate comments:
In `@management/server/idp/migration/migration.go`:
- Around line 214-225: The logs in the migration loop (see variables/functions
dryRun, user.Id, email, name and the log.Infof calls) currently print plaintext
PII; change both the dry-run and live log statements to avoid emitting
email/name values — instead log user.Id and which fields would/was changed (e.g.
"email changed", "name changed") or log masked values (e.g. redacted or hashed)
or simply "updated fields: [email,name]". Keep the UpdateUserInfo call and
updatedCount logic as-is, only replace the log.Infof format strings to omit raw
PII.
In `@tools/idp-migrate/main.go`:
- Around line 593-600: The code currently falls back to parsing
cfg.IdpManagerConfig.ClientConfig.Issuer via hostFromURL when determining the
management domain; remove this fallback so the domain is NOT derived from an
external IdP issuer. Specifically, eliminate or skip the block that checks
cfg.IdpManagerConfig != nil && cfg.IdpManagerConfig.ClientConfig != nil &&
cfg.IdpManagerConfig.ClientConfig.Issuer and any call to hostFromURL on that
Issuer so the function only uses HttpConfig.LetsEncryptDomain,
HttpConfig.AuthIssuer, or HttpConfig.OIDCConfigEndpoint to determine the domain
and returns the existing error if none are set.
In `@tools/idp-migrate/MIGRATION_GUIDE.md`:
- Line 22: Replace the unresolved placeholder "<INSERT_VERSION>" in
MIGRATION_GUIDE.md with the actual release version (or a documented template
variable that the docs build will substitute) so user-facing instructions are
unambiguous; search for all occurrences of "<INSERT_VERSION>" (including the
table cell under "NetBird version" and the duplicate at line ~458) and update
them to the concrete version string (e.g., vX.Y.Z) or a validated variable like
"{{RELEASE_VERSION}}" and ensure any build/process that generates the published
docs will replace that variable.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 092d0151-b07c-4f17-8105-25f661dd737c
📒 Files selected for processing (7)
management/server/activity/store/sql_store_idp_migration.gomanagement/server/idp/migration/migration.gomanagement/server/idp/migration/store.gotools/idp-migrate/DEVELOPMENT.mdtools/idp-migrate/MIGRATION_GUIDE.mdtools/idp-migrate/main.gotools/idp-migrate/main_test.go
There was a problem hiding this comment.
Actionable comments posted: 6
♻️ Duplicate comments (2)
tools/idp-migrate/DEVELOPMENT.md (1)
12-12:⚠️ Potential issue | 🟡 MinorAdd fence languages to the plain-text blocks.
The unlabeled fences at Line 12 and Line 37 still trigger MD040.
Minimal fix
-``` +```text tools/idp-migrate/ ├── config.go # migrationConfig struct, CLI flags, env vars, validation ├── main.go # CLI entry point, migration phases, config generation ├── main_test.go # 8 test functions (18 subtests) covering config, connector, URL builder, config generation └── DEVELOPMENT.md # this file @@ -``` +``` @@ -``` +```text netbird-idp-migrate_<version>_linux_<arch>.tar.gz -``` +```Also applies to: 37-37
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tools/idp-migrate/DEVELOPMENT.md` at line 12, In tools/idp-migrate/DEVELOPMENT.md the unlabeled triple-backtick fences (the plain-code blocks around the directory listing and the archive filename) are triggering MD040; add a fence language identifier (use "text") to both code fences so they read ```text instead of ```, updating the two occurrences that surround the directory tree and the netbird-idp-migrate_<version>_linux_<arch>.tar.gz line.tools/idp-migrate/main.go (1)
371-373:⚠️ Potential issue | 🟠 MajorRedact connector secrets from dry-run output.
newJSONincludes the static connector config, so this still prints credentials such asclientSecretto stdout/logs during--dry-run.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tools/idp-migrate/config.go`:
- Around line 109-123: The code currently treats any non-"true" NETBIRD_* env
value as false; instead validate and reject invalid boolean strings to avoid
silent coercion—use strconv.ParseBool (or explicitly allow only "true"/"false")
when reading NETBIRD_DRY_RUN, NETBIRD_FORCE, NETBIRD_SKIP_CONFIG, and
NETBIRD_SKIP_POPULATE_USER_INFO, and if parsing fails log a clear error and exit
(or return an error) rather than setting cfg.dryRun, cfg.force, cfg.skipConfig,
or cfg.skipPopulateUserInfo to false silently.
In `@tools/idp-migrate/main.go`:
- Around line 201-208: The code currently silences all errors from
activitystore.NewSqlStore by logging a warning; change this so only a "file not
found" error is downgraded to a warning and all other errors fail fast.
Specifically, in the block using activitystore.NewSqlStore(ctx, dataDir,
cfg.DataStoreEncryptionKey), inspect the returned err (e.g., with os.IsNotExist
or the store's sentinel error) and if it indicates the events DB is missing keep
the current warning behavior, otherwise return or log a fatal error and abort so
migrations don't proceed with a corrupted/unreadable activity store; still set
migEventStore and wrap cleanup only when actStore was successfully opened.
- Around line 223-230: The early return when pending == 0 prevents the
reconciliation pass; instead, after calling previewUsers(ctx, migStore) and
detecting pending == 0, do not return—log that there are no users to migrate but
still proceed to call MigrateUsersToStaticConnectors(ctx, migStore, ...) (or the
existing reconciliation routine) so activity-store references for
already-migrated users are repaired; update the log message to reflect "no users
to migrate — running reconciliation" and ensure MigrateUsersToStaticConnectors
is invoked even when pending == 0.
- Around line 399-402: When url.JoinPath(uri, path) fails the code currently
returns an empty string which leads to invalid issuer/redirect URIs and a broken
management.json; change the error handling so the failure is propagated instead
of returning "". Update the surrounding function (generateConfig or whichever
function contains the url.JoinPath call) to return (string, error) or return the
error to its caller, replace the bare return "" with returning the error (or
wrap it with context), and adjust all call sites to handle/propagate that error
(or fail fast in main) so malformed --api-domain / --dashboard-domain values
cause the program to exit with a clear error.
- Around line 77-80: The code unconditionally calls
decodeConnectorConfig(cfg.idpSeedInfo), making the seed mandatory and preventing
fallback to management.json; change the flow so if cfg.idpSeedInfo is empty (or
decodeConnectorConfig returns a specific "not provided" error) you read and
parse management.json to derive the connector instead. Concretely: update the
block that calls decodeConnectorConfig to first check cfg.idpSeedInfo != "" and
call decodeConnectorConfig only then; otherwise implement a new helper (or call
existing parsing logic) to load management.json and derive the connector, and
adjust validateConfig() to accept an absent seed when a connector is
successfully derived from management.json. Ensure you reference
decodeConnectorConfig, cfg.idpSeedInfo, validateConfig and the management.json
parsing/derivation helper when making the change.
- Around line 326-337: The code unguardedly asserts
configMap["HttpConfig"].(map[string]interface{}) which can panic if the key is
missing, null, or not an object; change it to first fetch the raw value (e.g.,
val, ok := configMap["HttpConfig"]), return a config error if !ok or val == nil,
then perform a safe type assertion (httpConfig, ok :=
val.(map[string]interface{})) and return a config error if !ok; once you have
the validated httpConfig, safely read "CertFile" and "CertKey" from it into
certFilePath/certKeyPath (handling missing keys), then proceed with
delete(configMap, "HttpConfig") and reassign configMap["HttpConfig"] =
map[string]interface{}{ "CertFile": certFilePath, "CertKey": certKeyPath } as
before.
---
Duplicate comments:
In `@tools/idp-migrate/DEVELOPMENT.md`:
- Line 12: In tools/idp-migrate/DEVELOPMENT.md the unlabeled triple-backtick
fences (the plain-code blocks around the directory listing and the archive
filename) are triggering MD040; add a fence language identifier (use "text") to
both code fences so they read ```text instead of ```, updating the two
occurrences that surround the directory tree and the
netbird-idp-migrate_<version>_linux_<arch>.tar.gz line.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c4cf776c-554a-4f83-9deb-638f71d8f5ee
📒 Files selected for processing (4)
tools/idp-migrate/DEVELOPMENT.mdtools/idp-migrate/config.gotools/idp-migrate/main.gotools/idp-migrate/main_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- tools/idp-migrate/main_test.go
Entire-Checkpoint: 5eaefec1fa77
…o/netbird into combined-migration-2
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
management/server/auth/manager_test.go (1)
55-55: Cover the non-nilkeyFetcherpath in this suite.All four updates still pass
nil, so the new embedded-IdP validator branch and its startup-failure behavior remain untested here.Also applies to: 95-95, 145-145, 228-228
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/auth/manager_test.go` at line 55, Test suite never exercises the non-nil keyFetcher branch because all calls to auth.NewManager pass nil for the keyFetcher; add tests that construct NewManager with a non-nil keyFetcher implementation to cover the embedded-IdP validator startup path. Implement a small test double (e.g., fakeKeyFetcher implementing the expected Fetch or FetchKeys method) that returns valid keys and use it when creating manager := auth.NewManager(store, "", "", "", "", []string{}, false, fakeKeyFetcher) to assert normal startup, and add another test double that returns an error to assert the startup-failure behavior; update the tests that currently call NewManager with nil (the occurrences around manager := auth.NewManager(...)) to include these non-nil keyFetcher variants.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@management/internals/server/controllers.go`:
- Around line 79-87: The code currently picks audiences[0] from
oauthProvider.GetClientIDs() and then builds the ClaimsExtractor to look only
for claims namespaced to that single audience; change this so ClaimsExtractor
checks all valid audiences instead of using a hardcoded first element: remove
the audiences[0] usage when creating the extractor in auth.NewManager and
implement the extractor logic to iterate over the audiences slice returned by
oauthProvider.GetClientIDs(), constructing each candidate claim name (e.g.,
"<aud>_wt_account_id") and returning the first match; keep the fallback
keyFetcher/local keys logic intact (oauthProvider.GetKeyFetcher(),
GetLocalKeysLocation()) but ensure claim extraction no longer assumes a single
audience.
In `@shared/auth/jwt/validator.go`:
- Around line 95-109: NewValidatorWithKeyFetcher currently assigns whatever the
keyFetcher returns, which can leave Validator.keys nil and later cause panics at
refreshedKeys.ExpiresInTime.UTC() or v.keys.stillValid(); change the constructor
to ensure Validator.keys is always non-nil by replacing nil returns with a
default &Jwks{} when err != nil or keys == nil, log the error as before, and
return the Validator with keys set to the empty &Jwks{}; apply the same
defensive change to the other similar constructors in this file (the ones around
the other noted ranges) so all paths guarantee Validator.keys is non-nil.
---
Nitpick comments:
In `@management/server/auth/manager_test.go`:
- Line 55: Test suite never exercises the non-nil keyFetcher branch because all
calls to auth.NewManager pass nil for the keyFetcher; add tests that construct
NewManager with a non-nil keyFetcher implementation to cover the embedded-IdP
validator startup path. Implement a small test double (e.g., fakeKeyFetcher
implementing the expected Fetch or FetchKeys method) that returns valid keys and
use it when creating manager := auth.NewManager(store, "", "", "", "",
[]string{}, false, fakeKeyFetcher) to assert normal startup, and add another
test double that returns an error to assert the startup-failure behavior; update
the tests that currently call NewManager with nil (the occurrences around
manager := auth.NewManager(...)) to include these non-nil keyFetcher variants.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b58b65b4-1b61-4205-affd-5178ed921657
📒 Files selected for processing (10)
.goreleaser.yamlgo.modidp/dex/provider.gomanagement/internals/server/controllers.gomanagement/internals/server/modules.gomanagement/server/auth/manager.gomanagement/server/auth/manager_test.gomanagement/server/http/testing/testing_tools/channel/channel.gomanagement/server/idp/embedded.goshared/auth/jwt/validator.go
✅ Files skipped from review due to trivial changes (1)
- management/internals/server/modules.go
🚧 Files skipped from review as they are similar to previous changes (2)
- management/server/idp/embedded.go
- .goreleaser.yaml
fd3c782 to
db41f4d
Compare
Entire-Checkpoint: cfd28dfcf51a
…o/netbird into combined-migration-2
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
shared/auth/jwt/validator.go (1)
164-176:⚠️ Potential issue | 🟡 MinorAdd nil guard after
fetchKeysto prevent potential panic.While the constructor now handles nil keys, the refresh methods (
refreshKeysandforceRefreshKeys) assignrefreshedKeysdirectly after error checking. IffetchKeysreturns(nil, nil)(e.g., a buggy fetcher implementation), Line 174 and Line 198 will panic onrefreshedKeys.ExpiresInTime.UTC().🛡️ Proposed defensive fix
func (v *Validator) refreshKeys(ctx context.Context) { v.lock.Lock() defer v.lock.Unlock() refreshedKeys, err := v.fetchKeys(ctx) if err != nil { log.WithContext(ctx).Debugf("cannot get JSONWebKey: %v, falling back to old keys", err) return } + if refreshedKeys == nil { + log.WithContext(ctx).Debugf("key fetcher returned nil keys, falling back to old keys") + return + } log.WithContext(ctx).Debugf("keys refreshed, new UTC expiration time: %s", refreshedKeys.ExpiresInTime.UTC()) v.keys = refreshedKeys } func (v *Validator) forceRefreshKeys(ctx context.Context) bool { v.lock.Lock() defer v.lock.Unlock() // Check cooldown inside lock to prevent multiple goroutines from refreshing if time.Since(v.lastForcedRefresh) <= forcedRefreshCooldown { return false } log.WithContext(ctx).Debugf("key not found in cache, forcing JWKS refresh") refreshedKeys, err := v.fetchKeys(ctx) if err != nil { log.WithContext(ctx).Debugf("cannot get JSONWebKey: %v, falling back to old keys", err) return false } + if refreshedKeys == nil { + log.WithContext(ctx).Debugf("key fetcher returned nil keys, falling back to old keys") + return false + } log.WithContext(ctx).Debugf("keys refreshed, new UTC expiration time: %s", refreshedKeys.ExpiresInTime.UTC()) v.keys = refreshedKeys v.lastForcedRefresh = time.Now() return true }Also applies to: 181-202
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@shared/auth/jwt/validator.go` around lines 164 - 176, The refresh functions (Validator.refreshKeys and Validator.forceRefreshKeys) assume fetchKeys returned a non-nil refreshedKeys and call refreshedKeys.ExpiresInTime.UTC(), which can panic if fetchKeys returns (nil, nil); after the existing err check, add a nil guard that logs a warning/debug and returns if refreshedKeys == nil (do not dereference), otherwise proceed to log refreshedKeys.ExpiresInTime and assign v.keys = refreshedKeys; apply the same nil-guard pattern to both refreshKeys and forceRefreshKeys.
🧹 Nitpick comments (5)
management/server/activity/store/sql_store_idp_migration.go (1)
4-4: Comment references an outdated interface name.The comment mentions
migration.MigrationEventStore, but this file implementsmigration.EventStore. Renaming the comment target would avoid confusion.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/activity/store/sql_store_idp_migration.go` at line 4, Update the top-of-file comment to reference the correct interface name: replace the mention of migration.MigrationEventStore with migration.EventStore so the comment accurately reflects that the types in this file satisfy the migration.EventStore interface via duck typing (keep the rest of the comment unchanged).management/server/store/sql_store_idp_migration.go (4)
39-46: Consider propagating context to the database query.The context parameter is used for logging but not passed to the GORM query. For consistency with cancellation/timeout propagation, consider using
WithContext:♻️ Suggested improvement
func (s *SqlStore) ListUsers(ctx context.Context) ([]*types.User, error) { - tx := s.db + tx := s.db.WithContext(ctx) var users []*types.User result := tx.Find(&users)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/store/sql_store_idp_migration.go` around lines 39 - 46, In ListUsers, the GORM query uses tx := s.db and tx.Find(&users) without propagating the provided ctx; update the DB call to use s.db.WithContext(ctx) (or assign tx := s.db.WithContext(ctx)) and then call Find(&users) so the query respects cancellation/timeouts and uses the passed context; ensure error logging still uses ctx and return behavior in ListUsers remains unchanged.
111-118: Consider verifying that the user exists before/after update.The method silently succeeds if
userIDdoesn't match any row (RowsAffected == 0). If a non-existent user ID is passed (e.g., due to a bug in the migration logic), this could mask issues.♻️ Optional: Add existence check
result := s.db.Model(&types.User{}).Where("id = ?", userID).Updates(map[string]any{ "email": user.Email, "name": user.Name, }) if result.Error != nil { log.WithContext(ctx).Errorf("error updating user info for %s: %s", userID, result.Error) return status.Errorf(status.Internal, "failed to update user info") } + if result.RowsAffected == 0 { + return status.Errorf(status.NotFound, "user %s not found", userID) + } return nil🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/store/sql_store_idp_migration.go` around lines 111 - 118, The update currently ignores the case where no rows match; after calling s.db.Model(&types.User{}).Where("id = ?", userID).Updates(...), check result.Error as you already do and then also verify result.RowsAffected > 0 (or perform a prior existence check with s.db.First(&types.User{}, "id = ?", userID)); if RowsAffected == 0 return a clear gRPC error (e.g., status.NotFound or status.Internal with a message indicating the user was not found) and log the situation via log.WithContext(ctx) including the userID and operation context; refer to the result variable, s.db, types.User, userID and the Updates call to locate where to add this check.
140-140: Inconsistent logging style.Lines 140 and 163 use
log.Info()without context, while lines 159 and 172 uselog.WithContext(ctx). For consistency and traceability, prefer using context throughout.♻️ Suggested fix
- log.Info("Updating user ID in the store") + log.WithContext(ctx).Info("Updating user ID in the store") err := s.transaction(func(tx *gorm.DB) error {- log.Info("Restoring FK constraints") + log.WithContext(ctx).Info("Restoring FK constraints") err = s.transaction(func(tx *gorm.DB) error {Also applies to: 163-163
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/store/sql_store_idp_migration.go` at line 140, The two bare log.Info(...) calls (e.g., log.Info("Updating user ID in the store")) are inconsistent with other calls using context; replace those with context-aware logging like log.WithContext(ctx).Info(...) where the surrounding function has a ctx parameter or available context, and if ctx is not present, propagate or create it so all logs in this migration use log.WithContext(ctx).Info(...) for consistent traceability.
70-83: The privilege and scope concerns are addressed by the transaction design; consider documenting database requirements.The code temporarily alters FK constraints to DEFERRABLE in a separate transaction, then restores them after updates complete—not a permanent change. Altering constraints requires table ownership (not superuser) and is appropriate for an operator-run migration tool. The global scope (selecting all constraints) is necessary because you cannot predict which tables have FKs to users without examining all constraints. However, the DEVELOPMENT.md documentation should explicitly state that the tool requires a database user with table ownership privileges on the affected tables (or a database owner/admin account).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@management/server/store/sql_store_idp_migration.go` around lines 70 - 83, The migration temporarily alters foreign key constraints using the tx.Exec block that runs the DO $$...ALTER TABLE ... ALTER CONSTRAINT ... DEFERRABLE INITIALLY IMMEDIATE and later calls tx.Exec("SET CONSTRAINTS ALL DEFERRED"); update the project documentation (DEVELOPMENT.md) to explicitly state the database privilege and scope requirements: document that the migration requires a DB user with table ownership (or a database owner/admin) to run ALTER TABLE/ALTER CONSTRAINT, that the migration runs globally across all non-deferrable FKs so operator privileges are needed, and include a short note that the change is only temporary within the migration transaction.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@management/server/store/sql_store_idp_migration.go`:
- Around line 163-174: The current pattern runs txRestoreFKConstraints in a
separate transaction after the main s.transaction commit, leaving constraints
stuck DEFERRABLE if the second transaction fails; change this by invoking
s.txRestoreFKConstraints inside the same transactional closure passed to
s.transaction (i.e., call txRestoreFKConstraints(tx) before returning from the
main transaction so restore runs and commits together), or if you must keep it
separate implement a startup recovery routine that scans and repairs
DEFERRABLE-only constraints and add a prominent error/warning log when
txRestoreFKConstraints fails that explains manual recovery steps; refer to the
s.transaction call and txRestoreFKConstraints function to locate where to apply
the change.
---
Outside diff comments:
In `@shared/auth/jwt/validator.go`:
- Around line 164-176: The refresh functions (Validator.refreshKeys and
Validator.forceRefreshKeys) assume fetchKeys returned a non-nil refreshedKeys
and call refreshedKeys.ExpiresInTime.UTC(), which can panic if fetchKeys returns
(nil, nil); after the existing err check, add a nil guard that logs a
warning/debug and returns if refreshedKeys == nil (do not dereference),
otherwise proceed to log refreshedKeys.ExpiresInTime and assign v.keys =
refreshedKeys; apply the same nil-guard pattern to both refreshKeys and
forceRefreshKeys.
---
Nitpick comments:
In `@management/server/activity/store/sql_store_idp_migration.go`:
- Line 4: Update the top-of-file comment to reference the correct interface
name: replace the mention of migration.MigrationEventStore with
migration.EventStore so the comment accurately reflects that the types in this
file satisfy the migration.EventStore interface via duck typing (keep the rest
of the comment unchanged).
In `@management/server/store/sql_store_idp_migration.go`:
- Around line 39-46: In ListUsers, the GORM query uses tx := s.db and
tx.Find(&users) without propagating the provided ctx; update the DB call to use
s.db.WithContext(ctx) (or assign tx := s.db.WithContext(ctx)) and then call
Find(&users) so the query respects cancellation/timeouts and uses the passed
context; ensure error logging still uses ctx and return behavior in ListUsers
remains unchanged.
- Around line 111-118: The update currently ignores the case where no rows
match; after calling s.db.Model(&types.User{}).Where("id = ?",
userID).Updates(...), check result.Error as you already do and then also verify
result.RowsAffected > 0 (or perform a prior existence check with
s.db.First(&types.User{}, "id = ?", userID)); if RowsAffected == 0 return a
clear gRPC error (e.g., status.NotFound or status.Internal with a message
indicating the user was not found) and log the situation via
log.WithContext(ctx) including the userID and operation context; refer to the
result variable, s.db, types.User, userID and the Updates call to locate where
to add this check.
- Line 140: The two bare log.Info(...) calls (e.g., log.Info("Updating user ID
in the store")) are inconsistent with other calls using context; replace those
with context-aware logging like log.WithContext(ctx).Info(...) where the
surrounding function has a ctx parameter or available context, and if ctx is not
present, propagate or create it so all logs in this migration use
log.WithContext(ctx).Info(...) for consistent traceability.
- Around line 70-83: The migration temporarily alters foreign key constraints
using the tx.Exec block that runs the DO $$...ALTER TABLE ... ALTER CONSTRAINT
... DEFERRABLE INITIALLY IMMEDIATE and later calls tx.Exec("SET CONSTRAINTS ALL
DEFERRED"); update the project documentation (DEVELOPMENT.md) to explicitly
state the database privilege and scope requirements: document that the migration
requires a DB user with table ownership (or a database owner/admin) to run ALTER
TABLE/ALTER CONSTRAINT, that the migration runs globally across all
non-deferrable FKs so operator privileges are needed, and include a short note
that the change is only temporary within the migration transaction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 782b4a20-6411-43ee-a399-9dbc564e2833
📒 Files selected for processing (13)
.github/workflows/check-license-dependencies.yml.goreleaser.yamlgo.modidp/dex/provider.gomanagement/internals/server/controllers.gomanagement/internals/server/modules.gomanagement/server/activity/store/sql_store_idp_migration.gomanagement/server/auth/manager.gomanagement/server/auth/manager_test.gomanagement/server/http/testing/testing_tools/channel/channel.gomanagement/server/idp/embedded.gomanagement/server/store/sql_store_idp_migration.goshared/auth/jwt/validator.go
✅ Files skipped from review due to trivial changes (2)
- management/server/http/testing/testing_tools/channel/channel.go
- management/internals/server/modules.go
🚧 Files skipped from review as they are similar to previous changes (3)
- go.mod
- idp/dex/provider.go
- .goreleaser.yaml
|



Describe your changes
This PR introduces a standalone
netbird-idp-migrateCLI tool to help operators migrate from an external IdP to the embedded Dex-based IdP introduced in Netbird some versions ago.What it does:
Key components added:
Stack
Checklist
Documentation
Select exactly one:
Docs PR URL (required if "docs added" is checked)
Paste the PR link from https://github.com/netbirdio/docs here:
netbirdio/docs#658
Summary by CodeRabbit
New Features
Tests
Documentation