Feature/azdo #1371
Feature/azdo #1371
Conversation
…aester into feature/azdo
SamErde
added
enhancement
| 3. Select Policies, locate the Request Access policy and toggle it to off. | ||
| 4. Provide the URL to your internal process for gaining access. Users see this URL in the error report when they try to access the organization or a project within the organization that they don't have permission to access. | ||
|
|
||
| **Results:** |
There was a problem hiding this comment.
Shouldn't the detailed 401 go to users in the organization and the 404 go to users not in the organization?
There was a problem hiding this comment.
I do agree!
Seems the experience has changed since;
I'll update with the new information the article.
| Rationale: Access control to Azure DevOps is to be a controlled process where access is granted and tracked. | ||
|
|
||
| #### Remediation action: | ||
| Disable the policy to stops these requests and notifications. |
There was a problem hiding this comment.
Fixed. Thank you!
| @@ -0,0 +1,19 @@ | |||
| Access to Azure DevOps SHOULD BE a controlled process provided by the IAM team. | |||
There was a problem hiding this comment.
Should this say "...by the IAM team or relevant Azure DevOps administrators roles" per below?
| Additionally, if a user is already a member of the organization, Project and Team Administrators can add that user to specific projects. | ||
|
|
||
| #### Remediation action: | ||
| Disable the policy to stops these invitations. |
There was a problem hiding this comment.
Pull request overview
This pull request adds 31 Azure DevOps security tests to the Maester project, providing comprehensive security assessments for Azure DevOps organizations. Each test includes both PowerShell implementation and markdown documentation describing the security rationale and remediation steps.
Changes:
- Added 31 new Azure DevOps security test functions covering authentication, access control, pipeline security, and resource management
- Created comprehensive markdown documentation for each test with remediation guidance
- Updated the module manifest to export all new test functions
- Added a test runner file that orchestrates all Azure DevOps security tests
Reviewed changes
Copilot reviewed 65 out of 65 changed files in this pull request and generated 28 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Maester/Azdo/Test-Azdo.Tests.ps1 | Test runner that executes all 31 Azure DevOps security tests |
| tests/Maester/Azdo/README.md | Overview documentation for Azure DevOps tests |
| powershell/public/maester/azdo/*.ps1 | 31 PowerShell functions implementing security checks |
| powershell/public/maester/azdo/*.md | 31 markdown documentation files with rationale and remediation steps |
| powershell/Maester.psd1 | Module manifest updated to export new functions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...ll/public/maester/azdo/Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipeline.ps1
Outdated
Show resolved
Hide resolved
|
|
||
| It "AZDO.1024: Disable Node 6 tasks. See https://learn.microsoft.com/en-us/azure/devops/release-notes/roadmap/2022/no-node-6-on-hosted-agents" -Tag "AZDO.1024" { | ||
|
|
||
| Test-AzdoOrganizationTaskRestrictionsDisableNode6Tasks | Should -Be $true -Because "With this enabled, pipelines will fail if they utilize a task with a Node 6 execution handler." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoOrganizationTaskRestrictionsDisableNode6Tasks (plural) but the function is named Test-AzdoOrganizationTaskRestrictionsDisableNode6Task (singular).
|
|
||
| It "AZDO.1027: Disable showing Gravatar images for users outside of your enterprise. See https://learn.microsoft.com/en-us/azure/devops/repos/git/repository-settings?view=azure-devops&tabs=browser#gravatar-images" -Tag "AZDO.1027" { | ||
|
|
||
| Test-AzdoOrganizationRepositorySettingsGravatarImages | Should -Be $false -Because "Gravatar images should not be exposed outside of your enterprise." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoOrganizationRepositorySettingsGravatarImages (plural) but the function is named Test-AzdoOrganizationRepositorySettingsGravatarImage (singular).
|
|
||
| It "AZDO.1012: Work Items Tags Limits. See https://learn.microsoft.com/en-us/azure/devops/organizations/settings/work/object-limits?view=azure-devops" -Tag "AZDO.1012" { | ||
|
|
||
| Test-AzdoResourceUsageWorkItemTags | Should -Be $true -Because "Azure DevOps supports up to 150,000 tag definitions per organization or collection." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoResourceUsageWorkItemTags (plural) but the function is named Test-AzdoResourceUsageWorkItemTag (singular).
|
|
||
| It "AZDO.1016: Limit job authorization scope to current project for non-release pipelines. See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/access-tokens?view=azure-devops&tabs=yaml#job-authorization-scope" -Tag "AZDO.1016" { | ||
|
|
||
| Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipelines | Should -Be $true -Because "With this option enabled, you can reduce the scope of access for all classic release pipelines to the current project." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipelines (plural) but the function is named Test-AzdoOrganizationLimitJobAuthorizationScopeNonReleasePipeline (singular).
|
|
||
| It "AZDO.1023: Disable Marketplace tasks. See https://learn.microsoft.com/en-us/azure/devops/pipelines/security/overview?view=azure-devops#prevent-malicious-code-execution" -Tag "AZDO.1023" { | ||
|
|
||
| Test-AzdoOrganizationTaskRestrictionsDisableMarketplaceTasks | Should -Be $false -Because "Disable the ability to install and run tasks from the Marketplace, which gives you greater control over the code that executes in a pipeline." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoOrganizationTaskRestrictionsDisableMarketplaceTasks (plural) but the function is named Test-AzdoOrganizationTaskRestrictionsDisableMarketplaceTask (singular).
|
|
||
| It "AZDO.1026: Enable automatic enrollment to Advanced Security for Azure DevOps. See https://learn.microsoft.com/en-us/azure/devops/repos/security/configure-github-advanced-security-features?view=azure-devops&tabs=yaml#organization-level-onboarding" -Tag "AZDO.1026" { | ||
|
|
||
| Test-AzdoOrganizationAutomaticEnrollmentAdvancedSecurityNewProjects | Should -Be $true -Because "Enable automatic enrollment for new git repositories to use GitHub Advanced Security for Azure DevOps. It adds GitHub Advanced Security's suite of security features to Azure Repos." |
There was a problem hiding this comment.
The function name called in the test does not match the actual function name defined in the PowerShell file. The test calls Test-AzdoOrganizationAutomaticEnrollmentAdvancedSecurityNewProjects (plural) but the function is named Test-AzdoOrganizationAutomaticEnrollmentAdvancedSecurityNewProject (singular).
| Disable the policy to stops these requests and notifications. | ||
| 1. Sign in to your organization | ||
| 2. Choose Organization settings. | ||
| 3. Select Settings under Pieplines, locate the "Disable stage chooser" policy and toggle it to on. |
There was a problem hiding this comment.
There is a typo in the documentation. "Pieplines" should be "Pipelines".
| @@ -0,0 +1,18 @@ | |||
| Connecting to Azure DevOps using SSH should be disabled. | |||
|
|
|||
| Rationale: Oauth is the prefered and most secure authentication method. | |||
There was a problem hiding this comment.
There is a typo in the description. "Oauth" should be "OAuth" (proper capitalization).
| 4. Locate the "SSH authentication" policy and toggle it to off. | ||
|
|
||
| **Results:** | ||
| Users can no longer user SSH to connect to Azure DevOps. |
There was a problem hiding this comment.
The description contains a grammatical error. "user" should be "use".
Corrected typos and formatting in the documentation.
Removed unnecessary commas and spaces. Added line breaks for MD linting.
Corrected grammatical errors and added punctuation for clarity.
Updated description to clarify the function's purpose and improve readability.
Corrected grammatical errors and improved clarity in remediation instructions.
Clarified the purpose of the Azure DevOps tests and updated the reference link.
| @@ -0,0 +1,195 @@ | |||
| BeforeAll { | |||
There was a problem hiding this comment.
You shouldn't need to dot-source all of these functions when they are included in the manifest's FunctionsToExport array.
There was a problem hiding this comment.
I suppose it's because I have had the tests locally, and not built the Maester PowerShell Module.
I'll remove it, build the module locally and make sure it works as expected.
| . $PSScriptRoot/Test-AzdoThirdPartyAccessViaOauth.ps1 | ||
| } | ||
|
|
||
| Describe "Azure DevOps" -Tag "Azure DevOps Security" { |
There was a problem hiding this comment.
Suggest changing this tag to just 'Azure DevOps' since 'security' is implied in every test that Maester performs.
There was a problem hiding this comment.
I agree. Fixed!
…thorizationScopeNonReleasePipeline.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
This is an awesome addition, @SebastianClaesson! Please review these suggestions from the initial review. I have gone through a number of the files and made some minor edits to spelling, punctuation, etc; Copilot has found a few other potential issues. I'm also curious if @maester365/core-tests has any thoughts on standardizing the location for added tests. Would the best location for these be in (Super minor nit-pick: I'm not sure how I feel about the casing of 'Azdo' vs 'azdo' vs potentially just using 'AzureDevOps'.) 🤓 |
| . $PSScriptRoot/Test-AzdoSSHAuthentication.ps1 | ||
| . $PSScriptRoot/Test-AzdoThirdPartyAccessViaOauth.ps1 | ||
| } | ||
|
|
There was a problem hiding this comment.
This test might benefit from having a check at the top to see if the organization is licensed to use Azure DevOps at all. If not in use, you could immediately skip the rest of the tests with a "not licensed" reason, which reduces the number of calls and the overall runtime of Maester.
I agree with the 'AzureDevOps' naming. I think that makes sense, as Azdo might be an abbreviation for something else for some! Thank you so much, I really apprieciate you taking the time for feedback and ensuring high quality! |
2 participants
Description
Fixes #1368
Adding 31 Azure DevOps security tests.
Each of the tests is added to the new folder azdo in maester\public\maester
All of the tests have a markdown file, with :
Contribution Checklist
Before submitting this PR, please confirm you have completed the following:
/powershell/tests/pester.ps1on your local system.