Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ For example, [BREAKING] FEAT or [BREAKING] MAINT -->

<!--- JupyText helps us see regressions in APIs or in our documentation by executing all code samples -->
<!--- Include how you/if ran JupyText here -->
<!--- This is described at: https://github.com/Azure/PyRIT/tree/main/doc -->
<!--- This is described at: https://github.com/microsoft/PyRIT/tree/main/doc -->
2 changes: 1 addition & 1 deletion .pyrit_conf_example
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# or specify a custom path when loading via --config-file.
#
# For documentation on configuration options, see:
# https://github.com/Azure/PyRIT/blob/main/doc/setup/configuration.md
# https://github.com/microsoft/PyRIT/blob/main/doc/setup/configuration.md

# Memory Database Type
# --------------------
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ authors:
title: "PyRIT: The Python Risk Identification Tool for generative AI"
doi: https://doi.org/10.48550/arXiv.2410.02828
date-released: 2024-02-21
url: "https://github.com/Azure/PyRIT"
url: "https://github.com/microsoft/PyRIT"
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The Python Risk Identification Tool for generative AI (PyRIT) is an open source
framework built to empower security professionals and engineers to proactively
identify risks in generative AI systems.

- Check out our [website](https://azure.github.io/PyRIT/) for more information
- Check out our [website](https://microsoft.github.io/PyRIT/) for more information
about how to use, install, or contribute to PyRIT.
- Visit our [Discord server](https://discord.gg/9fMpq3tc8u) to chat with the team and community.

Expand Down
8 changes: 4 additions & 4 deletions build_scripts/generate_rss.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,10 @@ def extract_date_from_filename(filename: str) -> str:
# Generate the RSS feed structure
print("Generating RSS feed structure...")
fg = FeedGenerator()
fg.link(href="https://azure.github.io/PyRIT/blog/rss.xml", rel="self")
fg.link(href="https://microsoft.github.io/PyRIT/blog/rss.xml", rel="self")
fg.title("PyRIT Blog")
fg.description("PyRIT Blog")
fg.logo("https://azure.github.io/PyRIT/_static/roakey.png")
fg.logo("https://microsoft.github.io/PyRIT/_static/roakey.png")
fg.language("en")

# Iterate over the blog source markdown files
Expand All @@ -93,8 +93,8 @@ def extract_date_from_filename(filename: str) -> str:
fe = fg.add_entry()
# Blog pages are served at blog/<filename_without_ext>
page_name = file.stem
fe.link(href=f"https://azure.github.io/PyRIT/blog/{page_name}")
fe.guid(f"https://azure.github.io/PyRIT/blog/{page_name}")
fe.link(href=f"https://microsoft.github.io/PyRIT/blog/{page_name}")
fe.guid(f"https://microsoft.github.io/PyRIT/blog/{page_name}")

title, description = parse_blog_markdown(file)
fe.title(title)
Expand Down
2 changes: 1 addition & 1 deletion doc/_static/custom.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ window.addEventListener("DOMContentLoaded", () => {
rssLink.rel = 'alternate';
rssLink.type = 'application/rss+xml';
rssLink.title = 'PyRIT Blog RSS Feed';
rssLink.href = 'https://azure.github.io/PyRIT/blog/rss.xml';
rssLink.href = 'https://microsoft.github.io/PyRIT/blog/rss.xml';

document.head.appendChild(rssLink);
}
Expand Down
4 changes: 2 additions & 2 deletions doc/blog/2024_12_3.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Over time, certain patterns have emerged—one of the most common being the mult

## The problem

If you look at some of the code from release [0.4.0](https://github.com/Azure/PyRIT/tree/releases/v0.4.0) in August, you may notice some weirdness.
If you look at some of the code from release [0.4.0](https://github.com/microsoft/PyRIT/tree/releases/v0.4.0) in August, you may notice some weirdness.

The Red Teaming Orchestrator, Crescendo [@russinovich2024crescendo], TAP [@mehrotra2023tap], and PAIR [@chao2023pair] all follow a similar setup: you configure your attack LLM, scorer, and target, then send prompts to achieve an objective. However, their implementation details vary.

Expand Down Expand Up @@ -51,6 +51,6 @@ See the updated documentation [here](../code/executor/attack/2_red_teaming_attac

## What's next?

Orchestrators are, at their core, meant to remain top-level components. While we've made strides in standardization, there's still room for improvement. For instance, we're planning to standardize the `PromptSendingOrchestrator` in a similar way (including updating its naming for consistency). And we've opened a [few issues](https://github.com/Azure/PyRIT/issues/585) for feature parity between MultiTurnOrchestrators.
Orchestrators are, at their core, meant to remain top-level components. While we've made strides in standardization, there's still room for improvement. For instance, we're planning to standardize the `PromptSendingOrchestrator` in a similar way (including updating its naming for consistency). And we've opened a [few issues](https://github.com/microsoft/PyRIT/issues/585) for feature parity between MultiTurnOrchestrators.

Hope you enjoyed this little post. There will be more content like this coming!
14 changes: 7 additions & 7 deletions doc/blog/2025_01_27.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Those who have interacted with me are aware that I'm a strong advocate for autom

Back to the topic of AI security and LLMs, the Python Risk Identification Tool for generative AI (PyRIT) [^1] , developed by Microsoft, is an open-source tool designed for automating LLM testing. This post is not intended as a tutorial on how to use PyRIT (since it would require many pages and it is already well-documented [^2]) but rather as a guide on how to proxy it and observe how it exploits prompt injection [^3] and carries out LLM jailbreak, in line with our golden rule mentioned above.

[^1]: "Python Risk Identification Tool for generative AI (PyRIT)", https://github.com/Azure/PyRIT
[^1]: "Python Risk Identification Tool for generative AI (PyRIT)", https://github.com/microsoft/PyRIT

[^2]: "PyRIT Documentation", https://azure.github.io/PyRIT/
[^2]: "PyRIT Documentation", https://microsoft.github.io/PyRIT/

[^3]: "Exploring Prompt Injection", https://www.nccgroup.com/us/research-blog/exploring-prompt-injection-attacks/

Expand All @@ -36,15 +36,15 @@ PyRIT implements many kinds of "targets", but I mostly use the "HTTPTarget" as t

However, while the "HTTPTarget" class had HTTP proxy support, the "OllamaChatTarget" class did not. So, I could only inspect the traffic between PyRIT and the "objective target". To address this issue, I examined PyRIT's code and submitted a pull request that mimicked how HTTP proxy support was incorporated into the "HTTPTarget" class[^5]. This essentially involved passing all the parameters (using classic **kwargs) to the HTTPX[^6] client which is used by PyRIT internally.

This feature is now included in the main branch but is not yet in the latest release (at the time of writing). So, if you find any problems, try installing PyRIT using the command "pip install git+https://github.com/Azure/PyRIT/". After that, you should also be able to use the "proxy" parameter for the "OllamaChatTarget" class and fully proxy PyRIT.
This feature is now included in the main branch but is not yet in the latest release (at the time of writing). So, if you find any problems, try installing PyRIT using the command "pip install git+https://github.com/microsoft/PyRIT/". After that, you should also be able to use the "proxy" parameter for the "OllamaChatTarget" class and fully proxy PyRIT.

![image](proxypyrit_figure1.png)

<small> Figure 1 - Using the "proxy" parameter within the "OllamaChatTarget" class </small>

[^4]: "Ollama: Get up and running with large language models", https://ollama.com

[^5]: "FEAT Passing HTTP client kwargs from OllamaChatTarget", https://github.com/Azure/PyRIT/pull/596
[^5]: "FEAT Passing HTTP client kwargs from OllamaChatTarget", https://github.com/microsoft/PyRIT/pull/596

[^6]: "HTTPX: A next-generation HTTP client for Python", https://www.python-httpx.org

Expand Down Expand Up @@ -78,11 +78,11 @@ Finally, when PyRIT gets a response from the Target LLM, it switches to another

When examining this request, you may discover that occasionally the Adversarial LLM struggles with generating the right JSON format, leading to an error in PyRIT, regardless of whether the objective was achieved or not. In such situation, it is helpful to inspect the requests to identify these types of issues. Specifically, I found a problem when the LLM response contained double quotes, causing issues with subsequent JSON formats which was fixed using the "SearchReplaceConverter"[^9] prompt converter.

[^7]: "Multi-Turn Attack - RedTeamingAttack Example", https://azure.github.io/PyRIT/code/executor/attack/2_red_teaming_attack.html
[^7]: "Multi-Turn Attack - RedTeamingAttack Example", https://microsoft.github.io/PyRIT/code/executor/attack/2_red_teaming_attack.html

[^8]: "PyRIT - SearchReplaceConverter", https://azure.github.io/PyRIT/_autosummary/pyrit.prompt_converter.SearchReplaceConverter.html
[^8]: "PyRIT - SearchReplaceConverter", https://microsoft.github.io/PyRIT/_autosummary/pyrit.prompt_converter.SearchReplaceConverter.html

[^9]: "PyRIT - True False Scoring", https://azure.github.io/PyRIT/code/scoring/2_true_false_scorers.html#true-false-scoring
[^9]: "PyRIT - True False Scoring", https://microsoft.github.io/PyRIT/code/scoring/2_true_false_scorers.html#true-false-scoring

### Final Thoughts

Expand Down
6 changes: 3 additions & 3 deletions doc/blog/2025_03_03.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<small>3 Mar 2025 - Rich Lundeen</small>

One of the first targets we built was OpenAIChatTarget. A lot of models are compatible with OpenAI, so this should work with all of those, right? There are some nuanced reasons why it didn't always work. But with [this commit](https://github.com/Azure/PyRIT/commit/924ba48ff2c56c2532190b7b6dca3bad412d3bc2), we should more broadly support OpenAI-compatible models.
One of the first targets we built was OpenAIChatTarget. A lot of models are compatible with OpenAI, so this should work with all of those, right? There are some nuanced reasons why it didn't always work. But with [this commit](https://github.com/microsoft/PyRIT/commit/924ba48ff2c56c2532190b7b6dca3bad412d3bc2), we should more broadly support OpenAI-compatible models.

DeepSeek launching about a month ago illustrates the problem well. We wanted to take a look at it with PyRIT, and because it has an "OpenAI-compatible API," it seemed like it should work out of the box... but it didn't. Since we have a dev team, we were able to quickly unblock people wanting to look at this. But the fact that it didn't work initially is interesting and a problem we wanted to tackle.

Expand All @@ -27,7 +27,7 @@ But the exact same request to an Azure OpenAI endpoint works as expected.

![alt text](2025_03_03_4.png)

When DeepSeek was released, some of our default parameters caused it to fail. This isn't the first time an extra parameter has bitten us. When o1 came out, we learned `max_tokens` was incompatible with `max_completion_tokens`, and because we were sending one by default, our target didn't work on o1 without [a fix](https://github.com/Azure/PyRIT/pull/501/).
When DeepSeek was released, some of our default parameters caused it to fail. This isn't the first time an extra parameter has bitten us. When o1 came out, we learned `max_tokens` was incompatible with `max_completion_tokens`, and because we were sending one by default, our target didn't work on o1 without [a fix](https://github.com/microsoft/PyRIT/pull/501/).

With this update, we're not sending most parameters by default to make the requests as simple and compatible as possible. In addition, we're working on scanner configurations, so you can set your own defaults for various targets. That work is coming soon.

Expand Down Expand Up @@ -129,6 +129,6 @@ async def test_connect_required_openai_text_targets(endpoint, api_key, model_nam

This is such a nuanced issue. I may have over-explained because, at first glance, it seems like "OpenAI-compatible" models should have just worked with PyRIT. Hopefully that's more true now than it used to be! As a concrete step in this directions, we've removed the `OllamaChatTarget` and `GroqChatTarget` since they are compatible now.

As always, we're open to any feedback, and please [open github issues](https://github.com/Azure/PyRIT/issues) if you find PyRIT doesn't work with specific OpenAI-compatible targets.
As always, we're open to any feedback, and please [open github issues](https://github.com/microsoft/PyRIT/issues) if you find PyRIT doesn't work with specific OpenAI-compatible targets.

Happy Hacking!
8 changes: 4 additions & 4 deletions doc/blog/2025_06_06.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ The [AI Recruiter](https://github.com/KutalVolkan/ai_recruiter) is designed to m

- Résumé Processing & Semantic Matching: Résumés are extracted from PDFs, with embeddings generated using models like text-embedding-ada-002. These embeddings enable semantic matching, while GPT-4o is later used to assign a match score based on relevance and extracted content.

- Automated RAG Vulnerability Testing: Attackers can manipulate résumé content by injecting hidden text (via a [PDF converter](https://github.com/Azure/PyRIT/blob/main/doc/code/converters/pdf_converter.ipynb)) that optimizes scoring, influencing the AI Recruiter’s ranking system.
- Automated RAG Vulnerability Testing: Attackers can manipulate résumé content by injecting hidden text (via a [PDF converter](https://github.com/microsoft/PyRIT/blob/main/doc/code/converters/pdf_converter.ipynb)) that optimizes scoring, influencing the AI Recruiter’s ranking system.

- [XPIA Attack](https://github.com/Azure/PyRIT/blob/main/doc/code/executor/workflow/2_xpia_ai_recruiter.ipynb) Integration: PyRIT enables full automation of prompt injections, making AI vulnerability research efficient and reproducible.
- [XPIA Attack](https://github.com/microsoft/PyRIT/blob/main/doc/code/executor/workflow/2_xpia_ai_recruiter.ipynb) Integration: PyRIT enables full automation of prompt injections, making AI vulnerability research efficient and reproducible.
---

## The Exploit in Detail: Step-by-Step
Expand Down Expand Up @@ -84,9 +84,9 @@ As we integrate AI into more facets of our lives, it’s imperative to build sys

*Explore More:*

- [XPIA Website Attack Notebook](https://github.com/Azure/PyRIT/blob/main/doc/code/executor/workflow/1_xpia_website.ipynb)
- [XPIA Website Attack Notebook](https://github.com/microsoft/PyRIT/blob/main/doc/code/executor/workflow/1_xpia_website.ipynb)

- [XPIA AI Recruiter Attack Notebook](https://github.com/Azure/PyRIT/blob/main/doc/code/executor/workflow/2_xpia_ai_recruiter.ipynb)
- [XPIA AI Recruiter Attack Notebook](https://github.com/microsoft/PyRIT/blob/main/doc/code/executor/workflow/2_xpia_ai_recruiter.ipynb)

- [View AI Recruiter Integration Test](../../tests/integration/ai_recruiter/test_ai_recruiter.py)

Expand Down
2 changes: 1 addition & 1 deletion doc/code/converters/3_image_converters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
"\n",
"await initialize_pyrit_async(memory_db_type=IN_MEMORY) # type: ignore\n",
"\n",
"prompt = \"https://github.com/Azure/PyRIT\"\n",
"prompt = \"https://github.com/microsoft/PyRIT\"\n",
"\n",
"qr_converter = QRCodeConverter()\n",
"qr_result = await qr_converter.convert_async(prompt=prompt) # type: ignore\n",
Expand Down
2 changes: 1 addition & 1 deletion doc/code/converters/3_image_converters.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@

await initialize_pyrit_async(memory_db_type=IN_MEMORY) # type: ignore

prompt = "https://github.com/Azure/PyRIT"
prompt = "https://github.com/microsoft/PyRIT"

qr_converter = QRCodeConverter()
qr_result = await qr_converter.convert_async(prompt=prompt) # type: ignore
Expand Down
4 changes: 3 additions & 1 deletion doc/code/converters/5_file_converters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,9 @@
"import requests\n",
"\n",
"# Download a sample PDF\n",
"url = \"https://raw.githubusercontent.com/Azure/PyRIT/main/pyrit/datasets/prompt_converters/pdf_converters/fake_CV.pdf\"\n",
"url = (\n",
" \"https://raw.githubusercontent.com/microsoft/PyRIT/main/pyrit/datasets/prompt_converters/pdf_converters/fake_CV.pdf\"\n",
")\n",
"\n",
"with tempfile.NamedTemporaryFile(delete=False, suffix=\".pdf\") as tmp_file:\n",
" response = requests.get(url)\n",
Expand Down
4 changes: 3 additions & 1 deletion doc/code/converters/5_file_converters.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,9 @@
import requests

# Download a sample PDF
url = "https://raw.githubusercontent.com/Azure/PyRIT/main/pyrit/datasets/prompt_converters/pdf_converters/fake_CV.pdf"
url = (
"https://raw.githubusercontent.com/microsoft/PyRIT/main/pyrit/datasets/prompt_converters/pdf_converters/fake_CV.pdf"
)

with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp_file:
response = requests.get(url)
Expand Down
2 changes: 1 addition & 1 deletion doc/code/executor/workflow/1_xpia_website.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
"source": [
"\n",
"_Note:_ to run this section of the demo you need to setup your `.env` file to properly authenticate to an Azure Storage Blob Container and an Azure OpenAI target.\n",
"See the section within [.env_example](https://github.com/Azure/PyRIT/blob/main/.env_example) if not sure where to find values for each of these variables.\n",
"See the section within [.env_example](https://github.com/microsoft/PyRIT/blob/main/.env_example) if not sure where to find values for each of these variables.\n",
"\n",
"Below, we define a simple agent using OpenAI's responses API to retrieve content from websites.\n",
"This is to simulate a processing target similar to what one might expect in an XPIA-oriented AI red teaming operation."
Expand Down
2 changes: 1 addition & 1 deletion doc/code/executor/workflow/1_xpia_website.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
# %% [markdown]
#
# _Note:_ to run this section of the demo you need to setup your `.env` file to properly authenticate to an Azure Storage Blob Container and an Azure OpenAI target.
# See the section within [.env_example](https://github.com/Azure/PyRIT/blob/main/.env_example) if not sure where to find values for each of these variables.
# See the section within [.env_example](https://github.com/microsoft/PyRIT/blob/main/.env_example) if not sure where to find values for each of these variables.
#
# Below, we define a simple agent using OpenAI's responses API to retrieve content from websites.
# This is to simulate a processing target similar to what one might expect in an XPIA-oriented AI red teaming operation.
Expand Down
4 changes: 2 additions & 2 deletions doc/code/gui/0_gui.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Then open `http://localhost:8000` in your browser.

### Docker

CoPyRIT is also available as a Docker container. See the [Docker setup](https://github.com/Azure/PyRIT/blob/main/docker/) for details.
CoPyRIT is also available as a Docker container. See the [Docker setup](https://github.com/microsoft/PyRIT/blob/main/docker/) for details.

### Azure Deployment

Expand Down Expand Up @@ -149,7 +149,7 @@ Click "New Target" to open the creation dialog. Fill in:

#### Auto-Populating Targets

Targets can also be auto-populated by adding an initializer (e.g., `airt`) to your `~/.pyrit/.pyrit_conf` file. This reads endpoints from your `.env` and `.env.local` files. See [.pyrit_conf_example](https://github.com/Azure/PyRIT/blob/main/.pyrit_conf_example) for details.
Targets can also be auto-populated by adding an initializer (e.g., `airt`) to your `~/.pyrit/.pyrit_conf` file. This reads endpoints from your `.env` and `.env.local` files. See [.pyrit_conf_example](https://github.com/microsoft/PyRIT/blob/main/.pyrit_conf_example) for details.

---

Expand Down
Loading
Loading