v0.6.0

What's Changed

• Cookbooks are live, and replace our How To Guide! Cookbooks try to tackle a problem and use the components that work best, instead of our typical documentation which illustrates that many pieces of PyRITs are swappable.

Cookbooks:

• Sending a Million Prompts

Targets:

• OllamaChatTarget: Implement ability to forward custom parameters directly to the HTTP client
• HuggingFaceChatTarget: Adds optional keywords device_map, torch_dtype and attn_implementation
• [New] PlaywrightTarget: Interact with web applications using Playwright. This is particularly useful for testing interactions with web interfaces like chatbots.
• [New] RealtimeTarget: Send and receive audio with the Realtime API.
• [New] GroqChatTarget: Interact with Groq's OpenAI-compatible API.

Converters:

• [New] ANSI Escape Code Converter: AnsiAttackConverter
• [New] BinaryConverter: Convert input text into binary with configurable bits per character
• PDFConverter: Updates to support templated and non-templated PDF generation & enabling text injection into existing PDFs
• [New] TextToHexConverter: Convert text to hexadecimal encoded utf-8 string
• Add easier querying for converter-supported input/output types

Orchestrators:

• RedTeamingOrchestrator & CrescendoOrchestrator now support prepended conversations. You can set a system prompt on the objective target using this feature, or provide conversation history as context to continue execution from a specific point.
• ScoringOrchestrator: Add ability to score responses using filters.
• PromptSendingOrchestrator: Set Skip Criteria to specify which prompts to skip being sent to the target with this orchestrator.
• [New] RolePlayingOrchestrator: Single-turn orchestrator which prepends some prompts which describe fictional scenarios to attempt and elicit harmful responses
• XPIAOrchestrator: Fix to BlobNotFound exception
   
  Memory:
• [BREAKING] All notebooks must explicitly initialize Central Memory through a new initialize_pyrit() function: #616. This puts ownership into the hands of the user to set where your prompts will be stored. Read more here: Memory
• Ability to add memory labels on a per-prompt level, specifically useful in Multimodal scenarios
• Conversation Scores now available when exporting Prompt Data
• Filter Data by various queries (e.g. prompt ID, orchestrator ID, labels, etc) using get_prompt_request_pieces()
• Consolidated method to Export Conversations using Filters: export_conversations()
• SeedPrompts: Support for Multimodal Seed Prompts
• [BREAKING] NormalizerRequestPieces replaced with SeedPrompts: #648

Scorers:

• Add tasks by default to scorers to improve scorer accuracy

Misc:

• Other changes include various maintenance improvements and bug fixes, addition of integration tests, new blog posts, and doc improvements.

Full list of changes

• MAINT Update release version to 0.5.3.dev0 by @rdheekonda in #592
• DOC: Multi-turn docs and blog post by @rlundeen2 in #593
• DOC: Fixing title by @rlundeen2 in #594
• MAINT: Update Memory Doc and Other Small Fixes by @jsong468 in #587
• FEAT Passing HTTP client kwargs from OllamaChatTarget by @rlundeen2 in #596
• MAINT: Refactoring Single-Turn by @rlundeen2 in #598
• DOC: Clarifying OpenAI docs by @rlundeen2 in #600
• FEAT - Adding optional kwargs to huggingface chat target by @perezbecker in #602
• FEAT: Ansi Escape Code Converter by @KutalVolkan in #597
• MAINT Update gcg_attack.py by @Tiger-Du in #606
• MAINT empty integration tests pipeline by @romanlutz in #603
• MAINT update integration-tests trigger to work with PRs by @romanlutz in #610
• FEAT: Playwright target by @AlexRRR in #583
• MAINT Add support for Local Multimodal Input Prompts When Using AzureSQLMemory by @rdheekonda in #613
• MAINT: Add Integration Test Directory + Refusal Scorer Eval Integration Test by @jsong468 in #605
• FEAT: Add Prepending Conversation Support to RedTeamingOrchestrator and CrescendoOrchestrator by @nina-msft in #578
• FIX: Adding SHA256 hashes to responses by @rlundeen2 in #615
• FEAT: binary converter by @AlexRRR in #611
• FIX: Update pyproject.toml for new versions for httpx, respx and openai by @jsong468 in #623
• FEAT Adding labels for individual prompts by @jbolor21 in #624
• FEAT Add Scores to Data Export with PromptRequestPiece data by @eugeniavkim in #617
• FEAT: Prompt Memory Consolidation and Filters by @rlundeen2 in #625
• FEAT: PDF Converter Updates by @KutalVolkan in #622
• FIX: small edits to populate_prompt_piece_scores by @jsong468 in #626
• DOC: Updating contributor docs by @rlundeen2 in #627
• FEAT Consolidate Export Conversations into one method by @eugeniavkim in #628
• FEAT: Adding tasks to scorers by @rlundeen2 in #629
• FIX: sort_request_pieces bug by @rlundeen2 in #631
• FEAT: Allowing header SeedPrompt configuration by @rlundeen2 in #630
• FEAT: Add Support for Multimodal Seed Prompts and Update Data Type Serializer by @rdheekonda in #632
• FEAT: Explicitly Initialize Central Memory + Remove Defaults by @nina-msft in #616
• FIX Refactor to join queries for entries and scores by @eugeniavkim in #635
• MAINT: Cleanup Import Naming for initialize_pyrit func by @nina-msft in #636
• FEAT: Score Responses by Filters in ScoringOrchestrator by @nina-msft in #639
• MAINT infrastructure for integration tests by @romanlutz in #612
• MAINT: Add JSON Mode for Supported Targets and Scorers by @rdheekonda in #640
• DOC: Zero Day Quest blog post by @rlundeen2 in #643
• MAINT: Add Import Sorting (isort) Pre-Commit Hook by @nina-msft in #644
• FIX: Rerun Output for Audio Converter Notebook by @nina-msft in #645
• MAINT: Add Import Sorting for Docs and Jupyter Notebooks (isort/nbqa-isort) by @nina-msft in #646
• TEST: Converter Notebook Integration Tests by @nina-msft in #647
• FEAT: Replacing NormalizerRequestPieces with SeedPrompts by @rlundeen2 in #648
• MAINT: Remove Azure SQL Example from Audio Converters Notebook by @nina-msft in #649
• FIX: adding hashes to retrieved PromptRequestPiece by @rlundeen2 in #652
• DOC: Clarifying PromptTargets from PromptChatTargets by @rlundeen2 in #658
• DOC update pyrit.common API reference by @paulinek13 in #657
• FEAT - Realtime Target by @jbolor21 in #638
• MAINT: Updating get_seed_prompt_groups to include individual seed_prompts by @rlundeen2 in #651
• DOC: Deleting extra doc by @rlundeen2 in #663
• FIX: Fixing circular import by @rlundeen2 in #665
• DOC Cleaning up Datasets and adding documentation for datasets and seed prompts by @eugeniavkim in #660
• DOC Adding NCC HTTPTarget Blog post by @jbolor21 in #664
• TEST Integration Tests for Target Notebooks by @jbolor21 in #667
• FEAT: Enhance PDFConverter to support text injection into existing PDFs by @KutalVolkan in #641
• FIX Target Integration test rename by @jbolor21 in #675
• FEAT: Adding Skip Criteria and Sending Prompts Cookbook by @rlundeen2 in #668
• FIX: http target bug by @ayeganov in #674
• FEAT add value hash columns and calc hash when committing seed prompt to memory by @jorisdg in #659
• TEST: Integration Tests for Python Notebooks (Auxiliary Attacks, Datasets, Memory) by @nina-msft in #670
• FIX: PDF Converter and Cookbook integration test by @rlundeen2 in #680
• FEAT: adding hex code converter (#666) by @millashin in #681
• FIX: Converter PDF Integration Build Pipeline by @rlundeen2 in #683
• TEST Integration Tests for Orchestrators & Scorers by @jbolor21 in #677
• FEAT add Discord link to the website by @paulinek13 in #682
• FIX: Integration Test Pipeline by @jbolor21 in #686
• FEAT: RolePlayingOrchestrator by @rlundeen2 in #679
• FEAT: Add Conversation/Prompt ID information to exceptions (for MultiTurnOrchestrators) by @jsong468 in #637
• FIX: Integration Pipeline by @jbolor21 in #689
• MAINT: Add Application Insights by @jsong468 in #692
• FIX Integration Test Pipeline by @jbolor21 in #693
• FEAT Role Play Trivia Game Template by @jsong468 in #700
• FIX: skip link checking in blog by @jbolor21 in #697
• DOC - Adding blog post for Seed Prompts by @eugeniavkim in #703
• FEAT POC for scanner with prompt sending orchestrator by @romanlutz in #673
• FEAT Add easier querying for supported input/output types by @paulinek13 in #688
• MAINT: address deprecation warnings in unit tests by @paulinek13 in #691
• Fix XPIAOrchestrator Blob Not Found Exception by @rdheekonda in #694
• FEAT: Add GroqChatTarget (#704) by @jsdlm in #705

New Contributors

• @perezbecker made their first contribution in #602
• @paulinek13 made their first contribution in #657
• @ayeganov made their first contribution in #674
• @millashin made their first contribution in #681
• @jsdlm made their first contribution in #705

Full Changelog: v0.5.2...v0.6.0