Skip to content

BUG: WordLevelConverter rejoins words with a space, discarding a custom word_split_separator #2592

Description

Describe the bug

WordLevelConverter splits the prompt on word_split_separator but always rejoins the words with a space, so a custom separator is silently replaced in the output.

WordLevelConverter.convert_async splits with prompt.split(self._word_split_separator) (pyrit/converter/word_level_converter.py:116), while the default join_words returns " ".join(words) (:91). The split and the join do not agree.

EmojiConverter and BinAsciiConverter both expose word_split_separator in their own constructors, so this is reachable through the public API.

Steps to reproduce

import asyncio

from pyrit.converter import EmojiConverter


async def main():
    converter = EmojiConverter(seed=1, word_split_separator=",")
    result = await converter.convert_async(prompt="alpha,beta,gamma", input_type="text")
    print(result.output_text)


asyncio.run(main())

Expected behavior

The commas that delimited the words are still present:

🅐🅛🄿🅗🄰,🄱🅴🅣🅰️,🄶🅐🅜🄼🅰️

Actual behavior

They are replaced with spaces:

🅐🅛🄿🅗🄰 🄱🅴🅣🅰️ 🄶🅐🅜🄼🅰️

Any structure carried by the delimiter is lost, and the output cannot be split back into the words that were converted.

Additional context

Rejoining with the separator the words were split on resolves this. The change is a no-op for existing callers:

  • word_split_separator=" " (the default) already joins with " "
  • word_split_separator=None splits on arbitrary whitespace, which has no single representation to restore, so it keeps joining with a space

Only the custom-separator path changes, and it is currently broken. No existing test asserts the space-join for a custom separator — tests/unit/converter/test_emoji_converter.py:53 sets word_split_separator="|" but only asserts the identifier params, not the output.

Subclasses that override join_words (BinaryConverter, NatoConverter, FirstLetterConverter, UnicodeReplacementConverter, BinAsciiConverter) keep their own joining behavior.

I have a PR ready with the fix and tests.

Environment: PyRIT 1.2.0.dev0 (main at 623d57a), Python 3.14, Linux.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions