RP Bench is Mana’s community benchmark for AI roleplay models. On a share of your turns in a Realism or Hardcore visual story, two anonymous models write the response and you pick the one you prefer. Those blind votes build the public RP Bench Leaderboard, a ranking of which models write the best roleplay, decided by the players who read it.
What happens on an RP Bench turn
- Two models answerBoth get the same story, the same characters and memories, and the same message from you. You see Response A and Response B, with no model names.
- You pick oneChoose the response you prefer. There is no right answer: pick the one you would rather continue the story with. If a response dodged the scene because of a safety filter, you can flag it instead (see below).
- The models are revealedAfter your pick you see which model wrote which response.
- The story continuesYour pick becomes the turn. It is saved to the story like any other response.
The narration model is free on an RP Bench turn. The other parts of the turn, such as character status and voices, cost what they normally cost.
RP Bench turns only happen in Realism and Hardcore visual stories, and never in a story’s first three turns.
Any text turn can become an RP Bench turn: Say, Do, Story, Instruct, Continue and Redo, including a Hardcore turn after its dice have been rolled (both models get the same roll results). See and the story’s automatic opening never do.
Both models are drawn from the RP Bench pool, whichever model you picked for the story. The pool is a selection of the models in the story model picker; the default Free and Paid models are never in it.
Some models avoid a scene without refusing outright: they steer the story somewhere else. If a response does that, choose “Response A (or B) Broke Narrative Immersion due to safety filter”. The story continues with the other response, the flag counts as a safety refusal for that model, and no vote is recorded.
A pick or flag made in under 5 seconds still continues your story, but it is not counted.
The vote waits up to 30 minutes. If you leave the story without picking, the turn is dropped: send your message again to continue.
Two models that have met enough times are not matched again. Once every pair has enough votes, RP Bench turns pause until a new model joins the pool.
Taking part
Taking part is optional. Accounts older than two days are asked once, on the home page, whether they want to join: Participate turns RP Bench on, and No Thanks (or closing the popup) turns it off. Nothing runs until you answer.
You can change your answer at any time in Gameplay Settings, under System Settings, with the RP Bench option. Guest accounts cannot take part.
How the scores work
- Scores are Elo-style ratings fitted from all counted votes at once (a Bradley–Terry fit, the same method as the LMArena leaderboard). A model that is picked over strong models scores higher than one picked over weak models.
- The scale is centred at 1200: an average model in the pool scores about 1200. A 100-point lead means the higher model is picked about 64% of the time when the two meet.
- The ± next to a score is its 95% range. When two models’ ranges overlap, the votes cannot tell them apart yet.
- Models with fewer than 100 counted votes are marked Provisional. Their scores can still move a lot.
- The leaderboard is refitted from every counted vote about once an hour.
- Which models meet is decided by the system. Models with fewer votes are matched more often, so a new model is rated quickly, and no single model takes more than about 40% of a day’s battles.
Frequently asked questions
What is RP Bench?
RP Bench is Mana’s benchmark for AI roleplay models. Instead of a fixed test, it compares models on real story turns: two anonymous models answer the same message, a player picks the better response, and the votes rank the models on a public leaderboard.
Does an RP Bench turn cost coins?
The narration model is free on an RP Bench turn. Other parts of the turn, such as character status updates and voices, cost what they normally cost.
Which models are on the leaderboard?
The models Mana has chosen to benchmark from its story model picker; the default Free and Paid models are never included. A model appears on the leaderboard once it has received votes; it is marked Provisional until it has 100 counted votes. A model Mana no longer offers or takes out of the benchmark is taken off the leaderboard, but its votes still count toward the scores of the models it met.
Can I see which model wrote each response before I vote?
No. The model names are hidden until after your pick, so every vote is blind. They are revealed right after you choose.
Does my vote change my story?
Yes. The response you pick becomes the turn and is saved to the story like any other response.
What if a response avoids the scene because of a safety filter?
Choose “Response A (or B) Broke Narrative Immersion due to safety filter” instead of picking. The story continues with the other response, and the flag counts toward that model’s Refusal Rate on the leaderboard. It is not a vote, so neither model’s score changes.
Why was my vote not counted?
A pick made in under 5 seconds continues the story but is not counted, since it cannot reflect a real read of both responses.
How often is the leaderboard updated?
The scores are refitted from every counted vote about once an hour. The leaderboard shows when it was last updated.
How do I turn RP Bench on or off?
Open Gameplay Settings, then System Settings, and use the RP Bench option. The change applies from your next turn.