Length without extra prompt
Median reply 32 vs 427 characters. The adapter was shorter on 29 of 29 turns.
Character agents
Keep the character in the conversation.
If your character keeps slipping into assistant mode, we can test whether fine-tuning helps. We prepare examples with your reviewers, train an adapter, and compare it with your best prompt. Projects to tune conversational style typically cost $25,000-$80,000 and take 4-6 weeks.
What we measured in an earlier character-model test
We replayed 29 turns through the base and tuned model, each with and without the same 760-character persona prompt. These four runs form a 2x2 comparison. They measure reply patterns, not which model people prefer. Later turns can favor the adapter because the human originally responded to its replies.
Questions and lists
Questions 3 vs 27. Lists 0 vs 6.
Same 760-character prompt
Median 50 vs 71 characters. Questions 0 vs 14.
Remaining mistakes
The adapter still produced a blank reply, missed requests, and invented personal details. The style changes don't establish better factual reasoning.
Earlier 29-turn replay without an extra system prompt
Internal comparison, not a benchmark of the current Humanlike Chat step-576, 0.7-strength release. The prompt-on figures above show a smaller length gap. Neither condition is a blinded preference study.
| Metric | Base 27B | Speaker adapter |
|---|---|---|
| Median reply length | 427 characters | 32 characters |
| Questions asked | 27 | 3 |
| Lists | 6 | 0 |
| Shorter replies | Not applicable | 29 / 29 turns |
Agree what a good reply sounds like
Ask reviewers to approve example replies and explain character breaks, missed requests, or invented facts. Test fine-tuning when a good prompt still produces those mistakes. Keep changing facts in documents or tools the model can look up.
Define good replies
Review the preceding context, approve the next turn, and label character breaks, unsupported claims, and missed intent.
Compare against the prompt
Hold out conversations and judge the adapter against the best agreed prompt-only baseline. Use blinded review for preference claims.
Test your deployment setup
Check the prompt, chat template, adapter strength, and quantization in the software you plan to use. An adapter trained with a system prompt may still need it.
Check for drops in other capabilities
Earlier character-model versions scored lower on instruction-following and reasoning tests. These aren't results for the current public release or predictions for your model. Each trained model needs its own checks.
IFEval strict
About −5.4 percentage points on a merged speaker run versus its parent (81.33% → 75.97%).
MMLU-Pro-200
About −10.5 percentage points versus stock Qwen3.8-27B (73.0% → 62.5%, CI −17.0 to −4.5).
Test preference separately
To learn which replies people prefer, ask reviewers to compare them without knowing which model wrote them. Length and benchmark scores don't answer that question.
Compare retrieve vs train on the fine-tuning pageWhen prompting or retrieval is the better fix
What your team receives
Reviewed training data, the agreed model experiment, test results, and a deployment recommendation.
Comparison setup
Test your best agreed prompt and compare the base and tuned model on separate conversations. Record any limitations of replay tests.
Scoring rules
Check tone, character consistency, understood requests, invented claims, required disclosures, and serious failures. Reply length and lists describe style; they aren't quality scores.
Training decision
A rationale for prompting, retrieval, or an adapter, rather than assuming every behavior problem needs training.
Capability report
Report instruction-following and reasoning regressions alongside target-behavior scores when an adapter is trained.
Run instructions
The agreed adapter, software settings, and steps to reproduce the setup. The contract sets out ownership and the base model's license terms.
Free template
Get the speaker 2x2 rubric
Compare character consistency, disclosures, reply length, and serious failures on the same recorded messages. The template uses made-up examples, not client data.
This template uses made-up examples, not client data. Adapt it to your policies, privacy requirements, and approval process.
Who this is for
- Founders, CTOs, and ML leads building character, companion, and voice products
- Teams whose agent is often correct and still sounds like a generic assistant
- Teams with permission to use their conversations and reviewers who can approve training replies
Who this is not for
- Buyers seeking a turnkey consumer chat app or managed inference subscription
- Teams without permission to use their conversation data for the agreed work
- Projects seeking guaranteed preference, retention, or safety gains before evaluation
Speaker-model sprint ($25,000-$80,000, 4-6 weeks)
Audit
A conversation sample, behavior rubric, strongest prompt baseline, and a decision on the next experiment.
Dataset and adapter
Approved reply examples, separate test conversations, and an adapter for your agreed setup if training goes ahead.
Tests and handover
Side-by-side results, new failures, settings to run the model, and a recommendation to deploy, revise, or stop.
Get Humanlike Chat on Hugging Face
Download the model from Hugging Face under Apache-2.0. It shows one example of conversational fine-tuning. A custom project uses your own data and tests; it may produce different results.
Limitations
- Publishing or reusing client conversations or weights requires permission, even if aggregate results have been shared.
- Later turns in a recorded conversation can favor the adapter that originally wrote the replies. They don't establish whether people find it more human-like.
- A merged GGUF that loads can still behave differently from the training runtime.
- An adapter trained with a system prompt may still need that prompt when served.
- Shorter replies do not establish preference, retention, task success, or safety.
How we test changes
We review real conversations with your team and agree what the agent should do differently. Approved corrections can become training data, test cases, or examples the agent looks up. We compare each change with your best existing setup on separate test cases, then check for new failures in the software you plan to deploy.
References:
Common questions
What does a character-model dataset include?
Approved replies with the preceding conversation, source references, and review notes on tone, character consistency, missed requests, and serious failures. We keep test conversations separate from training.
Do you fine-tune our model?
If the initial review shows it's worth testing. We start with your best prompt, then test the adapter on separate conversations and check for new failures. Humanlike Chat is one example, not a guarantee of the same result.
Can we download your GGUF?
Yes. Download the model from Hugging Face using the link below. Current merged files use step-576 at 0.7 strength; the separate adjustable adapter is the older step-863 checkpoint. Custom project deliverables are agreed separately.
How is success measured?
Before training, we agree how to score character consistency, understood requests, invented claims, and serious failures. We also test the capabilities the product must keep. Length and question counts help describe replies; they don't prove preference. That needs a blind comparison. Download counts aren't a success measure for your model.
Related
Discuss the character you want your model to keep.
Bring conversations with personal details removed and your current prompt. We'll discuss the replies you want, the data available, and whether fine-tuning is worth testing.