Mingyue Liao 廖明月

Context

A RAG product that generates medical literature reviews (two-way recall → rerank → outline → body). I owned generation. Reviews are long answers where every conclusion must trace back to a reference, so manual review was expensive.

Challenge

• Hallucination is broadly defined. Errors like “unsound reasoning” can't be captured by rules or prompts, and few-shot examples hit context limits.
• GPT and Gemini as teachers didn't exceed our own ceiling, so distillation was a dead end.
• The judge must output its reasoning and evidence, not just a label, to be explainable and iterable.
• Negative samples lacked diversity, so models only caught error types they had seen.

Why GRPO, not SFT / DPO

• Detection has verifiable answers and cheap feedback, the same conditions under which RL worked for math and code.
• SFT and DPO teach reasoning only through written samples, capping the model at human reasoning; GRPO lets it explore.
• The team's GRPO framework already beat SFT on GSM8K; PPO's longer feedback chain didn't train well.

Approach

1. Error attribution into three classes: missing premise, unsound reasoning, misattribution.
2. Task split: segment long answers into referenced claims, judged one by one.
3. Fully synthetic negatives: Qwen 72B rewrites correct text into a target error type; existing detectors filter out what they already catch, keeping hard cases. Three rounds.
4. Training: GRPO on Qwen 14B, reward = correctness + format.
5. Evaluation on 1,000+ human-labeled claims.

Results

• +7% F1 vs. RefChecker, +10%+ vs. SFT baseline (relative).
• The judge then improved the generator (relative gains on human-rated expertise score): self-reward alignment +10% with claim-level hallucination rate down ~50%; judge-built preference data for DPO +5%; bilingual terminology dictionary +5%; RAFT noise-robust training +4%.

My role

Owned the judge end to end: error attribution, data synthesis, training, evaluation. Owned alignment for the generation stage of the RAG pipeline.

Context

Identify, from news reports, whether a loan applicant has criminal records or is a sanctioned party. The prompt kept growing with every edge case and quality dropped noticeably. The team had no fine-tuning experience.

Challenge

• It's entity recognition plus relation judgment: is the person in the article the applicant, and does the event count as risk? Same names, aliases and mis-linked relations all cause errors.
• No eval set, so nobody knew whether a new rule broke other cases.
• No training data and inconsistent labeling guidelines.

Approach & results

1. Error taxonomy first: inconsistent labels, uncovered data, mis-linked relations, missing relations, entity-boundary errors; their distribution set the order of work.
2. Unified labeling guidelines: +1.2%
3. Synthetic data for coverage: +1.5%
4. Cross-constructed preference pairs over the full dataset for DPO, targeting error-prone cases: +3.5%
5. A repeatable loop: expand eval → diversify data → build pairs → SFT → DPO → evaluate.

My role

Error taxonomy, labeling guidelines, data synthesis, SFT + DPO training, evaluation system.