Improving AI-Driven CV Refinement: Fixing Logic Compounding and Data Integrity
The Problem
In our devlog-ist/landing project, we use an AI-assisted tool to help users refine their CVs. Recently, we discovered two critical issues in how the system handled experience data during the refinement process.
First, the AI was hallucinating metrics—inventing percentages and performance numbers—to satisfy automated ATS (Applicant Tracking System) rules, even when the user was only looking for minor text refinements. Second, we had a data integrity bug where experience duration strings were being compounded; a value like '5 years' would become '5 years years' after a subsequent refinement run.
The Fix
To address the metric fabrication, we implemented a strict preservation policy within our generation logic. We now explicitly forbid the AI from adding or modifying quantitative metrics when the system is in refinement mode. For the compounding bug, we updated our formatting logic to sanitize input before appending duration units.
public function formatExperience(string $rawInput): string
{
// Extract numeric part to prevent 'years years' compounding
$years = (int) preg_replace('/[^0-9]/', '', $rawInput);
return "{$years} years";
}
public function applyRefinementRule(array $content): array
{
// Enforce strict preservation of metrics
if ($this->isRefinementMode()) {
$content['metrics'] = $this->originalMetrics;
}
return $content;
}
Key Technical Decisions
- State-Aware Guards: We introduced a specific refinement-mode flag that disables the AI's ability to inject new quantitative data.
- Sanitization over Blind Concatenation: Instead of trusting the input string, we now strip non-numeric characters before applying the 'years' suffix, ensuring idempotent outputs.
Lessons Learned
Never assume the AI will interpret a 'refinement' task strictly. When a language model is prompted to optimize content, it often over-optimizes by adding 'standard' features like metrics unless specifically constrained. Always implement a deterministic layer to sanitize outputs before they reach the user.
Takeaway
Before you run your next AI-generated content through a formatter, ensure you have a robust sanitization layer that strips potentially compounded data formats to keep your output clean and reliable.
Generated with Gitvlg.com