Quality Lead - AI & Automation Auditing (BPO Contact Center)
Prefer a full account with a password? Sign Up to Apply
Job Description
AI QA Framework & Testing
• Operationalize and execute client-approved QA frameworks, evaluation rubrics, and test scripts tailored for AI chatbots, virtual agents, and copilot tools.
• Perform routine pre-release User Acceptance Testing (UAT) and regression testing across prompt deployments, Knowledge Base (KB) updates, and automated workflow releases within the operational environment.
Operational Auditing & Performance Monitoring
• Conduct daily audits of live AI-driven conversations, automated resolutions, and self-service interactions to detect intent misclassifications, system hallucinations, incorrect policy applications, or journey drops.
• Audit the handover bridge between AI and Human-Assist agents to evaluate context retention, CRM data pass-through accuracy, and agent handle times post-transfer.
Closed-Loop Vendor Feedback & Defect Analysis
• Perform root-cause defect analysis on resolution failures, escalation spikes, and recurring customer friction points.
• Synthesize audit data into actionable weekly reports; feed continuous improvement insights to Client AI Automation Leads, KB Managers, and Product Engineering for system retuning and sprint updates.
• Identify out-of-scope traveler queries and KB content gaps, recommending new intents and articles to safely broaden containment coverage.
Operational Metrics & SLA Compliance
• Measure, track, and report on primary operational and AI quality KPIs:
• Intent Recognition Accuracy
• Resolution Precision & Answer Accuracy
• Containment Rate & Defect Rate
• Handover Context Retention Accuracy
• Traveler Sentiment Score (TSS) / CSAT Impact
• Hold operational teams and frontline agents accountable for adhering to updated AI copilot workflows and knowledge standards.
Requirements
Function: Quality Assurance
Experience Level: Associate