Staff Research Engineer/Scientist

• Design & Architect Data Infrastructure — Design and oversee deployment of data architecture and pipelines to capture, manage, and store structured and unstructured data from internal and external sources; establish processes and data flows using cloud, local databases, and applicable storage forms
• Build & Automate Data Transformation — Develop technical tools using ML and data-engineering techniques to cleanse, organize, and transform data; implement automated processes to maintain data structure integrity and quality standards on an ongoing basis
• Define Agentic Evaluation Metrics & Ground Truth — Partner with product and AI teams to define evaluation metrics for agentic workflows (task completeness, tool use, workflow success); establish ground truth labeling standards and validation criteria; design evaluation datasets that reflect real-world agent execution
• Build Agentic Evaluation Pipelines — Design and implement automated evaluation infrastructure that measures AI agent performance using Now LLM Service models and execution logs; create dashboards, reporting, versioning, and reproducibility of evaluation datasets and results
• Establish Standards & Continuous Improvement — Create design standards and quality assurance processes for data systems; define quality gates and validation frameworks; analyze workflow performance and recommend optimizations to accommodate evolving CRM AI requirements
• Lead Cross-Functional Collaboration — Collaborate with product, engineering, and data science teams; mentor junior engineers on data engineering and evaluation design; generalize evaluation patterns and metrics across Agentic AI products

Department: Engineering, Infrastructure and Operations
Function: Engineering
Experience Level: Not Applicable