Senior Data Scientist - ML Operations & Analytics ›

Senior Data Scientist - ML Operations & Analytics Filevine is a Legal AI company delivering Legal Operating Intelligence for the future of legal work. Grounded in a singular system of truth, Filevine brings together data, documents, workflows, and teams into one unified platform—where modern legal work happens with clarity and consistency. Powered by LOIS, the Legal Operating Intelligence System, Filevine connects context across every matter to transform legal operations from reactive to proactive. LOIS reads, understands, and reasons across your data to surface insight, automate complexity, and give professionals the clarity and confidence to see more, know more, and do more. Fueled by a team of exceptional collaborators and innovators, Filevine’s rapid growth has earned AI awards and recognition from Deloitte and Inc. as one of the most innovative and fastest-growing technology companies in the country. Role Summary: We're a team of driven, enthusiastic problem solvers with strong backgrounds in machine learning, engineering, product management, legal and operations, on a mission to help attorneys resolve cases faster, for better outcomes. With two established ML teams pushing boundaries in transcription, NLP and GenAI applications, we're now building a critical analytics layer to ensure operational excellence, quality assurance, and continuous improvement across all our AI systems. Responsibilities Build ML operations analytics tracking pipeline health, costs, latency, and quality metrics across all services Lead production LLM evaluation initiatives, designing custom metrics, quality monitoring and LLM observability in general Drive development efforts in collaboration with different ML teams based on analytics outputs Collaborate with our data annotation team on prompt engineering and evaluation dataset creation Help with curating the Filevine dataset Create actionable dashboards and insights for ML teams and leadership Help define SLAs, cost optimization and failure prevention strategies for our ML pipelines Qualifications 3+ years in data science, ML operations, or analytics engineering Experience with LLM evaluation frameworks and quality metrics design Strong Python, SQL, and data visualization skills Knowledge of prompt optimization techniques Deep understanding of latest trends in LLMs and their production deployments Excellent communication skills in English to translate technical metrics into business insights Collaborative mindset with ability to work across machine learning teams