Ground Truth Ops Specialist

At Moody's, we unite the brightest minds to turn today’s risks into tomorrow’s opportunities. We do this by striving to create an inclusive environment where everyone feels welcome to be who they are—with the freedom to exchange ideas, think innovatively, and listen to each other and customers in meaningful ways. Moody’s is transforming how the world sees risk. As a global leader in ratings and integrated risk assessment, we’re advancing AI to move from insight to action—enabling intelligence that not only understands complexity but responds to it. We decode risk to unlock opportunity, helping our clients navigate uncertainty with clarity, speed, and confidence.

If you are excited about this opportunity but do not meet every single requirement, please apply! You still may be a great fit for this role or other open roles. We are seeking candidates who model our values: invest in every relationship, lead with curiosity, champion diverse perspectives, turn inputs into actions, and uphold trust through integrity.

Skills and Competencies

• Experience running or supporting AI data labeling and annotation efforts, including campaign management, QA, or vendor coordination
• Hands-on familiarity with professional AI labeling platforms such as Labelbox, Dataloop, Scale, or similar tools
• Strong analytical mindset with the ability to interpret label distributions, consensus metrics, and error patterns
• Working knowledge of Python for analysis and automation, including use of Jupyter notebooks
• Comfort working with spreadsheets and basic analysis using SQL and/or Python
• Strong operational and project management skills, with the ability to manage multiple concurrent campaigns
• High attention to detail and rigor in documentation, taxonomy definition, and QA processes
• Clear written and verbal communication skills for collaboration with technical teams and external vendors
• Ability to give and receive feedback constructively to improve labeling instructions and outcomes
• Exposure to computer vision, geospatial data, or ML workflows is an advantage

Education

• Master's degree in a quantitative, analytical, or technical field

Responsibilities

This role owns the end-to-end lifecycle of ground truth data collection campaigns, translating machine learning needs into high-quality labeled datasets that power core computer vision products.

• Partner with ML and Data Science leads to translate model requirements into clear label taxonomies and concrete labeling tasks
• Design and run internal gold-standard campaigns to validate taxonomies and ensure coverage of edge cases
• Configure and manage annotation projects, including uploading imagery, geometries, and metadata, and exporting labeled data
• Coordinate with external labeling vendors, including taxonomy training, example walkthroughs, and ongoing feedback
• Evaluate vendor performance using gold datasets, confusion matrices, and quality reports to determine readiness for scale
• Launch and monitor production labeling campaigns, tracking throughput, SLAs, and overall progress
• Triage worker questions, resolve taxonomy ambiguities, and proactively identify tooling or workflow issues
• Monitor label quality and consensus metrics, initiating additional vote rounds or manual QA where needed
• Review low-agreement areas of interest and propose taxonomy refinements, clarifications, or deprecations
• Build final ground truth datasets, ensuring quality thresholds are met and data is clearly documented for ML consumption
• Maintain campaign status and documentation so stakeholders have clear visibility into data readiness
• Contribute to continuous improvement of ground truth processes, tooling, and best practices

About the Team

You will join Cape by Moody’s, a team focused on delivering highly precise property insights derived from aerial imagery and advanced machine learning. Ground truth data is central to the team’s mission, and this role works closely with data scientists, ML engineers, and product partners across Europe and North America in a collaborative, remote-friendly environment.

Moody’s is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, protected veteran status, sexual orientation, gender expression, gender identity or any other characteristic protected by law.

Candidates for Moody's Corporation may be asked to disclose securities holdings pursuant to Moody’s Policy for Securities Trading and the requirements of the position. Employment is contingent upon compliance with the Policy, including remediation of positions in those holdings as necessary.