Observability Engineer

Job description

Overview About the Role & Team IDC is building the next generation of AI-powered intelligence platforms that transform how technology decisions get made. Our platform re-imagines the way decision-makers discover and interact with trusted research and data, and is foundational to IDC's future.We are looking for an Observability Engineer to join the Quality & Evaluation team. You will build and maintain the observability platform that gives the team visibility into AI system performance, cost, and quality. Working under the direction of the senior evaluation engineer, you will build the dashboards, alerting systems, and experimentation infrastructure that enable the team to understand and improve the platform's AI pipeline.Your work makes it possible for everyone on the team to see what the system is doing, how much it costs, and whether it is getting better or worse. Without your contributions, the team is flying blind. What You’ll Do - Build and maintain the observability platform that provides visibility into AI system performance, cost, and quality - Build dashboards that track key operational metrics: response latency, LLM token usage, cost per query, and error rates across the...