Staff Software Engineer, Observability & Profiling

Responsibilities

We are representing a confidential AI research organization seeking a Staff Software Engineer to join their Observability & Profiling team. In this role, you will:

• Design and build scalable telemetry ingest and storage pipelines for metrics, logs, traces, and error data across multi-cluster infrastructure
• Build observability solutions that give engineers deep, low-overhead visibility into system behavior across the fleet
• Own and evolve core observability platforms, driving migrations and architectural improvements that improve reliability, reduce cost, and scale with organizational growth
• Build instrumentation libraries, SDKs, and eBPF-based auto-instrumentation to emit high-quality telemetry
• Reduce mean time to detection and resolution by building cross-signal correlation—from kernel-level events up to application traces—unified query interfaces, and AI-assisted diagnostic tooling
• Drive fleet-wide efficiency by turning continuous profiling and utilization telemetry into actionable optimization insights across CPU, memory, and accelerator fleets
• Partner with research, inference, product, and infrastructure teams to ensure observability solutions meet organizational needs

What We're Looking For

Required qualifications:

• Hands-on experience building and operating large-scale observability or monitoring infrastructure
• Deep, hands-on experience with observability signals end to end—from instrumentation through ingest to query and analysis
• Understanding of high-throughput telemetry pipelines and the tradeoffs involved in collecting, storing, and querying operational data at scale
• Comfort digging below the application layer into the kernel, the network stack, or the hardware
• Excellent communication skills and enjoyment partnering with internal teams to improve their operational visibility and incident response capabilities
• Excitement about building foundational infrastructure and navigating ambiguous, high-impact technical challenges
• Bachelor's degree or equivalent combination of education, training, and experience in a relevant field

Preferred qualifications:

• 10+ years of relevant industry experience, including building and operating large-scale observability or monitoring infrastructure
• Experience building or operating eBPF-based observability in production—tracing, profiling, or network visibility
• Experience running continuous profiling at fleet scale, including managing overhead budgets and symbolization
• Kernel- and syscall-level debugging experience and performance engineering craft
• Experience profiling or instrumenting accelerator workloads
• Experience operating metrics systems at very high cardinality, or large-scale telemetry storage backends
• Experience with OpenTelemetry instrumentation, collector pipelines, and tail-based sampling strategies
• Interest in applying AI/LLMs to operational workflows such as automated root cause analysis, anomaly detection, or intelligent alerting

This is a hybrid role based in London, with an expectation to be in the office at least 25% of the time. Visa sponsorship may be available for qualified candidates.

How We Work with You

Our talent team at Get A Job.ai works closely with leading technology organizations to connect exceptional engineering talent with high-impact opportunities. When you apply through our platform, a recruiter will review your background and conduct an initial screening. If there's a strong match, we will submit your profile to our client for consideration. Please note that all applications must come through Get A Job.ai—do not contact the client directly.

Pay

This role offers competitive compensation. Pay details will be discussed during the screening process with our recruiting team.

Equal Employment Opportunity: Get A Job.ai is committed to fostering an inclusive recruitment process. We encourage applications from candidates of all backgrounds and experiences.