Senior Kafka Reliability & Performance Engineer — Freelance (1 month, renewable)

About Lenstra

Lenstra was founded by engineers with a passion for Computer Science and a track record of delivering high-quality solutions. We work at the intersection of Cloud Computing, DevOps, Data Platforms, and IT Security, helping top-tier clients in Banking, Insurance, Luxury, and Tech solve their hardest technical challenges. Our team takes a holistic approach to each client's environment, building solutions that are as elegant as they are impactful.

For this mission, we're staffing a diagnostic engagement for a regulated player in financial data / quantitative finance. The client's identity is confidential and will be shared once you're through the process.

About the mission

🔍 You will run a diagnostic-only mandate on a production Apache Kafka platform — open-source distribution, not Confluent.

⚙️ You will investigate a stack where clients are Python and C++ on top of librdkafka. The Java client and Kafka Streams are not in play here, so JVM-client experience alone won't cut it.

📉 You will chase down the root causes behind intermittent event loss, unexplained tail-latency spikes, and weak incident detection — from broker and JVM/GC level down into TCP, NIC, and clock-discipline layers.

📋 You will deliver a prioritised remediation plan and an observability framework the internal team can execute on. Implementation itself is out of scope for this mission and would be contracted separately if the client decides to pursue it.

🔒 You will work on a named-individual basis, at high allocation, in a regulated environment. Confidentiality applies before, during, and after the mission.

What we look for:

• Mindset and working style
• Evidence-driven. You instrument first, hypothesise broadly, and eliminate on data — not on hunches.
• Precision-minded. This is a diagnostic mandate, not a build. The deliverable is a plan someone else executes, so it has to be exact and unambiguous.
• Confidentiality-first. You operate comfortably under strict confidentiality in a regulated environment, without needing hand-holding on what can and can't be discussed.
• Autonomy-first. You're staffed as a named individual at high allocation — this is not a shared or junior-supported role.

Technical requirements (must-have):

• 10+ years in distributed systems, with 5+ years deep on Apache Kafka in production under reliability and latency constraints.
• Hands-on librdkafka experience in Python and/or C++ — this is the sharpest filter on this mission; JVM-client-only experience will not qualify.
• Broker tuning including JVM and GC (G1 or ZGC), heap vs page-cache sizing, and ISR/replication behaviour under stress.
• Offset reconciliation or equivalent end-to-end loss-investigation experience.
• Prometheus / Grafana observability build experience — exporters, dashboards, alerts — not just dashboard consumption.
• Financial services, trading, telecoms, or other low-latency / regulated domain experience.
• PTP (IEEE 1588) deployment experience, not just NTP, with an understanding of clock skew as a source of measurement-artefact latency.
• TCP and NIC-level diagnostic capability: tcpdump/tshark, NIC tuning, kernel page cache.
• Serialisation discipline: Avro, Protobuf, or custom binary, including schema-evolution scars and familiarity with silent-loss patterns through decode-failure paths.
• Prior diagnostic-mandate experience (vs build/run), with a track record of deliverables an internal team could actually execute.

Nice to have:

• eBPF / bpftrace for production-safe tracing.
• Experience with both KRaft and ZooKeeper deployments, including controller failure-mode experience.

What we offer

• Day rate from €700 to €900 — based on seniority and interview performance, no surprises. One-month freelance mission with a clear possibility to extend based on mutual fit and mission scope.
• Location: Paris, hybrid, 3 days onsite
• Contract: Freelance / independent contractor — 1 month, with possibility to extend
• Working language: English and French