Subodh Latkar
BACKEND ENGINEER · CLICKPOST · DISTRIBUTED SYSTEMS

I build distributed systems.

I’m a backend engineer at Clickpost, working on the service that predicts delivery dates. Most of the work is performance and reliability: making APIs fast, keeping data consistent, and finding out why something slowed down.

Outside work I’m building BeeDB, a key–value store with a Raft implementation I wrote by hand. I measure what it does, break it on purpose, and write down what happened.

GitHub ↗ BeeDB source ↗ Email ↗
Currently
  • buildingBeeDB's public deploy
  • shippingA live demo, early Oct 2026
  • writingBuilding BeeDB, 8 posts
  • next readRaft dissertation, ch. 6
  • at workDelivery-date prediction

Selected work

01
JAVA 21 · RAFT · WRITE-AHEAD LOG · MEMCACHED PROTOCOL

BeeDB

A distributed key-value store built to understand consensus by implementing it: leader election, log replication, commit rules, snapshots, log compaction, and a CRC-framed write-ahead log that has to survive the machine dying mid-write. It speaks the memcached text protocol, so any existing memcached client can talk to it.

The interesting part is not the API. It is being able to say exactly what the cluster does when a write arrives, or when the leader disappears halfway through one, and having a test that proves it.

Demo, stable: bee-2 is queen in term 25, all three nodes online, logs in step. Demo, kill announced: a notice says bee-2 goes down in 10 seconds. Demo, bee-2 down: bee-1 is the new queen in term 26, two of three nodes online. Demo, bee-2 still down: its log track stops six entries behind the others. Demo, recovered: bee-2 is back as a worker, three of three online, lag zero.

1/5Stable. bee-2 is queen in term 25; a demo writer commits one entry a second, and all three logs keep step.

Recorded from the BeeDB demo page against the real three-node cluster, during one kill round. The demo calls the leader the queen and the followers workers. Step through it with the arrows or the ← → keys.

Source ↗ What I got wrong What’s next
IIT Bombay

Key-value store over gRPC

A C++ server that holds keys in memory and answers concurrent GET, PUT and DELETE over gRPC at about 10,000 operations a second, with switchable LRU and LFU eviction so the two policies could be measured against each other rather than argued about.
M.Tech thesis

BPMN to Petri nets

A Java engine that turns business process models into executable Petri nets, with a simulation view, so a workflow can be checked for deadlock and reachability instead of reviewed by eye.
Operating systems

A shell for xv6

Pipes, redirection and job control written in C, inside the xv6 teaching kernel. Small, and the fastest way I know to stop treating fork and exec as magic.

Writing

02

Building BeeDB, from a coding challenge to three machines that argue about who is in charge. Eight posts, no prior knowledge of Raft assumed.

Post 01

Why I built a database from scratch

A coding challenge, a crush on Scylla, and the moment I wanted the harder promise.
Post 02

One computer isn’t enough

Phone numbers, a notebook, and every wall you hit on the way to a distributed database.
Post 03

Certainty in uncertainty

How three machines agree on one story, and why the answer involves rolling dice.
Post 04

Following one write through BeeDB

One set command, from the socket to the disk to the word STORED.
Deep dive

BeeDB: the architecture

The whole system for people who build these: the thread map, the commit rule, the gateway’s three layers, what it does when you break it, and what I would do differently.
All nine pieces →

Things I got wrong

03

Five things I believed until a measurement said otherwise.

Durability

A majority acknowledgement means the write is safe.

Not if one of that majority is the leader’s own memory. I killed the leader between acknowledging the write and its own fsync, and the write was gone. Only counting copies that are on disk cost me 19% of write throughput, and I kept the cost.

Observability

Logging is free.

About 40,000 lines a minute went to the same disk as the write-ahead log. The fsyncs queued behind the logs, heartbeats went out late, and the cluster started holding elections nobody needed. The thing I added to watch the system was the thing breaking it.

Testing

My durability test proves durability.

It kills nodes inside one process, so anything sitting in the page cache survives and looks durable. The suite passes even with force(true) deleted. It does prove nothing is lost to truncation or commit-index bugs, which is worth having, but that is not what I was claiming.

Failover

The failover probe lost 21 writes.

All 1,000 keys were there. The timed-out writes had landed after all, and my retries applied them a second time: the opposite failure to the one I was looking for. That is how I learned I had no request deduplication.

Determinism

A TTL is a fact I can put in the log.

It is not. “Expires in two seconds” means something different every time it is read, so a node replaying its log handed old keys a fresh lease: one node reported 60 items where the others had 59, and served a key the leader had already dropped. The node that receives the write now converts it once, and the log carries an absolute deadline every node stores as it is.

Experience

04
JUL 2024 –
PRESENT

Software Engineer, Backend

Clickpost · delivery-date prediction

  • Own the delivery-date platform end to end: prediction APIs, analytics and simulation services behind delivery promises for 100+ brands, at roughly 8M requests a day.
  • Took the serving path’s p99 from about a second to under 100 ms, with a Redis layer, Kafka-backed asynchronous writes off the hot SLA table, and query-plan work.
  • Led the rollout of a 3-node ScyllaDB cluster for uploaded SLA matrices, designing the partition keys and versioned config pointers that keep 2 to 3 million delivery events a day flowing.
AUG 2023 –
JUL 2024

Software Engineer, Android Platform

OnePlus R&D · certification and test automation

  • Automated Google’s Android certification suites with a Python orchestration framework covering scheduling, device allocation and result parsing, cutting QA execution time by about 75%.
JUL 2019 –
OCT 2020

Associate Software Engineer

Tata Consultancy Services · banking workflows

  • Built banking workflow interfaces for a global client, and the test harnesses that simulated a thousand or so cross-border transactions to find the patterns behind recurring defects.

About

05

I’m particularly interested in consensus algorithms, storage engines and the principles behind building fault-tolerant systems. I enjoy learning by building, questioning how things work, and turning complex engineering concepts into practical solutions.

Contact

06

If you'd like to talk about distributed systems, storage, or something on this page, send me an email. If you've found a mistake here, that's the most useful email you could send, it'll end up in section 02.