{"id":647,"date":"2026-10-01T14:10:00","date_gmt":"2026-10-01T12:10:00","guid":{"rendered":"https:\/\/grindloop.io\/blog\/?p=647"},"modified":"2026-09-27T19:29:50","modified_gmt":"2026-09-27T17:29:50","slug":"offline-sync-duplicates-after-retry","status":"publish","type":"post","link":"https:\/\/grindloop.ai\/blog\/offline-sync-duplicates-after-retry\/","title":{"rendered":"Why Offline Sync Duplicates Records After a Retry"},"content":{"rendered":"

Offline sync duplicates records because a retried POST<\/code> can’t tell the server it’s a retry. The server saves the note. Then the connection drops before the response arrives. The worker sees an IOException<\/code> and returns Result.retry()<\/code>. WorkManager replays the same POST<\/code>. The server saves a second copy. RFC 9110 defines POST<\/code> as non-idempotent for exactly this reason. Most candidates reach for an idempotency key next. That’s the right tool. The second bug hides in where the key gets created. If the key is generated inside the worker or an OkHttp interceptor, every attempt carries a brand-new key. The server sees two unrelated requests and saves both. The fix is to generate the key once, when the note is written locally. Store it in the outbox row. Send that same key on every attempt. The server returns the stored result when it sees the key again.<\/p>\n\n

The interviewer asks for offline note-taking in an Android app. Users write notes on a train with no signal. The notes sync later. The candidate draws a Room table, a WorkManager sync worker and a POST \/notes<\/code> call. Then the follow-up lands. “A user on a bad connection reports every note showing up twice on the web app. Where do the duplicates come from?”<\/p><\/blockquote>\n\n

How offline sync duplicates a note the user saved once<\/h2>\n\n

This is the shape most whiteboards end up with. It follows the lazy-write strategy from the Android offline-first guide<\/a>. Write locally first, then queue the network write.<\/p>\n\n\n

@Entity(tableName = \"outbox\")\ndata class OutboxNote(\n    @PrimaryKey(autoGenerate = true) val localId: Long = 0,\n    val text: String,\n    val createdAt: Long\n)\n\ninterface NotesApi {\n    @POST(\"notes\")\n    suspend fun createNote(@Body body: NoteDto): NoteDto\n}\n\nclass SyncWorker(\n    context: Context,\n    params: WorkerParameters,\n    private val outbox: OutboxDao,\n    private val api: NotesApi\n) : CoroutineWorker(context, params) {\n\n    override suspend fun doWork(): Result {\n        for (note in outbox.pending()) {\n            try {\n                api.createNote(NoteDto(note.text, note.createdAt))\n                outbox.delete(note)\n            } catch (e: IOException) {\n                return Result.retry()\n            }\n        }\n        return Result.success()\n    }\n}<\/pre>\n\n\n
Every piece here is reasonable on its own. The note survives process death because it’s in Room. The worker only deletes a row after the server accepts it. On failure it asks WorkManager to try again. The WorkManager retry docs<\/a> say the default backoff is EXPONENTIAL<\/code> with a 30-second delay. The minimum is 10 seconds. The offline-first guide covers queuing, backoff and conflict resolution in detail. It doesn’t discuss what a replayed write does on the server. The duplicate comes from that replay.<\/p>\n\n
Bug 1: an IOException doesn’t mean the server did nothing<\/h2>\n\n
A failed request has two very different causes that look identical on the client. The request may never have reached the server. Or the server may have committed the write and the response got lost on the way back. On a train, the second case is easy to hit. The client can’t distinguish them. It only sees the exception.<\/p>\n\n
RFC 9110, section 9.2.2<\/a> (June 2022) is the rule behind this. Of the methods it defines, only PUT<\/code>, DELETE<\/code> and the safe methods are idempotent. Those can be repeated after a communication failure because the effect is the same. POST<\/code> isn’t on that list. The RFC says a client “SHOULD NOT automatically retry a request with a non-idempotent method.” The exception is a client with some way to know the retry is safe. Result.retry()<\/code> is an automatic retry of a POST<\/code> with no such way.<\/p>\n\n
There’s a quieter retry underneath too. OkHttp’s retryOnConnectionFailure<\/code><\/a> is on by default. Its docs say the client “silently recovers” from stale pooled connections, among other problems. The logic lives in RetryAndFollowUpInterceptor<\/code><\/a>. Among its checks, it refuses to replay a request whose body is one-shot. None of the checks look at the HTTP method. So a POST<\/code> can be sent twice before your worker ever sees an exception. The same OkHttp docs suggest turning it off “when doing so is destructive.” That removes one retry layer. It does nothing about the worker’s own retry after a lost response.<\/p>\n\n
Bug 2: a fresh UUID per attempt is still a new request<\/h2>\n\n
The textbook fix is an Idempotency-Key<\/code> header. The IETF httpapi working group’s Idempotency-Key draft<\/a> describes it as a way to make POST<\/code> fault-tolerant. The draft is an expired Internet-Draft. Its latest version, -07, is from October 2025. It isn’t an RFC. The design is still the common pattern. The client sends a unique value. The server stores it with the result and recognizes later retries of the same request.<\/p>\n\n
Under interview pressure, the key often gets added in the wrong place.<\/p>\n\n\n
\/\/ Looks like the fix. Isn't.\nclass IdempotencyInterceptor : Interceptor {\n    override fun intercept(chain: Interceptor.Chain): Response {\n        val request = chain.request().newBuilder()\n            .header(\"Idempotency-Key\", UUID.randomUUID().toString())\n            .build()\n        return chain.proceed(request)\n    }\n}<\/pre>\n\n\n
The interceptor runs once per call. Each Result.retry()<\/code> leads to a new doWork()<\/code> run. That run makes a new call and gets a new UUID. Generating the key at the top of doWork()<\/code> has the same flaw. The server commits under key A. The response is lost. The retry arrives with key B. Key B has never been seen, so the server saves the note again. The header is present but deduplicates nothing. The draft’s own rule is that a key must not be reused for a different request. The reverse holds for retries. A retry of the same request has to carry the same key.<\/p>\n\n
The failure, reproduced in plain Kotlin<\/h2>\n\n
This program replaces Room, WorkManager and the network with small fakes. The fake network commits on the server and then throws on the first call, like a dropped connection. The loop stands in for a worker returning Result.retry()<\/code>. It compiles and runs on Kotlin 2.4.20.<\/p>\n\n\n
import java.io.IOException\nimport java.util.UUID\n\ndata class OutboxRow(val localId: Long, val text: String, val idempotencyKey: String? = null)\n\nclass FakeServer {\n    val notes = mutableListOf<String>()\n    private val seenKeys = mutableMapOf<String, Int>() \/\/ key -> stored note id\n\n    fun createNote(text: String, key: String?): Int {\n        if (key != null) seenKeys[key]?.let { return it } \/\/ replay: return stored result\n        notes += text\n        val id = notes.size\n        if (key != null) seenKeys[key] = id\n        return id\n    }\n}\n\n\/\/ Commits on the server, then \"loses\" the first response, like a dropped connection.\nclass FlakyNetwork(private val server: FakeServer) {\n    private var calls = 0\n    fun post(text: String, key: String?): Int {\n        val id = server.createNote(text, key)\n        if (++calls == 1) throw IOException(\"connection reset before response\")\n        return id\n    }\n}\n\n\/\/ Stand-in for a CoroutineWorker that returns Result.retry() on IOException.\nfun drain(row: OutboxRow, keyFor: (OutboxRow) -> String?): FakeServer {\n    val server = FakeServer()\n    val network = FlakyNetwork(server)\n    repeat(3) { attempt ->\n        try {\n            network.post(row.text, keyFor(row))\n            return server\n        } catch (e: IOException) {\n            println(\"  attempt ${attempt + 1}: $e -> Result.retry()\")\n        }\n    }\n    return server\n}\n\nfun main() {\n    val row = OutboxRow(localId = 1, text = \"Standup notes\")\n\n    println(\"1: no key\")\n    println(\"  server notes: ${drain(row) { null }.notes}\")\n\n    println(\"2: key generated per attempt\")\n    println(\"  server notes: ${drain(row) { UUID.randomUUID().toString() }.notes}\")\n\n    println(\"3: key stored in the outbox row at write time\")\n    val stored = row.copy(idempotencyKey = UUID.randomUUID().toString())\n    println(\"  server notes: ${drain(stored) { it.idempotencyKey }.notes}\")\n}<\/pre>\n\n\n\n
1: no key\n  attempt 1: java.io.IOException: connection reset before response -> Result.retry()\n  server notes: [Standup notes, Standup notes]\n2: key generated per attempt\n  attempt 1: java.io.IOException: connection reset before response -> Result.retry()\n  server notes: [Standup notes, Standup notes]\n3: key stored in the outbox row at write time\n  attempt 1: java.io.IOException: connection reset before response -> Result.retry()\n  server notes: [Standup notes]<\/pre>\n\n\n
Cases 1 and 2 produce the same duplicate. Adding a header changed nothing because the key changed with the attempt. Only case 3 keeps one note. The retry carries the key the server already stored. So the server returns the saved result instead of writing again.<\/p>\n\n
The fix: mint the key when the note is written, not when it’s sent<\/h2>\n\n\n
@Entity(tableName = \"outbox\")\ndata class OutboxNote(\n    @PrimaryKey(autoGenerate = true) val localId: Long = 0,\n    val text: String,\n    val createdAt: Long,\n    val idempotencyKey: String = UUID.randomUUID().toString() \/\/ set once, at insert\n)\n\ninterface NotesApi {\n    @POST(\"notes\")\n    suspend fun createNote(\n        @Header(\"Idempotency-Key\") key: String,\n        @Body body: NoteDto\n    ): NoteDto\n}\n\n\/\/ In SyncWorker.doWork()\nfor (note in outbox.pending()) {\n    try {\n        api.createNote(note.idempotencyKey, NoteDto(note.text, note.createdAt))\n        outbox.delete(note)\n    } catch (e: IOException) {\n        return Result.retry()\n    }\n}<\/pre>\n\n\n
The key now belongs to the note, not to the HTTP call. It’s written to Room in the same insert as the text. So it survives process death, a WorkManager reschedule and an app update. Every attempt for that row sends the same value. A client-generated note ID works the same way if the API accepts one. The server can then treat the create as an upsert on that ID.<\/p>\n\n
The client half only works if the server holds up its end. The server has to store each key with its result for some retention window. It returns that stored result when the key comes back. The draft also covers the in-flight case. A retry that arrives while the first request is still processing should get a 409<\/code>. A key reused with a different payload should get a 422<\/code>. In an interview, say out loud that this is a contract with the backend.<\/p>\n\n
Two smaller details stop the fix from leaking. Delete the outbox row only after a success response. The worker above already does that. And enqueue the sync as unique work. The offline-first guide’s sample uses ExistingWorkPolicy.KEEP<\/code> so only one sync worker runs at a time. The WorkManager unique-work docs<\/a> say REPLACE<\/code> cancels the existing work. A worker cancelled mid-request may already have reached the server. The new worker then sends the same rows again. With a stored key, that resend is harmless. Without one, it’s another duplicate.<\/p>\n\n
What a strong system design answer covers here<\/h2>\n\n
    \n
  1. Separate “the request failed” from “the client didn’t get a response.” Say that a lost response after a commit can happen on any flaky mobile network.<\/li>\n
  2. Name POST<\/code> as non-idempotent. Point out that there are two retry layers, WorkManager’s Result.retry()<\/code> and OkHttp’s silent connection retry.<\/li>\n
  3. Propose an idempotency key or a client-generated ID. Then say where it’s created. It’s made at local write time and persisted in the outbox row.<\/li>\n
  4. Describe the server side. It stores each key with its result. It replays that result for a repeat key, within a retention window.<\/li>\n
  5. Mention unique work with KEEP<\/code> and deleting rows only after success. Those stop a second worker from racing the first.<\/li>\n<\/ol>\n\n
    Scoping this in time is its own skill. Hass covers that side in what the senior Android system design round tests<\/a>. This post is the technical depth for one of its most common follow-ups.<\/p>\n\n
    Common wrong answers:<\/strong><\/p>\n