Resumable multipart uploader

Upload a 4 GB video in parts, a few at a time, and resume after a kill.

Part 1 of 5

Basics

The concepts this component is built on. With these in mind, every line of the solution has a reason.

1.1Key concepts

One idea per card, in the order the solution uses them.

01Multipart upload

A big file is split into fixed-size parts that upload separately; the server joins them at the end.

02ETag

A fingerprint the server returns for each uploaded part. The final 'complete' call lists every part's ETag to prove they all arrived.

03Checkpointing

After each part, save which parts are done. After a crash, the app reads the checkpoint and uploads only what is missing.

04Bounded concurrency

Upload a few parts at once for throughput, but not all of them, which would exhaust memory and the connection.

Part 2 of 5

Question

What exactly you are asked to build, what must be true when you are done, and what to ask before you start.

2.1Prompt

The question as the interviewer asks it, and the requirements to check your solution against.

Interviewer

Users upload long inspection videos on weak networks and the OS can kill the app at any time. Implement the uploader: parallel parts, limited concurrency, and resume without re-sending finished parts.

Requirements

  • Split into fixed-size parts
  • At most N parts in flight
  • Checkpoint each finished part
  • Resume uploads only missing parts
  • A failure loses only in-flight parts

2.2Clarify first

Questions worth asking, and the answers this interviewer gives.

You askInterviewer answers
Which storage API?S3-style multipart: a presigned URL per part, and the server returns an ETag per part.
Part size?8 MB: big enough for throughput, small enough to lose little on a kill.
Background?Run it from a WorkManager worker with setForeground, or a user-initiated data transfer job on Android 14+.
Part 3 of 5

Detailed answer

The design decisions, the complete Kotlin solution, and a walk through it one block at a time.

3.1Design decisions

Settle these before writing code; each one shows up in the solution.

The plan is the state

UploadPlan records completed parts and ETags; persist it and you can resume anywhere.

N workers, one channel

Remaining parts go into a closed Channel; N coroutines read from it. Concurrency can never exceed N.

Fail fast, resume cheaply

A failing part cancels its siblings through coroutineScope; the saved plan means the retry skips everything already done.

3.2Solution

Complete, compiling Kotlin with coroutines, built with warnings as errors and covered by tests/mobile/MobileTests.kt. The Java and TypeScript tabs do the same job with each language's own tools: Java compiled and run in our tests, TypeScript run in our tests.

KotlinJavaTypeScript
ResumableUploader.ktKotlinCopy
1import kotlinx.coroutines.channels.Channel2import kotlinx.coroutines.coroutineScope3import kotlinx.coroutines.launch4import kotlinx.coroutines.sync.Mutex5import kotlinx.coroutines.sync.withLock6 7/**8 * The resumable state of one multipart upload. Persist it after every part:9 * it is all you need to continue after the process is killed.10 */11data class UploadPlan(12  val fileId: String,13  val size: Int,14  val partSize: Int,15  val completed: Map<Int, String> = emptyMap(),        // part number → ETag16) {17  val partCount: Int get() = (size + partSize - 1) / partSize18  val remaining: List<Int> get() = (0 until partCount).filter { it !in completed }19  val isComplete: Boolean get() = completed.size == partCount20  fun byteRange(part: Int): IntRange = part * partSize until minOf(size, (part + 1) * partSize)21}22 23fun interface PartUploader {24  suspend fun upload(fileId: String, part: Int, bytes: ByteArray): String   // returns the ETag25}26 27fun interface CheckpointStore {28  suspend fun save(plan: UploadPlan)29}30 31/**32 * Uploads only the parts that are missing, at most [maxConcurrent] at a time, checkpointing33 * each one. A kill or a failure loses at most the parts that were in flight.34 */35class ResumableUploader(36  private val uploader: PartUploader,37  private val checkpoints: CheckpointStore,38  private val maxConcurrent: Int,39) {40  suspend fun upload(file: ByteArray, initial: UploadPlan): UploadPlan {41    require(file.size == initial.size) { "plan does not match the file" }42    var plan = initial43    val lock = Mutex()44    val parts = Channel<Int>(Channel.UNLIMITED)45    initial.remaining.forEach { parts.trySend(it) }46    parts.close()47    coroutineScope {                                   // one failing part cancels the others48      repeat(maxConcurrent) {                          // N workers share one queue of parts49        launch {50          for (part in parts) {51            val range = initial.byteRange(part)52            val bytes = file.copyOfRange(range.first, range.last + 1)53            val etag = uploader.upload(initial.fileId, part, bytes)54            lock.withLock {55              plan = plan.copy(completed = plan.completed + (part to etag))56              checkpoints.save(plan)                   // durable before this worker takes another part57            }58          }59        }60      }61    }62    return plan63  }64}
64 linesUTF-8Kotlin
ResumableUploader.javaJavaCopy
1import java.util.ArrayList;2import java.util.Arrays;3import java.util.HashMap;4import java.util.List;5import java.util.Map;6import java.util.concurrent.ConcurrentLinkedQueue;7import java.util.concurrent.ExecutorService;8import java.util.concurrent.Executors;9import java.util.concurrent.Future;10import java.util.concurrent.atomic.AtomicReference;11 12/**13 * Uploads only the parts that are missing, at most [maxConcurrent] at a time, checkpointing14 * each one. A kill or a failure loses at most the parts that were in flight.15 */16public final class ResumableUploader {17  /**18   * The resumable state of one multipart upload. Persist it after every part:19   * it is all you need to continue after the process is killed.20   */21  public record UploadPlan(String fileId, int size, int partSize, Map<Integer, String> completed) {   // part number → ETag22    public int partCount() { return (size + partSize - 1) / partSize; }23 24    public List<Integer> remaining() {25      List<Integer> out = new ArrayList<>();26      for (int i = 0; i < partCount(); i++) if (!completed.containsKey(i)) out.add(i);27      return out;28    }29 30    public boolean isComplete() { return completed.size() == partCount(); }31 32    /** {from, to}: the [from, to) byte range of one part. */33    public int[] byteRange(int part) { return new int[] {part * partSize, Math.min(size, (part + 1) * partSize)}; }34 35    UploadPlan withPart(int part, String etag) {36      Map<Integer, String> next = new HashMap<>(completed);37      next.put(part, etag);38      return new UploadPlan(fileId, size, partSize, Map.copyOf(next));39    }40  }41 42  @FunctionalInterface43  public interface PartUploader {44    String upload(String fileId, int part, byte[] bytes) throws Exception;   // returns the ETag45  }46 47  @FunctionalInterface48  public interface CheckpointStore {49    void save(UploadPlan plan) throws Exception;50  }51 52  private final PartUploader uploader;53  private final CheckpointStore checkpoints;54  private final int maxConcurrent;55 56  public ResumableUploader(PartUploader uploader, CheckpointStore checkpoints, int maxConcurrent) {57    this.uploader = uploader;58    this.checkpoints = checkpoints;59    this.maxConcurrent = maxConcurrent;60  }61 62  public UploadPlan upload(byte[] file, UploadPlan initial) throws Exception {63    if (file.length != initial.size()) throw new IllegalArgumentException("plan does not match the file");64    ConcurrentLinkedQueue<Integer> parts = new ConcurrentLinkedQueue<>(initial.remaining());   // one queue, N workers65    Object lock = new Object();66    UploadPlan[] plan = {initial};67    AtomicReference<Exception> failure = new AtomicReference<>();68    ExecutorService pool = Executors.newFixedThreadPool(maxConcurrent);69    try {70      List<Future<?>> workers = new ArrayList<>();71      for (int i = 0; i < maxConcurrent; i++) {72        workers.add(pool.submit(() -> {73          Integer part;74          while (failure.get() == null && (part = parts.poll()) != null) {75            try {76              int[] range = initial.byteRange(part);77              byte[] bytes = Arrays.copyOfRange(file, range[0], range[1]);78              String etag = uploader.upload(initial.fileId(), part, bytes);79              synchronized (lock) {80                plan[0] = plan[0].withPart(part, etag);81                checkpoints.save(plan[0]);                                  // durable before this worker takes another part82              }83            } catch (Exception e) {84              failure.compareAndSet(null, e);                               // one failing part stops the others from starting more85              return;86            }87          }88        }));89      }90      for (Future<?> worker : workers) worker.get();91    } finally {92      pool.shutdownNow();93    }94    if (failure.get() != null) throw failure.get();95    return plan[0];96  }97}
97 linesUTF-8Java
ResumableUploader.tsTypeScriptCopy
1/**2 * The resumable state of one multipart upload. Persist it after every part:3 * it is all you need to continue after the process is killed.4 */5export interface UploadPlan {6  fileId: string;7  size: number;8  partSize: number;9  completed: Record<number, string>;                   // part number → ETag10}11 12export const partCount = (p: UploadPlan) => Math.ceil(p.size / p.partSize);13export const remaining = (p: UploadPlan) => Array.from({ length: partCount(p) }, (_, i) => i).filter((i) => !(i in p.completed));14export const isComplete = (p: UploadPlan) => Object.keys(p.completed).length === partCount(p);15/** [from, to) byte range of one part. */16export const byteRange = (p: UploadPlan, part: number): [number, number] => [part * p.partSize, Math.min(p.size, (part + 1) * p.partSize)];17 18export interface PartUploader {19  upload(fileId: string, part: number, bytes: Uint8Array): Promise<string>;   // returns the ETag20}21 22export interface CheckpointStore {23  save(plan: UploadPlan): Promise<void>;24}25 26/** Runs one async section at a time, so checkpoints are saved in order. */27class Mutex {28  private tail: Promise<void> = Promise.resolve();29  async run<T>(section: () => Promise<T>): Promise<T> {30    const before = this.tail;31    let release!: () => void;32    this.tail = new Promise<void>((resolve) => (release = resolve));33    await before;34    try {35      return await section();36    } finally {37      release();38    }39  }40}41 42/**43 * Uploads only the parts that are missing, at most [maxConcurrent] at a time, checkpointing44 * each one. A kill or a failure loses at most the parts that were in flight.45 */46export class ResumableUploader {47  private uploader: PartUploader;48  private checkpoints: CheckpointStore;49  private maxConcurrent: number;50 51  constructor(uploader: PartUploader, checkpoints: CheckpointStore, maxConcurrent: number) {52    this.uploader = uploader;53    this.checkpoints = checkpoints;54    this.maxConcurrent = maxConcurrent;55  }56 57  async upload(file: Uint8Array, initial: UploadPlan): Promise<UploadPlan> {58    if (file.length !== initial.size) throw new Error("plan does not match the file");59    let plan: UploadPlan = { ...initial, completed: { ...initial.completed } };60    const parts = remaining(initial);                  // one queue of parts, shared by the workers61    const lock = new Mutex();62    let failure: unknown;63    let failed = false;64 65    const worker = async (): Promise<void> => {66      while (!failed) {67        const part = parts.shift();68        if (part === undefined) return;69        const [from, to] = byteRange(initial, part);70        try {71          const etag = await this.uploader.upload(initial.fileId, part, file.slice(from, to));72          await lock.run(async () => {73            plan = { ...plan, completed: { ...plan.completed, [part]: etag } };74            await this.checkpoints.save(plan);         // durable before this worker takes another part75          });76        } catch (e) {77          failed = true;                               // one failing part stops the others from starting more78          failure ??= e;79          return;80        }81      }82    };83 84    await Promise.all(Array.from({ length: this.maxConcurrent }, worker));85    if (failed) throw failure;86    return plan;87  }88}
88 linesUTF-8TypeScript

3.3Line-by-line walkthrough

The same code in the order you would explain it in the room. The line numbers follow the Kotlin file; the Java and TypeScript tabs above have the same steps in the same order.

1

A plan you can persist

Part count, remaining parts and byte ranges are all derived from size, part size and completed ETags.

ResumableUploader.ktKotlinCopy
11data class UploadPlan(12  val fileId: String,13  val size: Int,14  val partSize: Int,15  val completed: Map<Int, String> = emptyMap(),        // part number → ETag16) {17  val partCount: Int get() = (size + partSize - 1) / partSize18  val remaining: List<Int> get() = (0 until partCount).filter { it !in completed }19  val isComplete: Boolean get() = completed.size == partCount20  fun byteRange(part: Int): IntRange = part * partSize until minOf(size, (part + 1) * partSize)21}
lines 11–21UTF-8Kotlin
2

Only what is missing

Queue the remaining parts and close the channel: on resume, finished parts are never queued.

ResumableUploader.ktKotlinCopy
44    val parts = Channel<Int>(Channel.UNLIMITED)45    initial.remaining.forEach { parts.trySend(it) }46    parts.close()
lines 44–46UTF-8Kotlin
3

N workers

coroutineScope waits for all workers and cancels the rest if one throws. for (part in parts) ends when the channel is empty.

ResumableUploader.ktKotlinCopy
47    coroutineScope {                                   // one failing part cancels the others48      repeat(maxConcurrent) {                          // N workers share one queue of parts49        launch {50          for (part in parts) {
lines 47–50UTF-8Kotlin
4

One part, one request

Copy this part's byte range and upload it; the server returns its ETag.

ResumableUploader.ktKotlinCopy
51            val range = initial.byteRange(part)52            val bytes = file.copyOfRange(range.first, range.last + 1)53            val etag = uploader.upload(initial.fileId, part, bytes)
lines 51–53UTF-8Kotlin
5

Checkpoint, then take the next

Record the ETag and save the plan under a Mutex before this worker takes another part.

ResumableUploader.ktKotlinCopy
54            lock.withLock {55              plan = plan.copy(completed = plan.completed + (part to etag))56              checkpoints.save(plan)                   // durable before this worker takes another part57            }
lines 54–57UTF-8Kotlin

Complexity: O(parts) requests; at most N concurrent; one checkpoint per part

Part 4 of 5

Q&A

The questions interviewers ask after the code works. Answer out loud, then open.

4.1Follow-up questions

How does the server assemble it?
A complete-multipart call with the ordered ETags; verify a SHA-256 of the whole file.
Adaptive concurrency?
Measure throughput per part; grow N on fast Wi-Fi, shrink on cellular.
Real files are too big for a ByteArray
Stream each part from disk with RandomAccessFile and an OkHttp RequestBody, as in the transfer-manager deep dive.
Part 5 of 5

Extra depth

The cases that break naive solutions, how the same design looks on iOS, and what to practise next.

5.1Edge cases

Say these before the interviewer does.

Show detailsHide details
  • Last part is smaller than partSize
  • File changed between attempts (verify size and hash)
  • Presigned URLs expire on a long resume
  • Zero-byte file

5.2On iOS

The same design in Swift and Apple's frameworks.

Show detailsHide details
ApproachSwift / iOS
BackgroundA background URLSession uploadTask(fromFile:) per part, so uploads continue after the app is suspended.
ConcurrencywithThrowingTaskGroup: start N parts, and each completion starts the next.
CheckpointStore part ETags in SQLite after each part.