File Downloader Library

What is being asked

A low-level design question. Write the design of a download library that an app can call with a URL and a destination, which fetches the file in parallel pieces, reports progress, and can pause and resume.

Requirements

Functional

• Download a file to disk; pause, resume, cancel.
• Progress callbacks; several downloads at once.
• Verify integrity.

Non-functional

• Fast on good networks, robust on bad ones.
• Bounded memory whatever the file size.
• Safe to call from several threads.

Design

1. A HEAD request reads the file's size and whether the server supports range requests.
2. If it does, the file is divided into chunks of a few megabytes. The destination file is pre-allocated to full size.
3. Workers from a bounded pool each request one chunk with a Range header and write it at its offset in the file.
4. A small state file records which chunks are complete. Resuming reads it and fetches only the rest.
5. A failed chunk is retried with exponential back-off. When all chunks are done, the checksum is verified and the temporary file is renamed into place.

Key decisions and trade-offs

• Chunk size and parallelism. More connections help until the server or network throttles; four to eight is typical. Small chunks waste less on a retry but add request overhead.
• Write directly to offsets. Streaming each chunk to its position keeps memory constant; buffering chunks in memory does not scale.
• No range support. Fall back to one sequential stream, and resume is then impossible.
• Validating resume. Use the ETag or Last-Modified header to check the remote file has not changed since the partial download.

Follow-up questions

• How do you report smooth progress when chunks finish out of order?
• How would you limit bandwidth across all downloads?
• What classes and interfaces would you expose, and what stays private?
• How do you make cancel take effect promptly on a blocked socket?