P2P File Transfer (BitTorrent)

What is being asked

Design peer-to-peer file distribution: instead of everyone downloading from one server, a file is split into pieces and peers exchange pieces with each other, so capacity grows with the number of downloaders.

Requirements

Functional

• Publish a file; find peers that have it; download pieces from several peers at once.
• Verify every piece; resume interrupted downloads.

Non-functional

• Scales to huge audiences without a large central server.
• Tolerates peers joining and leaving constantly.
• A corrupted or malicious piece must be detected.

Design

1. The publisher splits the file into fixed-size pieces and creates a small metadata file with the piece size and a hash of every piece.
2. A new peer asks a tracker, or a distributed hash table shared among peers, for others exchanging the same file.
3. Peers tell each other which pieces they hold. A peer requests missing pieces from several peers in parallel.
4. Each received piece is checked against its hash from the metadata before it is saved and offered to others.

Key decisions and trade-offs

• Rarest piece first. Requesting the least available pieces first keeps every piece well replicated, so the file survives when the original publisher leaves.
• Incentives. A peer uploads preferentially to those that upload to it (tit-for-tat), with an occasional random slot so newcomers can start. Without this, most users only download.
• Tracker or hash table. A tracker is simple but central. A distributed hash table removes the central point at the cost of slower, less reliable peer discovery; real systems use both.
• Gossip. Peers exchange lists of other peers they know, which spreads membership without asking the tracker each time.
• Piece size. Small pieces verify and retry cheaply but enlarge the metadata.

Follow-up questions

• How does a peer behind a home router accept incoming connections?
• What happens at the very end, when a peer needs only the last few pieces?
• How would you verify pieces without downloading the whole metadata first?
• When would a content delivery network be a better choice than this?