Great question—let's unpack this clearly, because it's a common point of confusion when working with Hadoop MapReduce!

First, let's recap the default behavior: when you use the standard HashPartitioner, each reducer gets a subset of keys (based on hash value), and each reducer's output is sorted by key internally. But here's the catch: there's no guarantee of order across different reducers. For example, reducer 1's largest key could be smaller than reducer 2's smallest key. So if you concatenate all reducer outputs, the overall dataset is not sorted globally—it's just sorted in chunks.

Total order partitioning fixes this by ensuring that:

1. All keys assigned to reducer N are smaller than all keys assigned to reducer N+1
2. Each reducer still sorts its own keys internally

The end result? When you combine all reducer outputs (in order of reducer IDs), you get a globally sorted dataset.

Here are the most common use cases where total order partitioning is non-negotiable:

• Generating a globally sorted dataset: If you need to output a single (or combined) sorted file for reporting, archival, or downstream systems that expect ordered data (e.g., a time-series dataset sorted by timestamp across all logs).
• Range-based queries: If your next job needs to query a specific range of keys (e.g., "all records with IDs between 1000 and 2000"), global order lets you directly target the reducers that hold those keys instead of scanning every output file.
• Index building: When creating global indexes for a large dataset (like a search index), you need keys to be ordered across the entire dataset to ensure efficient lookups.
• Exporting to ordered storage systems: Some databases or data warehouses require input data to be globally sorted to optimize load performance and indexing.

Let's use a simple numeric key example to see the difference:

Default Hash Partitioner Behavior #

Raw input keys: 8, 3, 5, 1, 7, 2, 6, 4

Map phase → emits all keys as-is
┌─────────────────────────────────────────┐
│ HashPartitioner assigns keys by hash:   │
│ Reducer 1 gets: 8, 3, 7, 2             │
│ Reducer 2 gets: 5, 1, 6, 4             │
└─────────────────────────────────────────┘
          ↓
Each reducer sorts its own keys
┌─────────────────────────────────────────┐
│ Reducer 1 output (sorted): 2, 3, 7, 8   │
│ Reducer 2 output (sorted): 1, 4, 5, 6   │
└─────────────────────────────────────────┘

Global output (concatenated): 2,3,7,8,1,4,5,6 → GLOBALLY UNSORTED

Total Order Partitioner Behavior #

Raw input keys: 8, 3, 5, 1, 7, 2, 6, 4

Map phase → emits all keys as-is
┌─────────────────────────────────────────┐
│ TotalOrderPartitioner uses range splits:│
│ Partition boundary set at 4 (keys ≤4 go to R1, >4 to R2) │
│ Reducer 1 gets: 3, 1, 2, 4             │
│ Reducer 2 gets: 8, 5, 7, 6             │
└─────────────────────────────────────────┘
          ↓
Each reducer sorts its own keys
┌─────────────────────────────────────────┐
│ Reducer 1 output (sorted): 1, 2, 3, 4   │
│ Reducer 2 output (sorted): 5, 6, 7, 8   │
└─────────────────────────────────────────┘

Global output (concatenated): 1,2,3,4,5,6,7,8 → GLOBALLY SORTED

内容的提问来源于stack exchange，提问作者Learn Hadoop