Great question—let’s break down each part of your query clearly, since DNCs’ memory setup is one of their most interesting differentiators from traditional recurrent models.

1. DNC的外部内存是否有固定限制？ #

No, the external memory in a standard DNC does not have a hard-coded, innate limit like the human brain’s purported 7±2 working memory chunks. Instead, its capacity is defined by two critical hyperparameters set before training:

• N: The number of independent memory slots (think of each slot as a "chunk" of information)
• M: The dimensionality (size) of each memory slot

These values are chosen by the developer based on task complexity and available computational resources. For example, a simple sequence prediction task might use a small memory matrix like 64×16 (64 slots, each 16 dimensions), while a complex reasoning task could scale up to 512×128 or larger.

2. 它是否属于超参数？ #

Absolutely. As mentioned above, N (number of slots) and M (slot dimensionality) are core hyperparameters of the DNC architecture. Unlike the human brain’s working memory limits, these aren’t biological constraints—they’re design choices you tweak based on what your task needs and what your hardware can handle.

Some advanced DNC variants experiment with dynamic memory expansion (adding/removing slots during training), but this isn’t part of the original 2016 DNC design from DeepMind. In the standard implementation, memory size is fixed once training starts.

3. 借助短期内存可关联的数据量有多少？ #

The amount of data a DNC can actively associate depends on a few key factors:

• Number of read/write heads: Another hyperparameter—DNCs use multiple read heads to access different memory slots simultaneously. If you have k read heads, the model can explicitly pull and correlate information from up to k distinct memory chunks at once.
• Memory slot size (M): Larger slots can store more detailed information, so each "chunk" holds richer data that can be linked to other slots.
• Attention mechanisms: DNCs use content-based and location-based attention to retrieve relevant memory. Even with a fixed number of read heads, the model can implicitly connect information across multiple slots by weighting their relevance (e.g., a single read head might focus on a cluster of semantically related slots rather than just one).

In practice, the effective associative capacity is a balance between these hyperparameters and the task’s requirements—there’s no universal number, but it’s far more flexible than the human brain’s 7±2 rule.

内容的提问来源于stack exchange，提问作者James Luther