PyTorch中torch.from_numpy()与torch.Tensor()的内存模型及存在意义问询

Great question! Your experiment does a nice job highlighting the unexpected (at first glance) memory-sharing behavior between torch.Tensor()-created tensors and numpy arrays. Let’s unpack why PyTorch provides torch.from_numpy() even though torch.Tensor() can sometimes achieve the same result.

1. Explicit Type Preservation #

torch.from_numpy() guarantees that the resulting tensor will have the exact same data type as the input numpy array. This isn’t always the case with torch.Tensor():

• For example, if you have a numpy array of type np.int64, torch.Tensor(arr) will convert it to a torch.FloatTensor (since torch.Tensor is an alias for torch.FloatTensor by default), while torch.from_numpy(arr) returns a torch.LongTensor that matches the numpy dtype exactly.
• This strict type consistency avoids silent conversions that can break downstream computations or introduce unexpected precision losses.

import numpy as np
import torch

arr = np.arange(10, dtype=np.int64)
t_tensor = torch.Tensor(arr)
t_from_numpy = torch.from_numpy(arr)

print(t_tensor.dtype)  # Output: torch.float32
print(t_from_numpy.dtype)  # Output: torch.int64

2. Clear API Intent & Guaranteed Memory Sharing #

torch.from_numpy() exists to signal a clear, intentional numpy-PyTorch interop operation. PyTorch’s documentation explicitly states that tensors created with this function share memory with the input numpy array—this is a contractual guarantee.

• In contrast, while torch.Tensor() may share memory with a numpy array in practice (as your experiment shows), this behavior isn’t explicitly promised by the API. PyTorch could theoretically adjust this implementation detail in a future version without breaking backward compatibility for torch.Tensor() users, whereas torch.from_numpy()’s memory-sharing behavior is part of its core purpose.

3. Handling Non-Contiguous Arrays #

Numpy arrays can be non-contiguous in memory (e.g., after a transpose or advanced slicing). torch.from_numpy() directly wraps these non-contiguous arrays into a non-contiguous PyTorch tensor without copying data.

• torch.Tensor() may implicitly copy non-contiguous numpy data into a contiguous tensor buffer in some cases, breaking the memory sharing you observed with contiguous arrays. This inconsistency makes torch.from_numpy() the safer choice when you specifically want to avoid copying data.

4. Distinction from General-Purpose Constructors #

It’s also worth noting the difference between torch.Tensor() (the class constructor) and torch.tensor() (the factory function). The lowercase torch.tensor() always copies input data by default, even when given a numpy array. So if you accidentally use torch.tensor() instead of torch.from_numpy(), you’ll lose the memory-sharing optimization. torch.from_numpy() acts as a clear signal that you want zero-copy interop with numpy.

Wrap-Up #

In your specific case, both functions resulted in memory sharing because you used a contiguous float32 numpy array. But torch.from_numpy() is the more reliable, explicit choice for numpy-PyTorch conversions when:

• You need strict dtype matching
• You rely on guaranteed memory sharing
• You’re working with non-contiguous numpy arrays

内容的提问来源于stack exchange，提问作者kmario23