A fixed 2 by 3 float32 tensor holds six numbers in one flat line of memory, read through a (2, 3) shape lens — exactly like a NumPy array. Two extra fields make it a tensor. A device chip moves the block between a cpu room and a gpu room, animated as a real copy across, not a relabel. A requires_grad switch controls whether running s equals (t times 2).sum() records a tape: off gives just s equals 42.0 with grad_fn None; on grows a tape of MulBackward0 then SumBackward0 ending at s equals 42.0, and s.backward() then fills a t.grad strip with 2.0 in every cell, since ds by dt is 2 everywhere.
A tensor is an array with a passport and a memory
Same flat block as NumPy, same shape-lens. A tensor stamps two more things on it: a passport — which processor's memory holds the block — and a memory — a tape that records every op.
the block — one flat line, read as (2, 3)
row 0
row 1
t.grad
d s / d t = 2, everywhere
shape (2, 3)dtype float32
the passport — where the block lives
cpu
gpu (mps)
the memory — is the tape recording?
requires_gradFalse
s = (t * 2).sum()
t.shape(2, 3)
t.devicecpu
t.requires_gradFalse
t.gradNone
NumPy gave the flat block and the shape-lens. A tensor adds a passport — which processor's memory the block sits in — and a memory: with requires_grad on, every op writes itself onto a tape (grad_fn), which is exactly what backward() replays in reverse. Those two additions are the whole difference — and the whole reason deep learning runs on tensors.