PyTorch Tensor – A Detailed Overview With Examples [Latest]

Let’s get started!

PyTorch Tensor

Have you worked with Python numpy before? If yes, then this section is going to be very simple for you! Even if you don’t have experience with numpy, you can seamlessly transition between PyTorch and NumPy!

A Tensor in PyTorch is similar to numpy arrays, with the additional flexibility of using a GPU for calculations.

1. 2D Pytorch Tensor

Imagine a tensor as an array of numbers, with a potentially arbitrary number of dimensions. The only difference between a Tensor and a multidimensional array in C/C++/Java is that the size of all the columns in a dimension is the same.

For example, the below can be a valid representation of a 2 Dimensional Tensor.

Note, however, that the below example is NOT a valid example, since Tensors are not jagged arrays.

PyTorch Tensors are really convenient for programmers, since they are almost the same as numpy arrays.

There are a couple of differences to numpy methods, though, so it is advised that you also refer the official Documentation for further information.

2. Initializing an Empty PyTorch Tensor

Let’s consider the below example, which initializes an empty Tensor.

An empty tensor does NOT mean that it does not contain anything. It’s just that there is memory allocated for it.

Output

The first tensor is a result of PyTorch simply allocating memory for the tensor. Whatever previous content in the memory is not erased.

The second tensor is filled with zeros, since PyTorch allocates memory and zero-initializes the tensor elements.

Notice the similarity to numpy.empty() and numpy.zeros(). This is because PyTorch is designed to replace numpy, since the GPU is available.

3. Finding PyTorch Tensor Size

Let’s create a basic tensor and determine its size.

Output

To get the size of the tensor, we can use tensor.size()

Output

PyTorch Tensor Operations

Like numpy, PyTorch supports similar tensor operations.

The summary is given in the below code block.

1. Basic Mathematical Operations on Tensors

Output

We can also assign the result to a tensor. Add the following code snippet to the code above.

Output

2. Inline Addition and Subtraction with PyTorch Tensor

PyTorch also supports in-place operations like addition and subtraction, when suffixed with an underscore (_). Let’s continue on with the same variables from the operations summary code above.

Output

3. Accessing Tensor Index

We can also use numpy based indexing in PyTorch

Output

Reshape a PyTorch Tensor

Similar to numpy, we can use torch.reshape() to reshape a tensor. We can also use tensor.view() to achieve the same functionality.

Output

The list of all Tensor Operations is available in PyTorch’s Documentation.

PyTorch – NumPy Bridge

We can convert PyTorch tensors to numpy arrays and vice-versa pretty easily.

PyTorch is designed in such a way that a Torch Tensor on the CPU and the corresponding numpy array will have the same memory location. So if you change one of them, the other one will automatically be changed.

To prove this, let’s test it using the torch.numpy() and the torch.from_numpy() methods.

torch.numpy() is used to convert a Tensor to a numpy array, and torch.from_numpy() will do the reverse.

Output

Indeed, the numpy array has also changed it’s value!

Let’s do the reverse as well

Output

NOTE: If you do not use the numpy in-place addition using a += 3 or np.add(out=a), then the Tensor will not reflect the changes in the numpy array.

For example, if you try this:

Since you’re using =, this means that Python will create a new object and assign that new object to the name called c. So the original memory location is still unchanged.

Use the CUDA GPU with a PyTorch Tensor

We can make the NVIDIA CUDA GPU perform the computations and have a speedup, by moving the tensor to the GPU.

NOTE: This applies only if you have an NVIDIA GPU with CUDA enabled. If you’re not sure of what these terms are, I would advise you to search online.

We can check if we have the GPU available for PyTorch using torch.cuda.is_available()

For me, it is available, so just make sure you install CUDA before proceeding further if your laptop supports it.

We can move a tensor from the CPU to the GPU using tensor.to(device), where device is a device object.

This can be torch.device("cuda"), or simply cpu.

Output

As you can see, the output does show that our program is now being run on the GPU instead!

References