Transposing a tensor means you are swapping the rows and columns of the tensor. So, rows become columns, and columns become rows.
The torch.transpose() method swaps two specified dimensions (dim0 and dim1) and returns the transposed tensor. It rearranges tensor axes without altering the underlying data.
It only swaps two dimensions; the rest of the structure remains intact. It does not modify the original tensor.
The output tensor may not be contiguous, and you can check the contiguous tensor using torch.is_contiguous() method.
The main advantage is that the .transpose() method returns a view (no data copy) whenever possible, making it memory-efficient.
Syntax
torch.transpose(input, dim0, dim1)
Parameters
| Argument | Description |
| input (Tensor) | It is an input tensor that needs to be transposed. |
| dim0 (int) | It is the first dimension that needs to be swapped. |
| dim1 (int) | It is the second dimension that needs to be swapped. |
Transposing a 2D Tensor (Matrix)
As you can see from the above figure, the input matrix has 2 rows and 3 columns. After transposing, the output tensor (matrix) has 2 columns and 3 rows. Let’s implement this in a program.
import torch
# Creating a 2x3 matrix
tensor = torch.tensor([[11, 21, 31],
[40, 50, 60]])
print(tensor)
# Output:
# tensor([[11, 21, 31],
# [40, 50, 60]])
transposed_tensor = torch.transpose(tensor, 0, 1)
print(transposed_tensor)
# Output:
# tensor([[11, 40],
# [21, 50],
# [31, 60]])
In the output transposed_tensor, you can see that the row becomes the column, and the column becomes the row. The underlying value remains the same, but the shape is changed.
Higher-dimensional tensors
You can swap the arbitrary dimensions in a multi-dimensional tensor. For example, if you take a 3D tensor, you can swap at most two dimensions in a 3D tensor like this:
import torch three_d_tensor = torch.arange(24).reshape(2, 3, 4) print(three_d_tensor.shape) # Output: torch.Size([2, 3, 4]) transposed_3d = torch.transpose(three_d_tensor, 1, 2) print(transposed_3d.shape) # Output: torch.Size([2, 4, 3])
In the above code, the input tensor’s dimension was [2, 3, 4], and the output tensor’s dimension is [2, 4, 3], meaning 3 has been swapped with 4 and 4 has been swapped with 3.
Chained transpose operations
The transpose() allows us to chain operations.
Let’s chain the transpose operations.import torch d_tensor = torch.randn(2, 3, 4) print(d_tensor.shape) # Output: torch.Size([2, 3, 4]) complex_transpose = d_tensor.transpose(1, 2).transpose(0, 2) print(complex_transpose.shape) # Output: torch.Size([3, 4, 2])
.T vs. transpose()
The main difference between torch.T and torch.transpose() is that the .T attribute transposes the last two dimensions of the tensor, whereas the .transpose() method swaps any two specified dimensions.
import torch
matrix = torch.tensor([[1, 2, 3],
[4, 5, 6]])
print("Original Matrix:")
print(matrix)
print("Shape:", matrix.shape)
# Transpose using .T
matrix_t = matrix.T
print("\nTransposed with .T:")
print(matrix_t)
print("Shape:", matrix_t.shape)
# Transpose using transpose(0, 1)
matrix_transpose = matrix.transpose(0, 1)
print("\nTransposed with transpose(0, 1):")
print(matrix_transpose)
print("Shape:", matrix_transpose.shape)
# Check if both methods give same result
print("\nAre both transposes equal?", torch.equal(matrix_t, matrix_transpose))
# Output:
# Original Matrix:
# tensor([[1, 2, 3],
# [4, 5, 6]])
# Shape: torch.Size([2, 3])
# Transposed with .T:
# tensor([[1, 4],
# [2, 5],
# [3, 6]])
# Shape: torch.Size([3, 2])
# Transposed with transpose(0, 1):
# tensor([[1, 4],
# [2, 5],
# [3, 6]])
# Shape: torch.Size([3, 2])
# Are both transposes equal? True
If you are working with 2D tensors, you should use the .T attribute for transposing.
If you are working with 3D, 4D, or higher-dimensional tensors, use the .transpose() method.
torch.transpose() vs torch.permute()
When it comes to chaining the transpose operation, it may become complex due to many transformations. In that case, you can use the torch.permute() method.
For a single swap, always use the .transpose() method.
For multiple swaps, use the .permute() method.
import torch tensor = torch.arange(24).reshape(2, 3, 4) # Transpose swaps two dimensions transposed = torch.transpose(tensor, 1, 2) print(transposed.shape) # Output: Shape: (2, 4, 3) # Permute reorders all dimensions permuted = tensor.permute(2, 0, 1) print(permuted.shape) # Output: Shape: (4, 2, 3)
In the above code, you can analyze that the .transpose() function swaps only two dimensions, while the permute() function allows complete control over dimension order.
That’s all!
