Pytorch on ChengAo Shen

📚 Using Custom Dataset in PyTorch

Thu, 26 Oct 2023 00:00:00 +0000

In order to decouple dataset code and model code, PyTorch provides two data primitives: torch.utils.data.DataLoader and torch.utils.data.Dataset. Dataset stores the samples and their corresponding labels, and DataLoader wraps an iterable around the Dataset to enable easy access to the samples.

Load a Dataset Link to heading

If we want to use Data, we must have Data first. Fortunately, PyTorch domain libraries provide a number of pre-loaded datasets. All of them is the subclass of theDataset. Now we use Fashion-MNIST, one of them, to show how to load a dataset.

📝 Common PyTorch Code Snippets

Tue, 21 Mar 2023 00:00:00 +0000

Notes inspired by d2l’s PyTorch content

Device Switching Link to heading

Try using GPU:

def try_gpu(i=0):
    if torch.cuda.device_count() >= i + 1:
        return torch.device(f'cuda:{i}')
    return torch.device('cpu')

def try_all_gpus():
    """Find all available GPUs"""
    devices = [torch.device(f'cuda:{i}')
               for i in range(torch.cuda.device_count())]
    return devices if devices else [torch.device('cpu')]

Accumulator Link to heading

class Accumulator:
    """Used for accumulating values"""
    def __init__(self, n):
        self.data = [0.0] * n

    def add(self, *args):
        """Add a list of values to the existing data"""
        self.data = [a + float(b) for a, b in zip(self.data, args)]

    def reset(self):
        self.data = [0.0] * len(self.data)

    def __getitem__(self, idx):
        return self.data[idx]

Accuracy Link to heading

For training accuracy evaluation, you can use the following code

⚕️ Brief introduction of the Tensor in PyTorch

Wed, 26 Oct 2022 00:00:00 +0000

Tensor is a specialized data structure that is very similar to arrays and matrices. We can use it to encode the input and output of the model. Tensors can run on GPUs and other hardware.

Initializing a Tensors Link to heading

Tensors can be initialized in various ways,

# Import the library
import torch

# Directly from data
data=[[1,2],
      [2,3]]
t_data = torch.tensor(data)

# From Numpy
import numpy as np
np_data = np.array(data)
t_np =  torch.from_numpy(np_data)

# From other tensors
# In this way, it will retains same properties
t_ones = torch.ones_like(t_data) 
# override the datatype
t_random = torch.rand_like(t_data,dtype=torch.float) 

# With static shape
shape = (2,3,)
rand_t = torch.rand(shape)
ones_t = torch.ones(shape)
zeros_t = torch.zeros(shape)

Careful: If you initialize the tensors from Numpy, they will share the same underlying memory, which means that if changing the numpy array, the tensor will change too.