we are importing different libraries required for the tutorial here.
!git clone https://github.com/Adversarial-Deep-Learning/code-soup
Cloning into 'code-soup'... remote: Enumerating objects: 533, done. remote: Counting objects: 100% (289/289), done. remote: Compressing objects: 100% (230/230), done. remote: Total 533 (delta 145), reused 126 (delta 57), pack-reused 244 Receiving objects: 100% (533/533), 115.09 KiB | 2.30 MiB/s, done. Resolving deltas: 100% (207/207), done.
cd code-soup
/content/code-soup
from code_soup.ch5.models import gan
import torch.utils.data
import torchvision
from torch.utils.tensorboard import SummaryWriter
You will mostly familier with most of libraries above; except code-soup ;
Link for the repo is here
PATH_DATA="./input/data"
#directory for downloading and storing the dataset
IMAGE_SIZE=28
#Size of image
NUM_CHANNELS=1
#num of channels; for rgb images this value should be 3; for grey scale this value is 1
LATENT_DIM=64
#dimension of random noise vector; this randomly generated vectore will be used by generator to generate images
LEARNING_RATE=0.0002
DEVICE="cuda" if torch.cuda.is_available() else "cpu"
#if GPU is available; this variable helps in faster computing;
NUM_EPOCHS=25
#num of times model will learn over the data
BATCH_SIZE=32
#number of images used in one step of training the models
FIXED_NOICE=torch.randn(BATCH_SIZE,LATENT_DIM).to(DEVICE)
#this randomly generated vectore will be used to visualise with tensorboard how our generator is performing over the time period;
#keeping the vectore constant, it will be better to analyse how the model is performing
#summary writers for real and fake
writer_fake=SummaryWriter(f"./logs/fake") #creates instance of SummaryWriter and stores log files produced at path "logs/fake"
writer_real=SummaryWriter(f"./logs/real") #creates instance of SummaryWriter and stores log files produced at path "logs/real"
#Download the datasets using ImageClassificationDataset class
TRANSFORM=torchvision.transforms.Compose([ torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize([0.5],[0.5])
])
lets download the data from tochvision, and then with DataLoader class of torch.utils.data module; we will make different batches of the data having BATCH_SIZE images in each batch
refer this if having difficulty understanding next cell
dataloader_obj=torch.utils.data.DataLoader(torchvision.datasets.MNIST(root=PATH_DATA,
transform=TRANSFORM,
train=True,
download=True),
batch_size=BATCH_SIZE,
shuffle=True)
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ./input/data/MNIST/raw/train-images-idx3-ubyte.gz
Extracting ./input/data/MNIST/raw/train-images-idx3-ubyte.gz to ./input/data/MNIST/raw Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ./input/data/MNIST/raw/train-labels-idx1-ubyte.gz
Extracting ./input/data/MNIST/raw/train-labels-idx1-ubyte.gz to ./input/data/MNIST/raw Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ./input/data/MNIST/raw/t10k-images-idx3-ubyte.gz
Extracting ./input/data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./input/data/MNIST/raw Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ./input/data/MNIST/raw/t10k-labels-idx1-ubyte.gz
Extracting ./input/data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./input/data/MNIST/raw
/usr/local/lib/python3.7/dist-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:180.) return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)
gan_obj=gan.GAN(image_size=IMAGE_SIZE,
channels=NUM_CHANNELS,
latent_dims=LATENT_DIM,
lr=LEARNING_RATE,
device=DEVICE,
)
for epoch in range(NUM_EPOCHS):
for idx,data_ in enumerate(dataloader_obj):
#iterating through batches of data
gan_obj.generator.train()
gan_obj.discriminator.train()
#setting up D and G for training
(D_x, D_G_z1, errD, D_G_z2, errG)=gan_obj.step(data=data_)
#gan_obj.step returns 5 objects- D_x, D_G_z1, errD, D_G_z2, errG
print(f"EPOCH-- {epoch}/{NUM_EPOCHS} Disc_loss-- {errD}, Gen_loss-- {errG}")
gan_obj.generator.eval()
gan_obj.discriminator.eval()
#setting D and G for evaluation
with torch.no_grad():
tensorboard_fake=gan_obj.generator(FIXED_NOICE).view(BATCH_SIZE,NUM_CHANNELS,IMAGE_SIZE,IMAGE_SIZE).to(DEVICE)
(real_image,_)=data_
tensorboard_real=real_image.reshape(BATCH_SIZE,NUM_CHANNELS,IMAGE_SIZE,IMAGE_SIZE)
#making sure that the images are in shape for writing to tensorboard
image_grid_fake=torchvision.utils.make_grid(tensorboard_fake,normalize=True)
image_grid_real = torchvision.utils.make_grid(tensorboard_real, normalize=True)
writer_fake.add_image("MNIST fake Image",image_grid_fake,global_step=epoch)
writer_real.add_image("MNIST real Image", image_grid_real,global_step=epoch)
#adding logs to `logs` files and this will be later used in visualising
EPOCH-- 0/25 Disc_loss-- 1.2358217239379883, Gen_loss-- 1.0005135536193848 EPOCH-- 1/25 Disc_loss-- 1.2377655506134033, Gen_loss-- 0.9847954511642456 EPOCH-- 2/25 Disc_loss-- 1.2691676616668701, Gen_loss-- 0.7894760966300964 EPOCH-- 3/25 Disc_loss-- 1.385810375213623, Gen_loss-- 0.7533003091812134 EPOCH-- 4/25 Disc_loss-- 1.2962983846664429, Gen_loss-- 0.8748288154602051 EPOCH-- 5/25 Disc_loss-- 1.2990851402282715, Gen_loss-- 1.0871098041534424 EPOCH-- 6/25 Disc_loss-- 1.2749874591827393, Gen_loss-- 0.8466078042984009 EPOCH-- 7/25 Disc_loss-- 1.2267043590545654, Gen_loss-- 0.968805730342865 EPOCH-- 8/25 Disc_loss-- 1.3918840885162354, Gen_loss-- 0.7618818879127502 EPOCH-- 9/25 Disc_loss-- 1.2063560485839844, Gen_loss-- 0.8579467535018921 EPOCH-- 10/25 Disc_loss-- 1.3138501644134521, Gen_loss-- 0.8212562799453735 EPOCH-- 11/25 Disc_loss-- 1.182576298713684, Gen_loss-- 0.9731820821762085 EPOCH-- 12/25 Disc_loss-- 1.334852695465088, Gen_loss-- 0.9209056496620178 EPOCH-- 13/25 Disc_loss-- 1.3283047676086426, Gen_loss-- 0.7867284417152405 EPOCH-- 14/25 Disc_loss-- 1.415947437286377, Gen_loss-- 0.8491349220275879 EPOCH-- 15/25 Disc_loss-- 1.2332701683044434, Gen_loss-- 1.0271377563476562 EPOCH-- 16/25 Disc_loss-- 1.2704250812530518, Gen_loss-- 0.8522036075592041 EPOCH-- 17/25 Disc_loss-- 1.434761643409729, Gen_loss-- 0.9233605861663818 EPOCH-- 18/25 Disc_loss-- 1.1845335960388184, Gen_loss-- 0.9122719168663025 EPOCH-- 19/25 Disc_loss-- 1.2548960447311401, Gen_loss-- 1.1219961643218994 EPOCH-- 20/25 Disc_loss-- 1.2985920906066895, Gen_loss-- 0.9018627405166626 EPOCH-- 21/25 Disc_loss-- 1.2224535942077637, Gen_loss-- 0.8015543222427368 EPOCH-- 22/25 Disc_loss-- 1.3744053840637207, Gen_loss-- 0.8891228437423706 EPOCH-- 23/25 Disc_loss-- 1.33937668800354, Gen_loss-- 0.7981967926025391 EPOCH-- 24/25 Disc_loss-- 1.3143424987792969, Gen_loss-- 0.7740237712860107
To see the Logs; first load tensorflow extension
%load_ext tensorboard
The tensorboard extension is already loaded. To reload it, use: %reload_ext tensorboard
%tensorboard --logdir logs
Reusing TensorBoard on port 6006 (pid 192), started 0:13:39 ago. (Use '!kill 192' to kill it.)