Problem 2: Generative Adversarial Networks

Learning Objective: In this problem, you will implement a Generative Adversarial Network with the network structure proposed in Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, and learn a visualization technique, activation maximization.
Provided code: The code for constructing the two parts of the GAN, the discriminator and the generator, is done for you, along with the skeleton code for the training.
TODOs: You will need to figure out how to properly feed the data, compute the loss and update the parameters to complete the training and visualization. In addition, to test your understanding, you will answer some non-coding questions.



In [1]:

    
# Import required libraries
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import math
%matplotlib inline

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

Introduction: The forger versus the police

Note: read the story even if you are already familiar with GANs, as one of the questions is related to this story.

Generative models try to model the distribution of the data in an explicit way, in the sense that we can easily sample new data points from this model. This is in contrast to discriminative models that try to infer the output from the input. In the class we have seen one classic deep generative model, the Variational Autoencoder (VAE). Here, we will learn another generative model that has risen to prominence in recent years, the Generative Adversarial Network (GAN).

As the maths of Generative Adversarial Networks is somewhat tedious, a story is often told of a forger and a police to illustrate the idea.

Imagine a forger that makes fake bills, and a police that tries to find these forgeries. If the forger were a VAE, his goal would be to take some real bills, and try to replicate the real bills as precisely as possible. In GAN, he has a different idea in his mind: rather than trying to replicate the real bills, it suffices to make fake bills such that people think they are real.

Now lets start. In the beginning, the police knows nothing about how to distinguish between real and fake bills. The forger knows nothing either and only produces white paper.

In the first round, the police gets the fake bill and learns that the forgeries are white while the real bills are green. The forger then finds out that white papers can no longer fool the police and starts to produce green papers.

In the second round, the police learns that real bills have denominations printed on them while the forgeries do not. The forger then finds out that plain papers can no longer fool the police and starts to print numbers on them.

In the third round, the police learns that real bills have watermarks on them while the forgeries do not. The forger then has to reproduce the watermarks on his fake bills.

...

Finally, the police is able to spot the tiniest difference between real and fake bills and the forger has to make perfect replicas of real bills to fool the police.

Now in a GAN, the forger becomes the generator and the police becomes the discriminator. The discriminator is a binary classifier with the two classes being "taken from the real data" ("real") and "generated by the generator" ("fake"). Its objective is to minimize the classification loss. The generator's objective is to generate samples so that the discriminator misclassify them as real.

Here we have some complications: the goal is not to find one perfect fake sample. Such a sample will not actually fool the discriminator: if the forger makes hundreds of the exact same fake bill, they will all have the same serial number and the police will soon find out that they are fake. Instead, we want the generator to be able to generate a variety of fake samples such that when presented as a distribution alongside the distribution of real samples, these two are indistinguishable by the discriminator.

So how do we generate different samples with a diterministic generator? We provide it with random numbers as input.

Typically, for the discriminator we use binary cross entropy loss with label 1 being real and 0 being fake. For the generator, the input is a random vector drawn from a standard normal distribution. Denote the generator by $G_{\phi}(z)$, discriminator by $D_{\theta}(x)$, the distribution of the real samples by $p(x)$ and the input distribution to the generator by $q(z)$. Recall that the binary cross entropy loss with classifier output $y$ and label $\hat{y}$ is

$$L(y, \hat{y}) = -\hat{y} \log y - (1 - \hat{y}) \log (1 - y)$$

For the discriminator, the objective is

$$\min_{\theta} \mathrm{E}_{x \sim p(x)}[L(D_{\theta}(x), 1)] + \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 0)]$$

For the generator, the objective is

$$\max_{\phi} \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 0)]$$

The generator's objective corresponds to maximizing the classification loss of the discriminator on the generated samples. Alternatively, we can minimize the classification loss of the discriminator on the generated samples when labelled as real:

$$\min_{\phi} \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 1)]$$

And this is what we will use in our implementation. The strength of the two networks should be balanced, so we train the two networks alternatingly, updating the parameters in both networks once in each interation.

Problem 2-1: Implementing the GAN

We first load the data (CIFAR-10) and define some convenient functions. You should already have CIFAR-10 from assignment 1. Just copy the data from there or use data/get_datasets.sh if you don't have them.



In [2]:

    
def unpickle(file):
    import sys
    if sys.version_info.major == 2:
        import cPickle
        with open(file, 'rb') as fo:
            dict = cPickle.load(fo)
        return dict['data'], dict['labels']
    else:
        import pickle
        with open(file, 'rb') as fo:
            dict = pickle.load(fo, encoding='bytes')
        return dict[b'data'], dict[b'labels']

def load_train_data():
    X = []
    for i in range(5):
        X_, _ = unpickle('data/cifar-10-batches-py/data_batch_%d' % (i + 1))
        X.append(X_)
    X = np.concatenate(X)
    X = X.reshape((X.shape[0], 3, 32, 32)).transpose(0, 2, 3, 1)
    return X

def load_test_data():
    X_, _ = unpickle('data/cifar-10-batches-py/test_batch')
    X = X_.reshape((X_.shape[0], 3, 32, 32)).transpose(0, 2, 3, 1)
    return X

# Load cifar-10 data
train_samples = load_train_data() / 255.0
test_samples = load_test_data() / 255.0



In [3]:

    
def viz_grid(Xs, padding):
    N, H, W, C = Xs.shape
    grid_size = int(math.ceil(math.sqrt(N)))
    grid_height = H * grid_size + padding * (grid_size + 1)
    grid_width = W * grid_size + padding * (grid_size + 1)
    grid = np.zeros((grid_height, grid_width, C))
    next_idx = 0
    y0, y1 = padding, H + padding
    for y in range(grid_size):
        x0, x1 = padding, W + padding
        for x in range(grid_size):
            if next_idx < N:
                img = Xs[next_idx]
                grid[y0:y1, x0:x1] = img
                next_idx += 1
            x0 += W + padding
            x1 += W + padding
        y0 += H + padding
        y1 += H + padding
    return grid

def conv2d(input, kernel_size, stride, num_filter, name = 'conv2d'):
    with tf.variable_scope(name):
        stride_shape = [1, stride, stride, 1]
        filter_shape = [kernel_size, kernel_size, input.get_shape()[3], num_filter]

        W = tf.get_variable('w', filter_shape, tf.float32, tf.random_normal_initializer(0.0, 0.02))
        b = tf.get_variable('b', [1, 1, 1, num_filter], initializer = tf.constant_initializer(0.0))
        return tf.nn.conv2d(input, W, stride_shape, padding = 'SAME') + b

def conv2d_transpose(input, kernel_size, stride, num_filter, name = 'conv2d_transpose'):
    with tf.variable_scope(name):
        stride_shape = [1, stride, stride, 1]
        filter_shape = [kernel_size, kernel_size, num_filter, input.get_shape()[3]]
        output_shape = tf.stack([tf.shape(input)[0], tf.shape(input)[1] * 2, tf.shape(input)[2] * 2, num_filter])

        W = tf.get_variable('w', filter_shape, tf.float32, tf.random_normal_initializer(0.0, 0.02))
        b = tf.get_variable('b', [1, 1, 1, num_filter], initializer = tf.constant_initializer(0.0))
        return tf.nn.conv2d_transpose(input, W, output_shape, stride_shape, padding = 'SAME') + b

def fc(input, num_output, name = 'fc'):
    with tf.variable_scope(name):
        num_input = input.get_shape()[1]
        W = tf.get_variable('w', [num_input, num_output], tf.float32, tf.random_normal_initializer(0.0, 0.02))
        b = tf.get_variable('b', [num_output], initializer = tf.constant_initializer(0.0))
        return tf.matmul(input, W) + b

def batch_norm(input, is_training):
    out = tf.contrib.layers.batch_norm(input, decay = 0.99, center = True, scale = True,
                                       is_training = is_training, updates_collections = None)
    return out

def leaky_relu(input, alpha = 0.2):
    return tf.maximum(alpha * input, input)

To save you some mundane work, we have defined a discriminator and a generator for you, in _discriminator() and _generator() respectively. Look at the code to see what layers are there.

For this part, you need to complete code blocks marked with "Prob 2-1":

Build the computation graph for the losses: Complete the following definitions in _init_ops()
- fake_samples_op: generate famples from noise
- dis_loss_op: compute discriminator's loss, with real samples from real_input and fake samples generated by the generator
- gen_loss_op: compute generator's loss
Define the optimizer: We use RMSprop for training. Adam is observed to perform poorly with an unstable objective as is the case in GANs. We've defined dis_train_op and gen_train_op for you but those are wrong: rather than updating all the parameters all the time, when training one network we want to keep the other one fixed. Modify the definition to reflect this. Check here if you are not sure how this is possible.
Feed the data: Feed the proper samples and labels in train() for training and in generate_one_sample() for visualizing the generated samples.

The batch normalization layers should operate in training mode. As per How to Train a GAN? Tips and tricks to make GANs work, we put real samples and fake samples in different batches when training the discriminator.

Note: use the advices on that page with caution if you are doing GAN for your team project. It is already 10 months old, which is a really long time in deep learning research. It does not reflect the latest results.

For the discriminator, the objective is

$$\min_{\theta} \mathrm{E}_{x \sim p(x)}[L(D_{\theta}(x), 1)] + \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 0)]$$

For the generator, the objective is

$$\max_{\phi} \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 0)]$$

$$\min_{\phi} \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 1)]$$



In [4]:

    
class DCGAN(object):

    def __init__(self):
        self.num_epoch = 50
        self.batch_size = 32
        self.log_step = 50
        self.visualize_step = 200
        self.code_size = 64
        self.learning_rate = 1e-4
        self.vis_learning_rate = 1e-2
        self.recon_steps = 100
        self.actmax_steps = 100
        
        self._dis_called = False
        self._gen_called = False

        self.tracked_noise = np.random.normal(0, 1, [64, self.code_size])

        self.real_input = tf.placeholder(tf.float32, [None, 32, 32, 3])
        self.real_label = tf.placeholder(tf.float32, [None, 1])
        self.fake_label = tf.placeholder(tf.float32, [None, 1])
        self.noise = tf.placeholder(tf.float32, [None, self.code_size])
        
        self.is_train = tf.placeholder(tf.bool)
        
        self.recon_sample = tf.placeholder(tf.float32, [1, 32, 32, 3])
        self.actmax_label = tf.placeholder(tf.float32, [1, 1])
        
        with tf.variable_scope('actmax'):
            self.actmax_code = tf.get_variable('actmax_code', [1, self.code_size],
                                               initializer = tf.constant_initializer(0.0))
        
        self._init_ops()

    def _discriminator(self, input):
        # We have multiple instances of the discriminator in the same computation graph,
        # so set variable sharing if this is not the first invocation of this function.
        with tf.variable_scope('dis', reuse = self._dis_called):
            self._dis_called = True
            dis_conv1 = conv2d(input, 4, 2, 32, 'conv1')
            dis_lrelu1 = leaky_relu(dis_conv1)
            dis_conv2 = conv2d(dis_lrelu1, 4, 2, 64, 'conv2')
            dis_batchnorm2 = batch_norm(dis_conv2, self.is_train)
            dis_lrelu2 = leaky_relu(dis_batchnorm2)
            dis_conv3 = conv2d(dis_lrelu2, 4, 2, 128, 'conv3')
            dis_batchnorm3 = batch_norm(dis_conv3, self.is_train)
            dis_lrelu3 = leaky_relu(dis_batchnorm3)
            dis_reshape3 = tf.reshape(dis_lrelu3, [-1, 4 * 4 * 128])
            dis_fc4 = fc(dis_reshape3, 1, 'fc4')
            return dis_fc4

    def _generator(self, input):
        with tf.variable_scope('gen', reuse = self._gen_called):
            self._gen_called = True
            gen_fc1 = fc(input, 4 * 4 * 128, 'fc1')
            gen_reshape1 = tf.reshape(gen_fc1, [-1, 4, 4, 128])
            gen_batchnorm1 = batch_norm(gen_reshape1, self.is_train)
            gen_lrelu1 = leaky_relu(gen_batchnorm1)
            gen_conv2 = conv2d_transpose(gen_lrelu1, 4, 2, 64, 'conv2')
            gen_batchnorm2 = batch_norm(gen_conv2, self.is_train)
            gen_lrelu2 = leaky_relu(gen_batchnorm2)
            gen_conv3 = conv2d_transpose(gen_lrelu2, 4, 2, 32, 'conv3')
            gen_batchnorm3 = batch_norm(gen_conv3, self.is_train)
            gen_lrelu3 = leaky_relu(gen_batchnorm3)
            gen_conv4 = conv2d_transpose(gen_lrelu3, 4, 2, 3, 'conv4')
            gen_sigmoid4 = tf.sigmoid(gen_conv4)
            return gen_sigmoid4

    def _loss(self, labels, logits):
        loss = tf.nn.sigmoid_cross_entropy_with_logits(labels = labels, logits = logits)
        return tf.reduce_mean(loss)

    def _reconstruction_loss(self, generated, target):
        loss = tf.nn.l2_loss(generated - target)
        return tf.reduce_mean(loss)
    
    # Define operations
    def _init_ops(self):
        
        ################################################################################
        # Prob 2-1: complete the definition of these operations                        #
        ################################################################################
        ### fake_samples_op: generate famples from noise
        ### dis_loss_op: compute discriminator's loss, with real samples from real_input and 
        ## fake samples generated by the generator
        ##gen_loss_op: compute generator's loss
        self.fake_samples_op = self._generator(self.noise)
        fake_samples_dis = self._discriminator(self.fake_samples_op)

        self.dis_loss_op = self._loss(self.fake_label, fake_samples_dis) +\
                            self._loss(self.real_label, self._discriminator(self.real_input))
        self.gen_loss_op = self._loss(self.real_label, fake_samples_dis)
        
        ################################################################################
        # Prob 2-1: fix the definition of these operations                             #
        ################################################################################
        dis_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'dis')
        gen_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'gen')
        
        dis_optimizer = tf.train.RMSPropOptimizer(self.learning_rate)
        self.dis_train_op = dis_optimizer.minimize(self.dis_loss_op, var_list=dis_var_list)

        
        gen_optimizer = tf.train.RMSPropOptimizer(self.learning_rate)
        self.gen_train_op = gen_optimizer.minimize(self.gen_loss_op, var_list=gen_var_list)
        ################################################################################
        # Prob 2-4: check the definition of these operations                           #
        # skip this part when working on problem 2-1 and come back for problem 2-4     #
        ################################################################################
        
        self.actmax_sample_op = self._generator(self.actmax_code)
        actmax_dis = self._discriminator(self.actmax_sample_op)
        self.actmax_loss_op = self._loss(self.actmax_label, actmax_dis)

        actmax_optimizer = tf.train.AdamOptimizer(self.vis_learning_rate)
        self.actmax_op = actmax_optimizer.minimize(self.actmax_loss_op, var_list = [self.actmax_code])
        
        ################################################################################
        # Prob 2-4: complete the definition of these operations                        #
        # skip this part when working on problem 2-1 and come back for problem 2-4     #
        ################################################################################
        
        self.recon_loss_op =  self._reconstruction_loss(self.actmax_sample_op, self.recon_sample )
        
        recon_optimizer = tf.train.AdamOptimizer(self.vis_learning_rate)
        self.reconstruct_op = recon_optimizer.minimize(self.recon_loss_op)
        
        ################################################################################
        #                               END OF YOUR CODE                               #
        ################################################################################

    # Training function
    def train(self, sess, train_samples):
        sess.run(tf.global_variables_initializer())

        num_train = train_samples.shape[0]
        step = 0
        
        # smooth the loss curve so that it does not fluctuate too much
        smooth_factor = 0.95
        plot_dis_s = 0
        plot_gen_s = 0
        plot_ws = 0
        
        dis_losses = []
        gen_losses = []
        for epoch in range(self.num_epoch):
            for i in range(num_train // self.batch_size):
                step += 1

                batch_samples = train_samples[i * self.batch_size : (i + 1) * self.batch_size]
                noise = np.random.normal(0, 1, [self.batch_size, self.code_size])
                zeros = np.zeros([self.batch_size, 1])
                ones = np.ones([self.batch_size, 1])
        
                ################################################################################
                # Prob 2-1: complete the feed dictionary                                       #
                ################################################################################
                
                dis_feed_dict = {self.real_input: batch_samples,
                                 self.real_label: ones,
                                 self.fake_label: zeros,
                                 self.noise: noise,
                                 self.is_train: True}
        
        
        
                ################################################################################
                #                               END OF YOUR CODE                               #
                ################################################################################

                _, dis_loss = sess.run([self.dis_train_op, self.dis_loss_op], feed_dict = dis_feed_dict)
        
                ################################################################################
                # Prob 2-1: complete the feed dictionary                                       #
                ################################################################################
                
                gen_feed_dict = {self.real_input: batch_samples,
                                 self.real_label: ones,
                                 self.noise: noise,
                                 self.is_train: True}
        
                ################################################################################
                #                               END OF YOUR CODE                               #
                ################################################################################

                _, gen_loss = sess.run([self.gen_train_op, self.gen_loss_op], feed_dict = gen_feed_dict)

                plot_dis_s = plot_dis_s * smooth_factor + dis_loss * (1 - smooth_factor)
                plot_gen_s = plot_gen_s * smooth_factor + gen_loss * (1 - smooth_factor)
                plot_ws = plot_ws * smooth_factor + (1 - smooth_factor)
                dis_losses.append(plot_dis_s / plot_ws)
                gen_losses.append(plot_gen_s / plot_ws)

                if step % self.log_step == 0:
                    print('Iteration {0}: dis loss = {1:.4f}, gen loss = {2:.4f}'.format(step, dis_loss, gen_loss))

            fig = plt.figure(figsize = (8, 8))   
            ax1 = plt.subplot(111)
            ax1.imshow(viz_grid(self.generate(self.tracked_noise), 1))
            plt.show()

            plt.plot(dis_losses)
            plt.title('discriminator loss')
            plt.xlabel('iterations')
            plt.ylabel('loss')
            plt.show()

            plt.plot(gen_losses)
            plt.title('generator loss')
            plt.xlabel('iterations')
            plt.ylabel('loss')
            plt.show()

    # Find the reconstruction of one input sample
    def reconstruct_one_sample(self, sample):
        
        ################################################################################
        # Prob 2-4: initialize self.actmax_code                                        #
        # skip this part when working on problem 2-1 and come back for problem 2-4     #
        ################################################################################        
        actmax_init_val = np.zeros((1, self.code_size))
        ################################################################################
        #                               END OF YOUR CODE                               #
        ################################################################################
        
        sess.run(self.actmax_code.assign(actmax_init_val))
        last_reconstruction = None
        last_loss = None
        for i in range(self.recon_steps):
        
            ################################################################################
            # Prob 2-4: complete the feed dictionary                                       #
            # skip this part when working on problem 2-1 and come back for problem 2-4     #
            ################################################################################
                
            recon_feed_dict = {self.recon_sample: sample, 
                               self.is_train: False}         
        
            ################################################################################
            #                               END OF YOUR CODE                               #
            ################################################################################
            
            run_ops = [self.recon_loss_op, self.reconstruct_op, self.actmax_sample_op]
            last_loss, _, last_reconstruction = sess.run(run_ops, feed_dict = recon_feed_dict)
        return last_loss, last_reconstruction

    # Find the reconstruction of a batch of samples
    def reconstruct(self, samples):
        reconstructions = np.zeros(samples.shape)
        total_loss = 0
        for i in range(samples.shape[0]):
            loss, reconstructions[i:i+1] = self.reconstruct_one_sample(samples[i:i+1])
            total_loss += loss
        return total_loss / samples.shape[0], reconstructions

    # Generates a single sample from input code
    def generate_one_sample(self, code):
        
        ################################################################################
        # Prob 2-1: complete the feed dictionary                                       #
        ################################################################################

        gen_vis_feed_dict = {self.noise: code, self.is_train: False}
        
        ################################################################################
        #                               END OF YOUR CODE                               #
        ################################################################################
        
        generated = sess.run(self.fake_samples_op, feed_dict = gen_vis_feed_dict)
        return generated

    # Generates samples from input batch of codes
    def generate(self, codes):
        generated = np.zeros((codes.shape[0], 32, 32, 3))
        for i in range(codes.shape[0]):
            generated[i:i+1] = self.generate_one_sample(codes[i:i+1])
        return generated

    # Perform activation maximization on one initial code
    def actmax_one_sample(self, initial_code):
        
        ################################################################################
        # Prob 2-4: check this function                                                #
        # skip this part when working on problem 2-1 and come back for problem 2-4     #
        ################################################################################
        actmax_init_val = tf.convert_to_tensor(initial_code, dtype = tf.float32)
        sess.run(self.actmax_code.assign(actmax_init_val))
        for i in range(self.actmax_steps):
            actmax_feed_dict = {
                self.actmax_label: np.ones([1, 1]),
                self.is_train: False
            }
            _, last_actmax = sess.run([self.actmax_op, self.actmax_sample_op], feed_dict = actmax_feed_dict)
        return last_actmax

    # Perform activation maximization on a batch of different initial codes
    def actmax(self, initial_codes):
        actmax_results = np.zeros((initial_codes.shape[0], 32, 32, 3))
        for i in range(initial_codes.shape[0]):
            actmax_results[i:i+1] = self.actmax_one_sample(initial_codes[i:i+1])
        return actmax_results.clip(0, 1)

Now let's do the training!

Non't panic if the loss curve goes wild. The two networks are competing for the loss curve to go different directions, so virtually anything can happen. If your code is correct, the generated samples should have a high variety.



In [5]:

    
tf.reset_default_graph()

with tf.Session() as sess:
    with tf.device('/cpu:0'):
        dcgan = DCGAN()
        sess.run(tf.global_variables_initializer())
        dcgan.train(sess, train_samples)
        dis_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'dis')
        gen_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'gen')
        saver = tf.train.Saver(dis_var_list + gen_var_list)
        saver.save(sess, 'model/dcgan')









    



Iteration 50: dis loss = 0.5519, gen loss = 1.8783
Iteration 100: dis loss = 0.0752, gen loss = 3.8774
Iteration 150: dis loss = 0.1013, gen loss = 2.9514
Iteration 200: dis loss = 0.0588, gen loss = 3.9283
Iteration 250: dis loss = 0.0214, gen loss = 4.9267
Iteration 300: dis loss = 1.0243, gen loss = 0.5989
Iteration 350: dis loss = 0.3685, gen loss = 2.1306
Iteration 400: dis loss = 0.9597, gen loss = 0.9736
Iteration 450: dis loss = 0.7482, gen loss = 1.4007
Iteration 500: dis loss = 0.6420, gen loss = 1.8086
Iteration 550: dis loss = 0.7114, gen loss = 2.4071
Iteration 600: dis loss = 0.6209, gen loss = 1.5227
Iteration 650: dis loss = 0.8241, gen loss = 2.2775
Iteration 700: dis loss = 0.7848, gen loss = 2.8606
Iteration 750: dis loss = 0.6234, gen loss = 2.6968
Iteration 800: dis loss = 0.3681, gen loss = 2.2983
Iteration 850: dis loss = 0.4873, gen loss = 1.3640
Iteration 900: dis loss = 0.8068, gen loss = 0.5832
Iteration 950: dis loss = 0.6186, gen loss = 2.4051
Iteration 1000: dis loss = 0.4259, gen loss = 1.9035
Iteration 1050: dis loss = 1.0225, gen loss = 0.7979
Iteration 1100: dis loss = 1.0580, gen loss = 1.6900
Iteration 1150: dis loss = 1.6054, gen loss = 0.6213
Iteration 1200: dis loss = 1.2335, gen loss = 1.9042
Iteration 1250: dis loss = 1.0292, gen loss = 1.5155
Iteration 1300: dis loss = 1.2015, gen loss = 0.6946
Iteration 1350: dis loss = 1.2039, gen loss = 0.6289
Iteration 1400: dis loss = 1.3315, gen loss = 1.7493
Iteration 1450: dis loss = 1.2837, gen loss = 1.3437
Iteration 1500: dis loss = 1.3243, gen loss = 0.6717
Iteration 1550: dis loss = 1.2795, gen loss = 0.8028






    












    












    












    



Iteration 1600: dis loss = 1.2778, gen loss = 1.3161
Iteration 1650: dis loss = 1.2776, gen loss = 1.4618
Iteration 1700: dis loss = 1.3766, gen loss = 1.1925
Iteration 1750: dis loss = 1.1841, gen loss = 1.2079
Iteration 1800: dis loss = 1.2919, gen loss = 0.9679
Iteration 1850: dis loss = 1.0036, gen loss = 1.0293
Iteration 1900: dis loss = 1.1078, gen loss = 1.3305
Iteration 1950: dis loss = 1.2118, gen loss = 1.1141
Iteration 2000: dis loss = 1.0546, gen loss = 1.0567
Iteration 2050: dis loss = 1.2914, gen loss = 1.0741
Iteration 2100: dis loss = 1.5239, gen loss = 0.5157
Iteration 2150: dis loss = 1.1014, gen loss = 0.9925
Iteration 2200: dis loss = 1.1755, gen loss = 1.5574
Iteration 2250: dis loss = 1.4450, gen loss = 0.8307
Iteration 2300: dis loss = 1.0405, gen loss = 1.1419
Iteration 2350: dis loss = 1.2610, gen loss = 0.6859
Iteration 2400: dis loss = 1.2687, gen loss = 0.7018
Iteration 2450: dis loss = 0.9883, gen loss = 1.0222
Iteration 2500: dis loss = 1.2310, gen loss = 1.0493
Iteration 2550: dis loss = 1.2412, gen loss = 0.4923
Iteration 2600: dis loss = 1.1092, gen loss = 1.0860
Iteration 2650: dis loss = 1.2597, gen loss = 0.5524
Iteration 2700: dis loss = 1.1012, gen loss = 0.8439
Iteration 2750: dis loss = 1.0132, gen loss = 1.1839
Iteration 2800: dis loss = 0.9308, gen loss = 1.2051
Iteration 2850: dis loss = 0.8717, gen loss = 1.2413
Iteration 2900: dis loss = 1.0028, gen loss = 0.7415
Iteration 2950: dis loss = 1.0243, gen loss = 0.9184
Iteration 3000: dis loss = 0.9180, gen loss = 1.4987
Iteration 3050: dis loss = 0.8017, gen loss = 1.0213
Iteration 3100: dis loss = 0.9121, gen loss = 2.0867






    












    












    












    



Iteration 3150: dis loss = 1.1139, gen loss = 2.0534
Iteration 3200: dis loss = 1.0389, gen loss = 0.8384
Iteration 3250: dis loss = 0.9680, gen loss = 1.2628
Iteration 3300: dis loss = 0.7516, gen loss = 1.6942
Iteration 3350: dis loss = 0.9268, gen loss = 0.8615
Iteration 3400: dis loss = 1.0712, gen loss = 0.9476
Iteration 3450: dis loss = 0.8737, gen loss = 1.8674
Iteration 3500: dis loss = 1.1240, gen loss = 2.2074
Iteration 3550: dis loss = 0.9007, gen loss = 1.5221
Iteration 3600: dis loss = 1.0669, gen loss = 0.8262
Iteration 3650: dis loss = 0.5812, gen loss = 1.5648
Iteration 3700: dis loss = 0.9125, gen loss = 1.6559
Iteration 3750: dis loss = 1.0042, gen loss = 1.7220
Iteration 3800: dis loss = 0.8858, gen loss = 1.2294
Iteration 3850: dis loss = 0.8478, gen loss = 1.0318
Iteration 3900: dis loss = 0.8944, gen loss = 0.8754
Iteration 3950: dis loss = 1.2231, gen loss = 0.6501
Iteration 4000: dis loss = 0.9837, gen loss = 2.0138
Iteration 4050: dis loss = 0.9941, gen loss = 0.7801
Iteration 4100: dis loss = 0.8881, gen loss = 1.8829
Iteration 4150: dis loss = 1.1359, gen loss = 0.6495
Iteration 4200: dis loss = 1.0392, gen loss = 0.5494
Iteration 4250: dis loss = 0.8020, gen loss = 1.1564
Iteration 4300: dis loss = 1.2985, gen loss = 1.7977
Iteration 4350: dis loss = 1.1081, gen loss = 1.8425
Iteration 4400: dis loss = 0.9762, gen loss = 1.5689
Iteration 4450: dis loss = 0.8955, gen loss = 0.8102
Iteration 4500: dis loss = 1.4186, gen loss = 2.4216
Iteration 4550: dis loss = 0.7475, gen loss = 1.3930
Iteration 4600: dis loss = 0.8702, gen loss = 1.2817
Iteration 4650: dis loss = 1.0875, gen loss = 1.1356






    












    












    












    



Iteration 4700: dis loss = 1.5080, gen loss = 0.5214
Iteration 4750: dis loss = 0.9389, gen loss = 1.4476
Iteration 4800: dis loss = 1.0481, gen loss = 1.8110
Iteration 4850: dis loss = 0.9942, gen loss = 1.6410
Iteration 4900: dis loss = 0.8130, gen loss = 1.8838
Iteration 4950: dis loss = 0.9987, gen loss = 0.6462
Iteration 5000: dis loss = 0.8289, gen loss = 1.4590
Iteration 5050: dis loss = 0.8441, gen loss = 1.0418
Iteration 5100: dis loss = 0.7773, gen loss = 1.9306
Iteration 5150: dis loss = 0.9678, gen loss = 0.8393
Iteration 5200: dis loss = 1.1231, gen loss = 0.4680
Iteration 5250: dis loss = 0.9768, gen loss = 1.3402
Iteration 5300: dis loss = 0.8473, gen loss = 1.4123
Iteration 5350: dis loss = 1.2014, gen loss = 2.2932
Iteration 5400: dis loss = 0.8385, gen loss = 1.9112
Iteration 5450: dis loss = 1.0460, gen loss = 2.2558
Iteration 5500: dis loss = 0.8349, gen loss = 1.1654
Iteration 5550: dis loss = 1.0794, gen loss = 0.5908
Iteration 5600: dis loss = 0.9664, gen loss = 2.1734
Iteration 5650: dis loss = 1.0062, gen loss = 1.5558
Iteration 5700: dis loss = 1.0937, gen loss = 2.4293
Iteration 5750: dis loss = 0.8509, gen loss = 2.0602
Iteration 5800: dis loss = 0.9173, gen loss = 1.2247
Iteration 5850: dis loss = 0.8019, gen loss = 1.6375
Iteration 5900: dis loss = 1.1034, gen loss = 2.6638
Iteration 5950: dis loss = 0.8920, gen loss = 0.8787
Iteration 6000: dis loss = 0.8884, gen loss = 0.8757
Iteration 6050: dis loss = 0.9565, gen loss = 1.0224
Iteration 6100: dis loss = 0.9533, gen loss = 1.0496
Iteration 6150: dis loss = 0.7186, gen loss = 1.4092
Iteration 6200: dis loss = 0.7819, gen loss = 1.1224






    












    












    












    



Iteration 6250: dis loss = 1.1054, gen loss = 0.8771
Iteration 6300: dis loss = 0.9879, gen loss = 0.7626
Iteration 6350: dis loss = 0.9324, gen loss = 0.7523
Iteration 6400: dis loss = 0.8198, gen loss = 1.8954
Iteration 6450: dis loss = 1.0386, gen loss = 0.7446
Iteration 6500: dis loss = 1.0564, gen loss = 0.6972
Iteration 6550: dis loss = 0.9876, gen loss = 2.1176
Iteration 6600: dis loss = 1.0043, gen loss = 0.8638
Iteration 6650: dis loss = 1.0508, gen loss = 1.3462
Iteration 6700: dis loss = 0.8944, gen loss = 1.5725
Iteration 6750: dis loss = 0.9989, gen loss = 0.9075
Iteration 6800: dis loss = 1.2801, gen loss = 1.7196
Iteration 6850: dis loss = 0.9757, gen loss = 1.1779
Iteration 6900: dis loss = 0.9508, gen loss = 1.6972
Iteration 6950: dis loss = 0.8341, gen loss = 1.1551
Iteration 7000: dis loss = 0.9705, gen loss = 0.9790
Iteration 7050: dis loss = 1.0089, gen loss = 0.9115
Iteration 7100: dis loss = 1.1896, gen loss = 1.6422
Iteration 7150: dis loss = 1.1636, gen loss = 0.7890
Iteration 7200: dis loss = 0.8169, gen loss = 1.6857
Iteration 7250: dis loss = 1.0005, gen loss = 0.9146
Iteration 7300: dis loss = 0.9382, gen loss = 1.0031
Iteration 7350: dis loss = 0.7523, gen loss = 1.0837
Iteration 7400: dis loss = 0.8679, gen loss = 1.7017
Iteration 7450: dis loss = 1.0393, gen loss = 0.6381
Iteration 7500: dis loss = 1.2441, gen loss = 0.9995
Iteration 7550: dis loss = 1.4170, gen loss = 1.8455
Iteration 7600: dis loss = 0.8913, gen loss = 1.5308
Iteration 7650: dis loss = 1.3124, gen loss = 0.6471
Iteration 7700: dis loss = 1.0620, gen loss = 0.6064
Iteration 7750: dis loss = 0.9147, gen loss = 1.0016
Iteration 7800: dis loss = 1.2610, gen loss = 1.8008






    












    












    












    



Iteration 7850: dis loss = 1.4514, gen loss = 0.4911
Iteration 7900: dis loss = 1.0685, gen loss = 0.8157
Iteration 7950: dis loss = 0.9061, gen loss = 1.2989
Iteration 8000: dis loss = 1.1283, gen loss = 2.4356
Iteration 8050: dis loss = 1.0995, gen loss = 2.1129
Iteration 8100: dis loss = 1.2887, gen loss = 0.5098
Iteration 8150: dis loss = 1.1321, gen loss = 0.7589
Iteration 8200: dis loss = 1.2808, gen loss = 1.6828
Iteration 8250: dis loss = 0.9137, gen loss = 1.3652
Iteration 8300: dis loss = 1.2078, gen loss = 0.6447
Iteration 8350: dis loss = 0.9342, gen loss = 1.2447
Iteration 8400: dis loss = 0.9655, gen loss = 0.8761
Iteration 8450: dis loss = 1.0593, gen loss = 0.7547
Iteration 8500: dis loss = 1.7422, gen loss = 0.5819
Iteration 8550: dis loss = 1.0496, gen loss = 0.6761
Iteration 8600: dis loss = 0.7454, gen loss = 1.5700
Iteration 8650: dis loss = 0.9923, gen loss = 0.9555
Iteration 8700: dis loss = 1.0411, gen loss = 2.1178
Iteration 8750: dis loss = 1.1062, gen loss = 0.9070
Iteration 8800: dis loss = 0.8774, gen loss = 2.4191
Iteration 8850: dis loss = 1.0522, gen loss = 0.5294
Iteration 8900: dis loss = 1.0549, gen loss = 0.9826
Iteration 8950: dis loss = 1.3889, gen loss = 0.3549
Iteration 9000: dis loss = 0.7207, gen loss = 1.1777
Iteration 9050: dis loss = 1.0098, gen loss = 0.9819
Iteration 9100: dis loss = 0.9821, gen loss = 0.9718
Iteration 9150: dis loss = 0.7871, gen loss = 1.7630
Iteration 9200: dis loss = 0.9771, gen loss = 2.0123
Iteration 9250: dis loss = 1.1061, gen loss = 0.5712
Iteration 9300: dis loss = 0.7820, gen loss = 1.3796
Iteration 9350: dis loss = 1.3371, gen loss = 2.4622






    












    












    












    



Iteration 9400: dis loss = 1.1918, gen loss = 0.7610
Iteration 9450: dis loss = 1.1978, gen loss = 1.9324
Iteration 9500: dis loss = 0.8002, gen loss = 1.0723
Iteration 9550: dis loss = 1.0793, gen loss = 0.9369
Iteration 9600: dis loss = 0.8012, gen loss = 1.1199
Iteration 9650: dis loss = 0.8926, gen loss = 2.0727
Iteration 9700: dis loss = 0.8169, gen loss = 0.9866
Iteration 9750: dis loss = 0.6840, gen loss = 2.0669
Iteration 9800: dis loss = 1.0378, gen loss = 1.3561
Iteration 9850: dis loss = 1.0511, gen loss = 0.8414
Iteration 9900: dis loss = 0.8819, gen loss = 0.8073
Iteration 9950: dis loss = 1.1222, gen loss = 1.2119
Iteration 10000: dis loss = 1.2779, gen loss = 0.4081
Iteration 10050: dis loss = 1.0996, gen loss = 2.0202
Iteration 10100: dis loss = 1.0400, gen loss = 1.2346
Iteration 10150: dis loss = 1.1103, gen loss = 0.9642
Iteration 10200: dis loss = 1.2469, gen loss = 1.7167
Iteration 10250: dis loss = 0.9542, gen loss = 1.2360
Iteration 10300: dis loss = 0.8353, gen loss = 1.2177
Iteration 10350: dis loss = 1.1024, gen loss = 1.5963
Iteration 10400: dis loss = 1.2225, gen loss = 0.5095
Iteration 10450: dis loss = 0.9521, gen loss = 1.7917
Iteration 10500: dis loss = 0.9175, gen loss = 1.4591
Iteration 10550: dis loss = 0.8137, gen loss = 1.3352
Iteration 10600: dis loss = 0.8035, gen loss = 1.1756
Iteration 10650: dis loss = 1.1381, gen loss = 1.1926
Iteration 10700: dis loss = 0.7608, gen loss = 1.7188
Iteration 10750: dis loss = 0.9602, gen loss = 1.2396
Iteration 10800: dis loss = 1.2680, gen loss = 1.8488
Iteration 10850: dis loss = 1.0019, gen loss = 1.5973
Iteration 10900: dis loss = 1.0378, gen loss = 1.0681






    












    












    












    



Iteration 10950: dis loss = 1.0340, gen loss = 1.3919
Iteration 11000: dis loss = 1.2661, gen loss = 0.6048
Iteration 11050: dis loss = 0.9850, gen loss = 0.9786
Iteration 11100: dis loss = 1.1222, gen loss = 2.2137
Iteration 11150: dis loss = 1.1917, gen loss = 0.9038
Iteration 11200: dis loss = 0.9560, gen loss = 0.9739
Iteration 11250: dis loss = 0.6590, gen loss = 1.4138
Iteration 11300: dis loss = 0.8377, gen loss = 1.1793
Iteration 11350: dis loss = 1.0538, gen loss = 1.8770
Iteration 11400: dis loss = 0.8616, gen loss = 1.1006
Iteration 11450: dis loss = 1.1496, gen loss = 0.7115
Iteration 11500: dis loss = 0.8690, gen loss = 1.5121
Iteration 11550: dis loss = 1.0613, gen loss = 2.1134
Iteration 11600: dis loss = 0.9125, gen loss = 0.9023
Iteration 11650: dis loss = 1.1245, gen loss = 0.7684
Iteration 11700: dis loss = 1.1025, gen loss = 0.7971
Iteration 11750: dis loss = 1.0465, gen loss = 2.0606
Iteration 11800: dis loss = 0.8901, gen loss = 1.0215
Iteration 11850: dis loss = 0.8381, gen loss = 1.1783
Iteration 11900: dis loss = 1.0522, gen loss = 1.5685
Iteration 11950: dis loss = 0.8859, gen loss = 0.9737
Iteration 12000: dis loss = 0.9433, gen loss = 1.2130
Iteration 12050: dis loss = 1.1597, gen loss = 2.4359
Iteration 12100: dis loss = 0.9468, gen loss = 1.3711
Iteration 12150: dis loss = 0.8349, gen loss = 1.1550
Iteration 12200: dis loss = 1.2955, gen loss = 2.9516
Iteration 12250: dis loss = 0.9360, gen loss = 2.3203
Iteration 12300: dis loss = 1.1062, gen loss = 0.5912
Iteration 12350: dis loss = 1.0234, gen loss = 0.8367
Iteration 12400: dis loss = 0.8836, gen loss = 1.8286
Iteration 12450: dis loss = 1.1602, gen loss = 2.3970






    












    












    












    



Iteration 12500: dis loss = 0.9812, gen loss = 1.1322
Iteration 12550: dis loss = 0.8202, gen loss = 1.0408
Iteration 12600: dis loss = 1.0682, gen loss = 0.7118
Iteration 12650: dis loss = 1.1250, gen loss = 1.6356
Iteration 12700: dis loss = 1.1159, gen loss = 1.8741
Iteration 12750: dis loss = 0.8378, gen loss = 1.7091
Iteration 12800: dis loss = 0.9321, gen loss = 1.4802
Iteration 12850: dis loss = 0.8633, gen loss = 1.1381
Iteration 12900: dis loss = 1.4456, gen loss = 3.2283
Iteration 12950: dis loss = 1.1716, gen loss = 1.6815
Iteration 13000: dis loss = 0.9646, gen loss = 0.7284
Iteration 13050: dis loss = 0.8990, gen loss = 1.8764
Iteration 13100: dis loss = 1.0456, gen loss = 2.0678
Iteration 13150: dis loss = 0.9127, gen loss = 1.9920
Iteration 13200: dis loss = 0.8939, gen loss = 0.7303
Iteration 13250: dis loss = 1.1776, gen loss = 0.6926
Iteration 13300: dis loss = 1.1385, gen loss = 0.5214
Iteration 13350: dis loss = 0.8330, gen loss = 1.6506
Iteration 13400: dis loss = 1.1123, gen loss = 0.4458
Iteration 13450: dis loss = 1.3550, gen loss = 2.6086
Iteration 13500: dis loss = 1.4449, gen loss = 0.4720
Iteration 13550: dis loss = 0.8701, gen loss = 1.5055
Iteration 13600: dis loss = 1.1334, gen loss = 0.6253
Iteration 13650: dis loss = 0.7985, gen loss = 1.4811
Iteration 13700: dis loss = 1.0544, gen loss = 0.9184
Iteration 13750: dis loss = 1.1992, gen loss = 2.4512
Iteration 13800: dis loss = 0.8359, gen loss = 1.6861
Iteration 13850: dis loss = 1.0501, gen loss = 2.0474
Iteration 13900: dis loss = 0.8779, gen loss = 1.3385
Iteration 13950: dis loss = 0.8620, gen loss = 1.0876
Iteration 14000: dis loss = 0.8637, gen loss = 1.6565
Iteration 14050: dis loss = 1.0720, gen loss = 1.9502






    












    












    












    



Iteration 14100: dis loss = 1.0043, gen loss = 0.9119
Iteration 14150: dis loss = 0.9812, gen loss = 1.0054
Iteration 14200: dis loss = 1.3224, gen loss = 0.6502
Iteration 14250: dis loss = 0.8509, gen loss = 2.0974
Iteration 14300: dis loss = 0.8458, gen loss = 2.5666
Iteration 14350: dis loss = 0.8406, gen loss = 2.1648
Iteration 14400: dis loss = 0.8920, gen loss = 1.0997
Iteration 14450: dis loss = 0.7826, gen loss = 1.4266
Iteration 14500: dis loss = 1.1486, gen loss = 2.1299
Iteration 14550: dis loss = 0.7541, gen loss = 1.0448
Iteration 14600: dis loss = 0.8336, gen loss = 1.2344
Iteration 14650: dis loss = 1.0271, gen loss = 0.8047
Iteration 14700: dis loss = 1.0485, gen loss = 2.2004
Iteration 14750: dis loss = 0.9664, gen loss = 0.8901
Iteration 14800: dis loss = 0.6007, gen loss = 1.4490
Iteration 14850: dis loss = 0.7252, gen loss = 1.8077
Iteration 14900: dis loss = 0.9607, gen loss = 1.0556
Iteration 14950: dis loss = 0.9929, gen loss = 1.6590
Iteration 15000: dis loss = 0.8517, gen loss = 1.0463
Iteration 15050: dis loss = 0.7178, gen loss = 1.5073
Iteration 15100: dis loss = 0.9583, gen loss = 1.7863
Iteration 15150: dis loss = 1.1674, gen loss = 2.0693
Iteration 15200: dis loss = 1.2746, gen loss = 0.8666
Iteration 15250: dis loss = 1.0119, gen loss = 0.8245
Iteration 15300: dis loss = 0.8679, gen loss = 0.7629
Iteration 15350: dis loss = 0.9886, gen loss = 0.9719
Iteration 15400: dis loss = 0.9896, gen loss = 2.0984
Iteration 15450: dis loss = 0.9797, gen loss = 1.0781
Iteration 15500: dis loss = 0.7516, gen loss = 1.0426
Iteration 15550: dis loss = 0.7770, gen loss = 1.9499
Iteration 15600: dis loss = 1.0567, gen loss = 1.7198






    












    












    












    



Iteration 15650: dis loss = 1.2440, gen loss = 0.8299
Iteration 15700: dis loss = 0.8769, gen loss = 2.3804
Iteration 15750: dis loss = 0.7893, gen loss = 0.8786
Iteration 15800: dis loss = 0.8429, gen loss = 1.9301
Iteration 15850: dis loss = 0.5907, gen loss = 1.4704
Iteration 15900: dis loss = 1.2822, gen loss = 2.3902
Iteration 15950: dis loss = 0.7054, gen loss = 2.2396
Iteration 16000: dis loss = 0.9266, gen loss = 1.9518
Iteration 16050: dis loss = 1.3144, gen loss = 2.2529
Iteration 16100: dis loss = 0.6632, gen loss = 1.8549
Iteration 16150: dis loss = 0.9533, gen loss = 1.7878
Iteration 16200: dis loss = 1.0556, gen loss = 1.1497
Iteration 16250: dis loss = 0.7700, gen loss = 1.3037
Iteration 16300: dis loss = 0.8252, gen loss = 0.9955
Iteration 16350: dis loss = 0.8315, gen loss = 2.3467
Iteration 16400: dis loss = 0.5475, gen loss = 2.5575
Iteration 16450: dis loss = 0.8032, gen loss = 1.3545
Iteration 16500: dis loss = 0.7127, gen loss = 1.7740
Iteration 16550: dis loss = 0.7811, gen loss = 1.2768
Iteration 16600: dis loss = 0.8088, gen loss = 2.0153
Iteration 16650: dis loss = 0.5764, gen loss = 1.6689
Iteration 16700: dis loss = 0.7352, gen loss = 2.1702
Iteration 16750: dis loss = 0.7376, gen loss = 2.1823
Iteration 16800: dis loss = 0.8139, gen loss = 1.3168
Iteration 16850: dis loss = 0.8009, gen loss = 1.3577
Iteration 16900: dis loss = 0.6520, gen loss = 1.8072
Iteration 16950: dis loss = 0.9472, gen loss = 1.3191
Iteration 17000: dis loss = 0.7112, gen loss = 1.5714
Iteration 17050: dis loss = 1.2371, gen loss = 1.5470
Iteration 17100: dis loss = 1.1445, gen loss = 1.4795
Iteration 17150: dis loss = 0.6653, gen loss = 1.3270






    












    












    












    



Iteration 17200: dis loss = 0.7741, gen loss = 1.1781
Iteration 17250: dis loss = 1.1096, gen loss = 1.9087
Iteration 17300: dis loss = 0.8186, gen loss = 1.7059
Iteration 17350: dis loss = 1.0707, gen loss = 0.8451
Iteration 17400: dis loss = 0.7128, gen loss = 1.8926
Iteration 17450: dis loss = 0.7193, gen loss = 1.6214
Iteration 17500: dis loss = 0.9334, gen loss = 2.6027
Iteration 17550: dis loss = 0.8857, gen loss = 1.1047
Iteration 17600: dis loss = 0.7625, gen loss = 2.6730
Iteration 17650: dis loss = 0.6258, gen loss = 2.4436
Iteration 17700: dis loss = 0.5574, gen loss = 1.7098
Iteration 17750: dis loss = 1.1394, gen loss = 2.6354
Iteration 17800: dis loss = 1.0640, gen loss = 2.7942
Iteration 17850: dis loss = 0.8242, gen loss = 1.1576
Iteration 17900: dis loss = 0.8646, gen loss = 0.6452
Iteration 17950: dis loss = 0.7859, gen loss = 1.3571
Iteration 18000: dis loss = 1.1033, gen loss = 2.0893
Iteration 18050: dis loss = 0.8135, gen loss = 1.3875
Iteration 18100: dis loss = 0.8004, gen loss = 1.4828
Iteration 18150: dis loss = 0.8170, gen loss = 1.4295
Iteration 18200: dis loss = 1.1616, gen loss = 0.7246
Iteration 18250: dis loss = 0.6958, gen loss = 1.1519
Iteration 18300: dis loss = 0.7461, gen loss = 0.9148
Iteration 18350: dis loss = 0.8807, gen loss = 0.8042
Iteration 18400: dis loss = 0.7436, gen loss = 1.1732
Iteration 18450: dis loss = 1.1955, gen loss = 0.6440
Iteration 18500: dis loss = 0.8488, gen loss = 1.6010
Iteration 18550: dis loss = 0.6040, gen loss = 1.4991
Iteration 18600: dis loss = 0.5964, gen loss = 1.8557
Iteration 18650: dis loss = 0.7204, gen loss = 2.2167
Iteration 18700: dis loss = 0.8969, gen loss = 2.9930






    












    












    












    



Iteration 18750: dis loss = 0.7635, gen loss = 1.3794
Iteration 18800: dis loss = 0.7024, gen loss = 2.2154
Iteration 18850: dis loss = 0.7323, gen loss = 1.3072
Iteration 18900: dis loss = 0.8055, gen loss = 1.5784
Iteration 18950: dis loss = 0.8553, gen loss = 2.2203
Iteration 19000: dis loss = 0.7274, gen loss = 2.1348
Iteration 19050: dis loss = 0.6201, gen loss = 1.6590
Iteration 19100: dis loss = 0.9027, gen loss = 2.8270
Iteration 19150: dis loss = 0.7857, gen loss = 1.3051
Iteration 19200: dis loss = 1.1787, gen loss = 0.5276
Iteration 19250: dis loss = 1.0309, gen loss = 0.6743
Iteration 19300: dis loss = 0.7038, gen loss = 1.2944
Iteration 19350: dis loss = 0.7573, gen loss = 1.2878
Iteration 19400: dis loss = 0.7925, gen loss = 1.3058
Iteration 19450: dis loss = 0.8252, gen loss = 2.7921
Iteration 19500: dis loss = 1.0174, gen loss = 0.9941
Iteration 19550: dis loss = 0.8676, gen loss = 1.5196
Iteration 19600: dis loss = 0.6843, gen loss = 1.9561
Iteration 19650: dis loss = 0.8170, gen loss = 1.0429
Iteration 19700: dis loss = 0.5035, gen loss = 2.0808
Iteration 19750: dis loss = 0.7598, gen loss = 1.3145
Iteration 19800: dis loss = 1.1399, gen loss = 2.0927
Iteration 19850: dis loss = 1.2300, gen loss = 0.7000
Iteration 19900: dis loss = 0.7512, gen loss = 1.8705
Iteration 19950: dis loss = 0.5184, gen loss = 1.8057
Iteration 20000: dis loss = 0.5434, gen loss = 1.6478
Iteration 20050: dis loss = 0.7966, gen loss = 1.1607
Iteration 20100: dis loss = 1.0293, gen loss = 1.7881
Iteration 20150: dis loss = 0.9658, gen loss = 2.1835
Iteration 20200: dis loss = 0.8434, gen loss = 3.2934
Iteration 20250: dis loss = 0.7976, gen loss = 1.1377
Iteration 20300: dis loss = 0.8402, gen loss = 2.1064






    












    












    












    



Iteration 20350: dis loss = 0.8215, gen loss = 1.0565
Iteration 20400: dis loss = 0.7894, gen loss = 2.3604
Iteration 20450: dis loss = 0.6830, gen loss = 2.0008
Iteration 20500: dis loss = 0.6628, gen loss = 2.1899
Iteration 20550: dis loss = 0.6614, gen loss = 2.0576
Iteration 20600: dis loss = 0.3258, gen loss = 2.2534
Iteration 20650: dis loss = 0.4617, gen loss = 1.8292
Iteration 20700: dis loss = 0.9236, gen loss = 1.0053
Iteration 20750: dis loss = 0.6563, gen loss = 1.8303
Iteration 20800: dis loss = 0.7676, gen loss = 2.0942
Iteration 20850: dis loss = 0.7066, gen loss = 1.3928
Iteration 20900: dis loss = 0.7490, gen loss = 1.4631
Iteration 20950: dis loss = 0.7749, gen loss = 2.4496
Iteration 21000: dis loss = 0.5593, gen loss = 1.4074
Iteration 21050: dis loss = 0.9368, gen loss = 0.7016
Iteration 21100: dis loss = 0.8890, gen loss = 2.5232
Iteration 21150: dis loss = 0.7476, gen loss = 2.7064
Iteration 21200: dis loss = 0.9339, gen loss = 1.1518
Iteration 21250: dis loss = 1.0436, gen loss = 1.2739
Iteration 21300: dis loss = 0.7751, gen loss = 2.2857
Iteration 21350: dis loss = 1.0135, gen loss = 2.9287
Iteration 21400: dis loss = 0.7096, gen loss = 2.1253
Iteration 21450: dis loss = 0.6398, gen loss = 2.2276
Iteration 21500: dis loss = 0.5528, gen loss = 1.5360
Iteration 21550: dis loss = 0.9982, gen loss = 1.0638
Iteration 21600: dis loss = 0.9103, gen loss = 0.8817
Iteration 21650: dis loss = 0.6209, gen loss = 2.7516
Iteration 21700: dis loss = 0.7226, gen loss = 1.4067
Iteration 21750: dis loss = 0.6515, gen loss = 2.4116
Iteration 21800: dis loss = 0.6906, gen loss = 1.1218
Iteration 21850: dis loss = 0.9917, gen loss = 1.8265






    












    












    












    



Iteration 21900: dis loss = 0.5813, gen loss = 2.0955
Iteration 21950: dis loss = 0.7421, gen loss = 1.5152
Iteration 22000: dis loss = 0.8596, gen loss = 2.6139
Iteration 22050: dis loss = 0.8330, gen loss = 1.3573
Iteration 22100: dis loss = 0.9541, gen loss = 2.2140
Iteration 22150: dis loss = 0.8453, gen loss = 0.8198
Iteration 22200: dis loss = 0.7402, gen loss = 2.1225
Iteration 22250: dis loss = 0.8259, gen loss = 2.3354
Iteration 22300: dis loss = 0.9354, gen loss = 2.6343
Iteration 22350: dis loss = 0.7514, gen loss = 1.3893
Iteration 22400: dis loss = 1.0815, gen loss = 2.3525
Iteration 22450: dis loss = 0.9937, gen loss = 0.7405
Iteration 22500: dis loss = 0.9314, gen loss = 2.2924
Iteration 22550: dis loss = 0.5185, gen loss = 1.6642
Iteration 22600: dis loss = 0.5859, gen loss = 1.8361
Iteration 22650: dis loss = 1.1772, gen loss = 3.2495
Iteration 22700: dis loss = 0.4657, gen loss = 2.4554
Iteration 22750: dis loss = 0.7691, gen loss = 2.5794
Iteration 22800: dis loss = 0.8104, gen loss = 0.9724
Iteration 22850: dis loss = 0.9436, gen loss = 2.2476
Iteration 22900: dis loss = 0.6304, gen loss = 1.8514
Iteration 22950: dis loss = 0.6277, gen loss = 2.2485
Iteration 23000: dis loss = 0.9052, gen loss = 2.5996
Iteration 23050: dis loss = 0.7520, gen loss = 2.7938
Iteration 23100: dis loss = 0.5812, gen loss = 2.3404
Iteration 23150: dis loss = 0.6937, gen loss = 2.0151
Iteration 23200: dis loss = 0.7339, gen loss = 1.4591
Iteration 23250: dis loss = 0.7490, gen loss = 2.0994
Iteration 23300: dis loss = 0.8145, gen loss = 1.3966
Iteration 23350: dis loss = 0.7417, gen loss = 1.0565
Iteration 23400: dis loss = 0.6561, gen loss = 3.0404






    












    












    












    



Iteration 23450: dis loss = 0.5190, gen loss = 1.7959
Iteration 23500: dis loss = 0.8144, gen loss = 2.0980
Iteration 23550: dis loss = 0.6136, gen loss = 2.8639
Iteration 23600: dis loss = 0.5031, gen loss = 2.2689
Iteration 23650: dis loss = 0.6037, gen loss = 2.6066
Iteration 23700: dis loss = 1.0524, gen loss = 2.7450
Iteration 23750: dis loss = 0.8997, gen loss = 1.4319
Iteration 23800: dis loss = 0.4656, gen loss = 2.5969
Iteration 23850: dis loss = 0.6348, gen loss = 2.7996
Iteration 23900: dis loss = 0.4571, gen loss = 2.5920
Iteration 23950: dis loss = 0.6478, gen loss = 2.0807
Iteration 24000: dis loss = 0.6249, gen loss = 1.8170
Iteration 24050: dis loss = 0.6038, gen loss = 1.5373
Iteration 24100: dis loss = 0.6391, gen loss = 2.4272
Iteration 24150: dis loss = 0.6533, gen loss = 1.4332
Iteration 24200: dis loss = 0.6345, gen loss = 1.4662
Iteration 24250: dis loss = 0.6073, gen loss = 1.9992
Iteration 24300: dis loss = 1.0240, gen loss = 3.2418
Iteration 24350: dis loss = 0.9176, gen loss = 1.8456
Iteration 24400: dis loss = 0.6204, gen loss = 1.5107
Iteration 24450: dis loss = 1.0283, gen loss = 1.0869
Iteration 24500: dis loss = 0.2915, gen loss = 3.2869
Iteration 24550: dis loss = 0.7502, gen loss = 1.2569
Iteration 24600: dis loss = 0.6203, gen loss = 1.4715
Iteration 24650: dis loss = 0.7415, gen loss = 2.8026
Iteration 24700: dis loss = 0.9823, gen loss = 2.8603
Iteration 24750: dis loss = 0.5381, gen loss = 1.9296
Iteration 24800: dis loss = 0.4858, gen loss = 1.7340
Iteration 24850: dis loss = 0.3615, gen loss = 3.0730
Iteration 24900: dis loss = 0.7231, gen loss = 1.0378
Iteration 24950: dis loss = 0.7001, gen loss = 2.5760






    












    












    












    



Iteration 25000: dis loss = 0.5390, gen loss = 2.0361
Iteration 25050: dis loss = 0.5843, gen loss = 1.5722
Iteration 25100: dis loss = 0.7155, gen loss = 1.5129
Iteration 25150: dis loss = 0.8602, gen loss = 0.9269
Iteration 25200: dis loss = 0.6874, gen loss = 2.3083
Iteration 25250: dis loss = 0.8642, gen loss = 1.3035
Iteration 25300: dis loss = 0.9162, gen loss = 2.8817
Iteration 25350: dis loss = 0.5722, gen loss = 2.3228
Iteration 25400: dis loss = 0.8354, gen loss = 1.8264
Iteration 25450: dis loss = 0.6296, gen loss = 1.6844
Iteration 25500: dis loss = 0.9508, gen loss = 0.6360
Iteration 25550: dis loss = 0.6358, gen loss = 2.8564
Iteration 25600: dis loss = 0.7308, gen loss = 2.4138
Iteration 25650: dis loss = 0.8103, gen loss = 2.0738
Iteration 25700: dis loss = 0.4742, gen loss = 1.8319
Iteration 25750: dis loss = 0.5988, gen loss = 1.8587
Iteration 25800: dis loss = 0.4790, gen loss = 2.1018
Iteration 25850: dis loss = 0.6585, gen loss = 2.6037
Iteration 25900: dis loss = 0.6164, gen loss = 1.7599
Iteration 25950: dis loss = 0.7326, gen loss = 2.2438
Iteration 26000: dis loss = 0.7119, gen loss = 1.9282
Iteration 26050: dis loss = 0.6364, gen loss = 2.2691
Iteration 26100: dis loss = 0.8048, gen loss = 1.0778
Iteration 26150: dis loss = 0.7562, gen loss = 3.3121
Iteration 26200: dis loss = 1.1074, gen loss = 1.0334
Iteration 26250: dis loss = 0.8441, gen loss = 2.7308
Iteration 26300: dis loss = 0.8057, gen loss = 1.0836
Iteration 26350: dis loss = 0.5265, gen loss = 1.9207
Iteration 26400: dis loss = 0.4451, gen loss = 2.8284
Iteration 26450: dis loss = 1.2199, gen loss = 3.3692
Iteration 26500: dis loss = 0.6218, gen loss = 3.3709
Iteration 26550: dis loss = 0.6639, gen loss = 1.5004






    












    












    












    



Iteration 26600: dis loss = 0.8137, gen loss = 0.9840
Iteration 26650: dis loss = 0.5298, gen loss = 2.2150
Iteration 26700: dis loss = 0.5683, gen loss = 1.7290
Iteration 26750: dis loss = 0.5634, gen loss = 1.9658
Iteration 26800: dis loss = 0.6804, gen loss = 2.7363
Iteration 26850: dis loss = 1.1590, gen loss = 0.5322
Iteration 26900: dis loss = 0.9682, gen loss = 2.9978
Iteration 26950: dis loss = 0.5044, gen loss = 1.8485
Iteration 27000: dis loss = 0.5332, gen loss = 1.8209
Iteration 27050: dis loss = 0.6574, gen loss = 1.3948
Iteration 27100: dis loss = 1.0227, gen loss = 3.5294
Iteration 27150: dis loss = 0.4352, gen loss = 2.0307
Iteration 27200: dis loss = 0.9191, gen loss = 0.6550
Iteration 27250: dis loss = 0.3973, gen loss = 1.9550
Iteration 27300: dis loss = 0.4916, gen loss = 3.0342
Iteration 27350: dis loss = 0.5918, gen loss = 1.7018
Iteration 27400: dis loss = 0.5636, gen loss = 2.7817
Iteration 27450: dis loss = 0.9600, gen loss = 1.0145
Iteration 27500: dis loss = 0.4552, gen loss = 2.2689
Iteration 27550: dis loss = 0.6997, gen loss = 1.6240
Iteration 27600: dis loss = 0.5244, gen loss = 2.3719
Iteration 27650: dis loss = 0.9242, gen loss = 2.5465
Iteration 27700: dis loss = 0.8479, gen loss = 1.3189
Iteration 27750: dis loss = 0.7623, gen loss = 0.9194
Iteration 27800: dis loss = 1.0354, gen loss = 0.9809
Iteration 27850: dis loss = 0.4776, gen loss = 1.6839
Iteration 27900: dis loss = 0.8252, gen loss = 1.4531
Iteration 27950: dis loss = 0.8575, gen loss = 1.0526
Iteration 28000: dis loss = 0.5062, gen loss = 2.4192
Iteration 28050: dis loss = 0.4174, gen loss = 2.3267
Iteration 28100: dis loss = 0.6181, gen loss = 1.4900






    












    












    












    



Iteration 28150: dis loss = 1.6775, gen loss = 3.8245
Iteration 28200: dis loss = 0.6539, gen loss = 2.3494
Iteration 28250: dis loss = 0.9509, gen loss = 1.3939
Iteration 28300: dis loss = 0.6455, gen loss = 1.5662
Iteration 28350: dis loss = 0.6149, gen loss = 1.6227
Iteration 28400: dis loss = 0.3283, gen loss = 2.1920
Iteration 28450: dis loss = 0.7826, gen loss = 1.1047
Iteration 28500: dis loss = 0.4148, gen loss = 2.3514
Iteration 28550: dis loss = 0.4688, gen loss = 2.3184
Iteration 28600: dis loss = 0.7297, gen loss = 0.9346
Iteration 28650: dis loss = 0.5381, gen loss = 2.2124
Iteration 28700: dis loss = 0.6142, gen loss = 1.5542
Iteration 28750: dis loss = 1.3409, gen loss = 0.6184
Iteration 28800: dis loss = 0.6771, gen loss = 2.7739
Iteration 28850: dis loss = 0.4951, gen loss = 2.5600
Iteration 28900: dis loss = 0.7299, gen loss = 2.3935
Iteration 28950: dis loss = 0.7603, gen loss = 1.2670
Iteration 29000: dis loss = 0.6039, gen loss = 1.7802
Iteration 29050: dis loss = 0.5623, gen loss = 1.9800
Iteration 29100: dis loss = 0.5970, gen loss = 2.2764
Iteration 29150: dis loss = 0.7846, gen loss = 2.2947
Iteration 29200: dis loss = 0.9217, gen loss = 3.8231
Iteration 29250: dis loss = 0.5303, gen loss = 1.5264
Iteration 29300: dis loss = 1.2544, gen loss = 3.9273
Iteration 29350: dis loss = 0.9188, gen loss = 1.6280
Iteration 29400: dis loss = 0.4258, gen loss = 3.0475
Iteration 29450: dis loss = 0.9044, gen loss = 0.7544
Iteration 29500: dis loss = 0.5726, gen loss = 2.7956
Iteration 29550: dis loss = 0.5468, gen loss = 1.6312
Iteration 29600: dis loss = 0.6653, gen loss = 1.6056
Iteration 29650: dis loss = 0.7389, gen loss = 1.2648






    












    












    












    



Iteration 29700: dis loss = 0.7146, gen loss = 2.8308
Iteration 29750: dis loss = 0.6495, gen loss = 3.2032
Iteration 29800: dis loss = 0.6382, gen loss = 2.6098
Iteration 29850: dis loss = 0.7649, gen loss = 1.9283
Iteration 29900: dis loss = 0.5299, gen loss = 2.0207
Iteration 29950: dis loss = 0.8184, gen loss = 1.0991
Iteration 30000: dis loss = 0.4269, gen loss = 2.1002
Iteration 30050: dis loss = 0.5291, gen loss = 1.6293
Iteration 30100: dis loss = 0.7503, gen loss = 3.4873
Iteration 30150: dis loss = 0.5291, gen loss = 1.9349
Iteration 30200: dis loss = 0.6553, gen loss = 3.6857
Iteration 30250: dis loss = 0.6428, gen loss = 3.7815
Iteration 30300: dis loss = 0.8514, gen loss = 2.8918
Iteration 30350: dis loss = 1.3759, gen loss = 4.1466
Iteration 30400: dis loss = 0.8142, gen loss = 1.1225
Iteration 30450: dis loss = 0.5384, gen loss = 2.0968
Iteration 30500: dis loss = 0.5565, gen loss = 2.7268
Iteration 30550: dis loss = 0.6003, gen loss = 3.2038
Iteration 30600: dis loss = 0.5131, gen loss = 2.5599
Iteration 30650: dis loss = 0.9578, gen loss = 3.7491
Iteration 30700: dis loss = 0.8120, gen loss = 1.4457
Iteration 30750: dis loss = 0.5298, gen loss = 2.5662
Iteration 30800: dis loss = 0.7545, gen loss = 1.1593
Iteration 30850: dis loss = 1.0878, gen loss = 0.8052
Iteration 30900: dis loss = 0.5288, gen loss = 2.2401
Iteration 30950: dis loss = 0.3124, gen loss = 2.5219
Iteration 31000: dis loss = 0.9199, gen loss = 1.1121
Iteration 31050: dis loss = 0.9329, gen loss = 3.4582
Iteration 31100: dis loss = 0.7381, gen loss = 1.4715
Iteration 31150: dis loss = 0.2996, gen loss = 3.7395
Iteration 31200: dis loss = 0.5431, gen loss = 3.3033






    












    












    












    



Iteration 31250: dis loss = 0.8954, gen loss = 2.4962
Iteration 31300: dis loss = 0.4979, gen loss = 2.2982
Iteration 31350: dis loss = 0.6592, gen loss = 1.5886
Iteration 31400: dis loss = 0.4415, gen loss = 2.0395
Iteration 31450: dis loss = 0.7467, gen loss = 1.4526
Iteration 31500: dis loss = 0.8261, gen loss = 1.6358
Iteration 31550: dis loss = 0.6045, gen loss = 2.6492
Iteration 31600: dis loss = 0.6665, gen loss = 1.2568
Iteration 31650: dis loss = 0.5205, gen loss = 1.4201
Iteration 31700: dis loss = 0.8029, gen loss = 3.3034
Iteration 31750: dis loss = 0.6228, gen loss = 1.2808
Iteration 31800: dis loss = 0.6374, gen loss = 2.4093
Iteration 31850: dis loss = 0.5321, gen loss = 2.7493
Iteration 31900: dis loss = 0.3985, gen loss = 2.0798
Iteration 31950: dis loss = 0.6872, gen loss = 1.4614
Iteration 32000: dis loss = 0.5033, gen loss = 2.5009
Iteration 32050: dis loss = 0.9092, gen loss = 3.0328
Iteration 32100: dis loss = 0.3920, gen loss = 2.4675
Iteration 32150: dis loss = 0.8122, gen loss = 1.0795
Iteration 32200: dis loss = 0.5069, gen loss = 2.1784
Iteration 32250: dis loss = 0.3314, gen loss = 2.4899
Iteration 32300: dis loss = 0.3886, gen loss = 2.5986
Iteration 32350: dis loss = 0.4248, gen loss = 2.1708
Iteration 32400: dis loss = 0.6758, gen loss = 2.1183
Iteration 32450: dis loss = 0.2478, gen loss = 2.5267
Iteration 32500: dis loss = 0.9530, gen loss = 0.6604
Iteration 32550: dis loss = 0.5737, gen loss = 1.6349
Iteration 32600: dis loss = 0.2671, gen loss = 2.9098
Iteration 32650: dis loss = 0.8444, gen loss = 2.9892
Iteration 32700: dis loss = 0.6420, gen loss = 1.1950
Iteration 32750: dis loss = 0.3497, gen loss = 2.7105
Iteration 32800: dis loss = 0.6184, gen loss = 1.7356






    












    












    












    



Iteration 32850: dis loss = 0.7835, gen loss = 1.5175
Iteration 32900: dis loss = 0.4688, gen loss = 2.6294
Iteration 32950: dis loss = 0.5921, gen loss = 3.1083
Iteration 33000: dis loss = 0.6821, gen loss = 2.9934
Iteration 33050: dis loss = 0.6071, gen loss = 3.0744
Iteration 33100: dis loss = 0.3621, gen loss = 2.6472
Iteration 33150: dis loss = 1.2466, gen loss = 3.1211
Iteration 33200: dis loss = 0.5955, gen loss = 1.5673
Iteration 33250: dis loss = 0.5474, gen loss = 3.2581
Iteration 33300: dis loss = 0.5559, gen loss = 2.2928
Iteration 33350: dis loss = 0.9394, gen loss = 3.5121
Iteration 33400: dis loss = 0.6668, gen loss = 2.2094
Iteration 33450: dis loss = 0.4075, gen loss = 1.8332
Iteration 33500: dis loss = 0.4928, gen loss = 3.2328
Iteration 33550: dis loss = 0.4899, gen loss = 1.9406
Iteration 33600: dis loss = 1.1523, gen loss = 3.9335
Iteration 33650: dis loss = 1.3017, gen loss = 0.6756
Iteration 33700: dis loss = 0.4698, gen loss = 1.7804
Iteration 33750: dis loss = 0.3931, gen loss = 3.4831
Iteration 33800: dis loss = 0.5011, gen loss = 2.3215
Iteration 33850: dis loss = 0.4433, gen loss = 2.0255
Iteration 33900: dis loss = 0.5122, gen loss = 3.0808
Iteration 33950: dis loss = 0.6523, gen loss = 3.5107
Iteration 34000: dis loss = 0.7880, gen loss = 1.1693
Iteration 34050: dis loss = 0.7607, gen loss = 1.2417
Iteration 34100: dis loss = 0.4499, gen loss = 3.6102
Iteration 34150: dis loss = 0.3751, gen loss = 2.6597
Iteration 34200: dis loss = 0.7040, gen loss = 1.2890
Iteration 34250: dis loss = 0.4660, gen loss = 3.0383
Iteration 34300: dis loss = 1.0098, gen loss = 5.0336
Iteration 34350: dis loss = 0.5263, gen loss = 1.6585






    












    












    












    



Iteration 34400: dis loss = 0.7052, gen loss = 2.2236
Iteration 34450: dis loss = 0.8606, gen loss = 2.7983
Iteration 34500: dis loss = 0.4646, gen loss = 1.9037
Iteration 34550: dis loss = 0.4652, gen loss = 2.6250
Iteration 34600: dis loss = 0.4287, gen loss = 2.2207
Iteration 34650: dis loss = 0.6202, gen loss = 3.7467
Iteration 34700: dis loss = 0.3457, gen loss = 2.9169
Iteration 34750: dis loss = 0.3862, gen loss = 2.5455
Iteration 34800: dis loss = 0.5676, gen loss = 1.7653
Iteration 34850: dis loss = 1.5632, gen loss = 2.8372
Iteration 34900: dis loss = 0.5505, gen loss = 2.4881
Iteration 34950: dis loss = 0.5116, gen loss = 3.2929
Iteration 35000: dis loss = 0.4869, gen loss = 1.6120
Iteration 35050: dis loss = 0.5308, gen loss = 1.5230
Iteration 35100: dis loss = 0.4005, gen loss = 2.2705
Iteration 35150: dis loss = 0.6416, gen loss = 3.0451
Iteration 35200: dis loss = 0.4648, gen loss = 2.1435
Iteration 35250: dis loss = 0.4626, gen loss = 1.9462
Iteration 35300: dis loss = 0.5111, gen loss = 1.6602
Iteration 35350: dis loss = 0.6325, gen loss = 1.5202
Iteration 35400: dis loss = 0.2453, gen loss = 2.7721
Iteration 35450: dis loss = 0.6791, gen loss = 3.8936
Iteration 35500: dis loss = 0.4389, gen loss = 3.2107
Iteration 35550: dis loss = 0.6299, gen loss = 2.5006
Iteration 35600: dis loss = 0.4754, gen loss = 1.8648
Iteration 35650: dis loss = 0.9269, gen loss = 2.6160
Iteration 35700: dis loss = 0.5232, gen loss = 1.8450
Iteration 35750: dis loss = 0.8480, gen loss = 3.1607
Iteration 35800: dis loss = 0.3743, gen loss = 2.3847
Iteration 35850: dis loss = 0.5955, gen loss = 2.0671
Iteration 35900: dis loss = 0.3251, gen loss = 3.0464






    












    












    












    



Iteration 35950: dis loss = 1.1669, gen loss = 4.5609
Iteration 36000: dis loss = 0.5669, gen loss = 2.2205
Iteration 36050: dis loss = 0.4501, gen loss = 1.9740
Iteration 36100: dis loss = 0.4931, gen loss = 2.0909
Iteration 36150: dis loss = 0.5348, gen loss = 2.4882
Iteration 36200: dis loss = 0.4567, gen loss = 2.5121
Iteration 36250: dis loss = 0.4517, gen loss = 2.2843
Iteration 36300: dis loss = 0.6867, gen loss = 3.4692
Iteration 36350: dis loss = 0.5529, gen loss = 3.9179
Iteration 36400: dis loss = 0.3165, gen loss = 3.0593
Iteration 36450: dis loss = 0.6101, gen loss = 2.3008
Iteration 36500: dis loss = 0.9154, gen loss = 3.6855
Iteration 36550: dis loss = 0.2902, gen loss = 2.1767
Iteration 36600: dis loss = 0.5419, gen loss = 1.6031
Iteration 36650: dis loss = 0.4706, gen loss = 2.2391
Iteration 36700: dis loss = 0.3560, gen loss = 3.6066
Iteration 36750: dis loss = 0.7382, gen loss = 3.2928
Iteration 36800: dis loss = 0.5484, gen loss = 3.7652
Iteration 36850: dis loss = 0.5049, gen loss = 1.8144
Iteration 36900: dis loss = 0.4617, gen loss = 3.1438
Iteration 36950: dis loss = 0.6695, gen loss = 1.2996
Iteration 37000: dis loss = 0.7715, gen loss = 1.9119
Iteration 37050: dis loss = 0.8560, gen loss = 4.1665
Iteration 37100: dis loss = 0.4261, gen loss = 1.8897
Iteration 37150: dis loss = 0.5670, gen loss = 1.6597
Iteration 37200: dis loss = 0.3861, gen loss = 2.1803
Iteration 37250: dis loss = 0.8947, gen loss = 4.0851
Iteration 37300: dis loss = 0.6357, gen loss = 1.3314
Iteration 37350: dis loss = 0.3049, gen loss = 2.7622
Iteration 37400: dis loss = 0.3919, gen loss = 2.5871
Iteration 37450: dis loss = 1.5015, gen loss = 3.7988






    












    












    












    



Iteration 37500: dis loss = 0.3275, gen loss = 2.2231
Iteration 37550: dis loss = 0.5981, gen loss = 3.5317
Iteration 37600: dis loss = 0.4454, gen loss = 3.1476
Iteration 37650: dis loss = 0.4225, gen loss = 2.4273
Iteration 37700: dis loss = 0.6544, gen loss = 1.1186
Iteration 37750: dis loss = 0.6003, gen loss = 0.9445
Iteration 37800: dis loss = 0.4772, gen loss = 2.8404
Iteration 37850: dis loss = 0.2982, gen loss = 2.6577
Iteration 37900: dis loss = 0.4351, gen loss = 3.0706
Iteration 37950: dis loss = 0.4567, gen loss = 3.0745
Iteration 38000: dis loss = 0.7632, gen loss = 2.0303
Iteration 38050: dis loss = 0.6045, gen loss = 1.4637
Iteration 38100: dis loss = 0.4635, gen loss = 3.0441
Iteration 38150: dis loss = 0.4449, gen loss = 4.1899
Iteration 38200: dis loss = 0.5073, gen loss = 1.6682
Iteration 38250: dis loss = 0.2865, gen loss = 2.1555
Iteration 38300: dis loss = 0.6114, gen loss = 3.0585
Iteration 38350: dis loss = 0.4718, gen loss = 2.0973
Iteration 38400: dis loss = 0.9303, gen loss = 0.9370
Iteration 38450: dis loss = 0.4360, gen loss = 2.9514
Iteration 38500: dis loss = 0.7473, gen loss = 1.0712
Iteration 38550: dis loss = 0.4502, gen loss = 3.6265
Iteration 38600: dis loss = 0.5586, gen loss = 3.2713
Iteration 38650: dis loss = 0.3785, gen loss = 2.7993
Iteration 38700: dis loss = 0.8695, gen loss = 0.7916
Iteration 38750: dis loss = 0.9655, gen loss = 1.2254
Iteration 38800: dis loss = 0.7940, gen loss = 1.0346
Iteration 38850: dis loss = 0.5512, gen loss = 2.5946
Iteration 38900: dis loss = 0.8779, gen loss = 1.3325
Iteration 38950: dis loss = 0.5286, gen loss = 2.0518
Iteration 39000: dis loss = 0.5939, gen loss = 3.4406
Iteration 39050: dis loss = 0.5313, gen loss = 2.3304






    












    












    












    



Iteration 39100: dis loss = 0.6570, gen loss = 3.2521
Iteration 39150: dis loss = 0.4118, gen loss = 2.4337
Iteration 39200: dis loss = 0.2305, gen loss = 2.9017
Iteration 39250: dis loss = 0.2996, gen loss = 2.4385
Iteration 39300: dis loss = 0.4137, gen loss = 3.0080
Iteration 39350: dis loss = 0.7391, gen loss = 3.5013
Iteration 39400: dis loss = 0.5566, gen loss = 3.0458
Iteration 39450: dis loss = 1.3353, gen loss = 0.6128
Iteration 39500: dis loss = 0.4654, gen loss = 2.0522
Iteration 39550: dis loss = 0.7591, gen loss = 3.6490
Iteration 39600: dis loss = 0.7872, gen loss = 1.2197
Iteration 39650: dis loss = 0.9663, gen loss = 0.7522
Iteration 39700: dis loss = 0.4092, gen loss = 3.1468
Iteration 39750: dis loss = 1.4375, gen loss = 3.4854
Iteration 39800: dis loss = 0.3444, gen loss = 2.8489
Iteration 39850: dis loss = 0.5134, gen loss = 2.2257
Iteration 39900: dis loss = 0.5869, gen loss = 1.4649
Iteration 39950: dis loss = 0.4804, gen loss = 2.8782
Iteration 40000: dis loss = 1.1007, gen loss = 2.7614
Iteration 40050: dis loss = 0.4601, gen loss = 2.0392
Iteration 40100: dis loss = 0.1895, gen loss = 3.1754
Iteration 40150: dis loss = 0.4189, gen loss = 3.3141
Iteration 40200: dis loss = 0.6106, gen loss = 3.4825
Iteration 40250: dis loss = 0.7437, gen loss = 1.3301
Iteration 40300: dis loss = 0.9620, gen loss = 4.3459
Iteration 40350: dis loss = 0.8233, gen loss = 3.9171
Iteration 40400: dis loss = 0.5367, gen loss = 2.1133
Iteration 40450: dis loss = 0.3398, gen loss = 2.6652
Iteration 40500: dis loss = 0.4198, gen loss = 2.5033
Iteration 40550: dis loss = 0.4615, gen loss = 1.6857
Iteration 40600: dis loss = 0.1807, gen loss = 3.0929






    












    












    












    



Iteration 40650: dis loss = 0.5937, gen loss = 1.8401
Iteration 40700: dis loss = 1.2658, gen loss = 4.3706
Iteration 40750: dis loss = 0.4813, gen loss = 3.4633
Iteration 40800: dis loss = 0.3579, gen loss = 2.2822
Iteration 40850: dis loss = 0.4555, gen loss = 2.3177
Iteration 40900: dis loss = 0.5605, gen loss = 3.2087
Iteration 40950: dis loss = 0.6280, gen loss = 1.7706
Iteration 41000: dis loss = 0.4015, gen loss = 2.4548
Iteration 41050: dis loss = 1.9876, gen loss = 1.5677
Iteration 41100: dis loss = 0.4303, gen loss = 2.8906
Iteration 41150: dis loss = 0.3901, gen loss = 2.5723
Iteration 41200: dis loss = 0.7776, gen loss = 1.2461
Iteration 41250: dis loss = 1.0632, gen loss = 3.9299
Iteration 41300: dis loss = 0.4664, gen loss = 3.3399
Iteration 41350: dis loss = 0.6071, gen loss = 1.5032
Iteration 41400: dis loss = 0.3783, gen loss = 3.6654
Iteration 41450: dis loss = 0.6380, gen loss = 1.4064
Iteration 41500: dis loss = 0.6877, gen loss = 3.3963
Iteration 41550: dis loss = 0.4106, gen loss = 2.6986
Iteration 41600: dis loss = 0.4313, gen loss = 1.8779
Iteration 41650: dis loss = 0.5203, gen loss = 2.4152
Iteration 41700: dis loss = 0.7033, gen loss = 1.2663
Iteration 41750: dis loss = 0.3879, gen loss = 3.2074
Iteration 41800: dis loss = 0.9156, gen loss = 4.0671
Iteration 41850: dis loss = 0.4242, gen loss = 4.1243
Iteration 41900: dis loss = 0.4908, gen loss = 3.4631
Iteration 41950: dis loss = 0.8837, gen loss = 1.1804
Iteration 42000: dis loss = 0.3227, gen loss = 2.9599
Iteration 42050: dis loss = 0.3961, gen loss = 2.6791
Iteration 42100: dis loss = 0.2900, gen loss = 3.2206
Iteration 42150: dis loss = 0.5058, gen loss = 1.6608






    












    












    












    



Iteration 42200: dis loss = 0.8803, gen loss = 1.5189
Iteration 42250: dis loss = 0.5080, gen loss = 1.6094
Iteration 42300: dis loss = 0.5102, gen loss = 4.1105
Iteration 42350: dis loss = 0.4787, gen loss = 3.1617
Iteration 42400: dis loss = 0.3990, gen loss = 1.9379
Iteration 42450: dis loss = 0.9957, gen loss = 3.7071
Iteration 42500: dis loss = 0.4837, gen loss = 3.6715
Iteration 42550: dis loss = 0.6590, gen loss = 4.5376
Iteration 42600: dis loss = 0.3403, gen loss = 3.1952
Iteration 42650: dis loss = 0.6387, gen loss = 2.7601
Iteration 42700: dis loss = 0.2241, gen loss = 2.7238
Iteration 42750: dis loss = 0.6740, gen loss = 4.1566
Iteration 42800: dis loss = 0.3554, gen loss = 2.9042
Iteration 42850: dis loss = 0.4376, gen loss = 2.4566
Iteration 42900: dis loss = 0.4833, gen loss = 1.9588
Iteration 42950: dis loss = 0.7120, gen loss = 1.0897
Iteration 43000: dis loss = 0.5984, gen loss = 1.9905
Iteration 43050: dis loss = 0.8223, gen loss = 4.5933
Iteration 43100: dis loss = 0.5149, gen loss = 3.6438
Iteration 43150: dis loss = 0.4207, gen loss = 2.6808
Iteration 43200: dis loss = 0.8194, gen loss = 1.5003
Iteration 43250: dis loss = 0.4395, gen loss = 1.4860
Iteration 43300: dis loss = 0.3302, gen loss = 2.5808
Iteration 43350: dis loss = 0.4358, gen loss = 2.5124
Iteration 43400: dis loss = 0.6293, gen loss = 3.0081
Iteration 43450: dis loss = 0.6915, gen loss = 4.0244
Iteration 43500: dis loss = 0.5416, gen loss = 3.9976
Iteration 43550: dis loss = 0.7536, gen loss = 1.0231
Iteration 43600: dis loss = 0.5819, gen loss = 3.5259
Iteration 43650: dis loss = 0.5280, gen loss = 2.4098
Iteration 43700: dis loss = 0.6887, gen loss = 1.5356






    












    












    












    



Iteration 43750: dis loss = 0.4348, gen loss = 2.3337
Iteration 43800: dis loss = 0.4496, gen loss = 1.9241
Iteration 43850: dis loss = 0.4400, gen loss = 2.5635
Iteration 43900: dis loss = 0.4146, gen loss = 2.0838
Iteration 43950: dis loss = 0.4446, gen loss = 3.2005
Iteration 44000: dis loss = 0.5830, gen loss = 1.9174
Iteration 44050: dis loss = 0.4280, gen loss = 2.2789
Iteration 44100: dis loss = 0.9146, gen loss = 4.1472
Iteration 44150: dis loss = 0.1628, gen loss = 3.8866
Iteration 44200: dis loss = 0.7741, gen loss = 1.8062
Iteration 44250: dis loss = 0.4944, gen loss = 3.1546
Iteration 44300: dis loss = 0.5576, gen loss = 1.4809
Iteration 44350: dis loss = 0.2657, gen loss = 3.1914
Iteration 44400: dis loss = 0.5297, gen loss = 1.9163
Iteration 44450: dis loss = 0.4219, gen loss = 1.8462
Iteration 44500: dis loss = 0.4586, gen loss = 3.2338
Iteration 44550: dis loss = 0.2519, gen loss = 3.3297
Iteration 44600: dis loss = 0.4045, gen loss = 2.5998
Iteration 44650: dis loss = 0.3345, gen loss = 2.1644
Iteration 44700: dis loss = 0.3407, gen loss = 3.0790
Iteration 44750: dis loss = 0.5239, gen loss = 2.2765
Iteration 44800: dis loss = 0.9371, gen loss = 4.2494
Iteration 44850: dis loss = 0.4695, gen loss = 1.8008
Iteration 44900: dis loss = 0.3510, gen loss = 3.2207
Iteration 44950: dis loss = 0.2890, gen loss = 2.7646
Iteration 45000: dis loss = 0.4006, gen loss = 1.9301
Iteration 45050: dis loss = 0.6694, gen loss = 0.4823
Iteration 45100: dis loss = 0.9321, gen loss = 4.3186
Iteration 45150: dis loss = 0.5338, gen loss = 1.8170
Iteration 45200: dis loss = 0.6109, gen loss = 3.8945
Iteration 45250: dis loss = 0.3928, gen loss = 3.0403






    












    












    












    



Iteration 45300: dis loss = 0.3368, gen loss = 3.3515
Iteration 45350: dis loss = 0.3936, gen loss = 2.5066
Iteration 45400: dis loss = 0.4667, gen loss = 2.0634
Iteration 45450: dis loss = 0.6257, gen loss = 3.9550
Iteration 45500: dis loss = 0.5099, gen loss = 3.4361
Iteration 45550: dis loss = 0.3349, gen loss = 3.1857
Iteration 45600: dis loss = 0.5801, gen loss = 1.4749
Iteration 45650: dis loss = 0.2118, gen loss = 3.3459
Iteration 45700: dis loss = 0.5087, gen loss = 2.1883
Iteration 45750: dis loss = 0.3551, gen loss = 2.5504
Iteration 45800: dis loss = 0.5514, gen loss = 1.6452
Iteration 45850: dis loss = 0.4710, gen loss = 2.0329
Iteration 45900: dis loss = 0.6119, gen loss = 2.2029
Iteration 45950: dis loss = 0.2860, gen loss = 3.1826
Iteration 46000: dis loss = 0.6716, gen loss = 1.2316
Iteration 46050: dis loss = 0.4746, gen loss = 3.3052
Iteration 46100: dis loss = 0.3585, gen loss = 3.0425
Iteration 46150: dis loss = 1.1687, gen loss = 4.2816
Iteration 46200: dis loss = 0.5852, gen loss = 2.1672
Iteration 46250: dis loss = 0.4787, gen loss = 3.5711
Iteration 46300: dis loss = 0.9833, gen loss = 4.9628
Iteration 46350: dis loss = 0.2661, gen loss = 3.2051
Iteration 46400: dis loss = 0.5753, gen loss = 3.2718
Iteration 46450: dis loss = 0.3187, gen loss = 3.6729
Iteration 46500: dis loss = 0.4478, gen loss = 2.0611
Iteration 46550: dis loss = 0.9248, gen loss = 0.8130
Iteration 46600: dis loss = 0.5405, gen loss = 2.9202
Iteration 46650: dis loss = 0.4816, gen loss = 2.0867
Iteration 46700: dis loss = 0.5709, gen loss = 2.2678
Iteration 46750: dis loss = 0.6731, gen loss = 4.5582
Iteration 46800: dis loss = 0.3687, gen loss = 3.1554
Iteration 46850: dis loss = 0.4329, gen loss = 3.1355






    












    












    












    



Iteration 46900: dis loss = 0.6088, gen loss = 1.6828
Iteration 46950: dis loss = 0.3884, gen loss = 4.1726
Iteration 47000: dis loss = 0.5054, gen loss = 2.7340
Iteration 47050: dis loss = 0.5659, gen loss = 1.3827
Iteration 47100: dis loss = 0.3206, gen loss = 3.7765
Iteration 47150: dis loss = 0.4434, gen loss = 3.3523
Iteration 47200: dis loss = 0.4452, gen loss = 2.0933
Iteration 47250: dis loss = 0.4335, gen loss = 3.4227
Iteration 47300: dis loss = 0.3962, gen loss = 2.5368
Iteration 47350: dis loss = 0.5797, gen loss = 3.0371
Iteration 47400: dis loss = 0.2680, gen loss = 3.1294
Iteration 47450: dis loss = 0.5445, gen loss = 4.0460
Iteration 47500: dis loss = 0.4092, gen loss = 2.8737
Iteration 47550: dis loss = 0.5196, gen loss = 3.8258
Iteration 47600: dis loss = 0.3010, gen loss = 2.5897
Iteration 47650: dis loss = 0.6744, gen loss = 4.5887
Iteration 47700: dis loss = 0.3842, gen loss = 3.7543
Iteration 47750: dis loss = 0.5680, gen loss = 3.4520
Iteration 47800: dis loss = 0.3023, gen loss = 4.0577
Iteration 47850: dis loss = 0.2102, gen loss = 3.0661
Iteration 47900: dis loss = 0.9099, gen loss = 4.3354
Iteration 47950: dis loss = 0.3709, gen loss = 1.8628
Iteration 48000: dis loss = 0.3513, gen loss = 3.6999
Iteration 48050: dis loss = 0.3185, gen loss = 2.7404
Iteration 48100: dis loss = 0.5621, gen loss = 4.1350
Iteration 48150: dis loss = 0.5458, gen loss = 1.5825
Iteration 48200: dis loss = 0.6215, gen loss = 4.9415
Iteration 48250: dis loss = 0.4416, gen loss = 1.8100
Iteration 48300: dis loss = 0.5263, gen loss = 1.5066
Iteration 48350: dis loss = 0.3758, gen loss = 2.2700
Iteration 48400: dis loss = 0.4126, gen loss = 4.1660






    












    












    












    



Iteration 48450: dis loss = 0.3949, gen loss = 2.4202
Iteration 48500: dis loss = 1.4446, gen loss = 0.9127
Iteration 48550: dis loss = 0.5553, gen loss = 2.2694
Iteration 48600: dis loss = 0.4182, gen loss = 3.7781
Iteration 48650: dis loss = 0.3380, gen loss = 3.2663
Iteration 48700: dis loss = 0.2687, gen loss = 2.8102
Iteration 48750: dis loss = 1.0531, gen loss = 3.7160
Iteration 48800: dis loss = 0.3291, gen loss = 3.7727
Iteration 48850: dis loss = 0.4396, gen loss = 2.3365
Iteration 48900: dis loss = 0.5727, gen loss = 3.0648
Iteration 48950: dis loss = 0.2831, gen loss = 3.4172
Iteration 49000: dis loss = 0.3970, gen loss = 2.1691
Iteration 49050: dis loss = 0.9518, gen loss = 4.1694
Iteration 49100: dis loss = 0.2640, gen loss = 3.1472
Iteration 49150: dis loss = 0.7900, gen loss = 3.3886
Iteration 49200: dis loss = 0.5691, gen loss = 1.5107
Iteration 49250: dis loss = 0.8973, gen loss = 0.9378
Iteration 49300: dis loss = 0.5707, gen loss = 3.3313
Iteration 49350: dis loss = 0.3185, gen loss = 2.2190
Iteration 49400: dis loss = 1.4463, gen loss = 5.2843
Iteration 49450: dis loss = 0.8150, gen loss = 1.4750
Iteration 49500: dis loss = 0.2994, gen loss = 3.7685
Iteration 49550: dis loss = 0.3416, gen loss = 4.4618
Iteration 49600: dis loss = 0.3334, gen loss = 2.5660
Iteration 49650: dis loss = 0.2337, gen loss = 3.0466
Iteration 49700: dis loss = 0.3414, gen loss = 2.4401
Iteration 49750: dis loss = 0.2477, gen loss = 2.7221
Iteration 49800: dis loss = 0.3826, gen loss = 2.3031
Iteration 49850: dis loss = 0.5311, gen loss = 4.0044
Iteration 49900: dis loss = 0.5950, gen loss = 3.7667
Iteration 49950: dis loss = 1.6550, gen loss = 0.6305






    












    












    












    



Iteration 50000: dis loss = 0.7347, gen loss = 0.8473
Iteration 50050: dis loss = 0.5797, gen loss = 1.6655
Iteration 50100: dis loss = 0.8035, gen loss = 4.8004
Iteration 50150: dis loss = 0.4576, gen loss = 2.0983
Iteration 50200: dis loss = 0.4546, gen loss = 2.9503
Iteration 50250: dis loss = 0.2631, gen loss = 2.9010
Iteration 50300: dis loss = 0.2215, gen loss = 3.0064
Iteration 50350: dis loss = 0.5080, gen loss = 3.9797
Iteration 50400: dis loss = 0.8238, gen loss = 2.1313
Iteration 50450: dis loss = 1.1185, gen loss = 0.8166
Iteration 50500: dis loss = 1.3216, gen loss = 1.2802
Iteration 50550: dis loss = 0.4379, gen loss = 1.8034
Iteration 50600: dis loss = 0.3589, gen loss = 2.4869
Iteration 50650: dis loss = 0.4420, gen loss = 2.0103
Iteration 50700: dis loss = 0.4980, gen loss = 1.9363
Iteration 50750: dis loss = 0.2445, gen loss = 3.3689
Iteration 50800: dis loss = 0.9340, gen loss = 4.1766
Iteration 50850: dis loss = 0.3854, gen loss = 2.0296
Iteration 50900: dis loss = 0.4674, gen loss = 3.1467
Iteration 50950: dis loss = 0.3448, gen loss = 3.5359
Iteration 51000: dis loss = 0.5666, gen loss = 2.8355
Iteration 51050: dis loss = 0.4207, gen loss = 2.6740
Iteration 51100: dis loss = 0.4172, gen loss = 2.2891
Iteration 51150: dis loss = 0.4676, gen loss = 3.9700
Iteration 51200: dis loss = 0.4403, gen loss = 2.0123
Iteration 51250: dis loss = 0.3422, gen loss = 3.3202
Iteration 51300: dis loss = 0.4476, gen loss = 4.8844
Iteration 51350: dis loss = 0.6270, gen loss = 2.7932
Iteration 51400: dis loss = 0.5608, gen loss = 2.6217
Iteration 51450: dis loss = 0.2997, gen loss = 2.7565
Iteration 51500: dis loss = 0.3428, gen loss = 3.4968






    












    












    












    



Iteration 51550: dis loss = 0.2854, gen loss = 3.4686
Iteration 51600: dis loss = 0.1567, gen loss = 3.8200
Iteration 51650: dis loss = 1.1024, gen loss = 1.3465
Iteration 51700: dis loss = 0.4154, gen loss = 3.3453
Iteration 51750: dis loss = 0.4078, gen loss = 2.3225
Iteration 51800: dis loss = 0.3673, gen loss = 3.6876
Iteration 51850: dis loss = 0.4975, gen loss = 3.0606
Iteration 51900: dis loss = 0.2878, gen loss = 3.3488
Iteration 51950: dis loss = 0.5268, gen loss = 2.3109
Iteration 52000: dis loss = 0.4505, gen loss = 3.5145
Iteration 52050: dis loss = 0.3252, gen loss = 3.7404
Iteration 52100: dis loss = 0.2719, gen loss = 3.0280
Iteration 52150: dis loss = 0.9812, gen loss = 4.3644
Iteration 52200: dis loss = 0.2701, gen loss = 3.1828
Iteration 52250: dis loss = 0.3047, gen loss = 3.1410
Iteration 52300: dis loss = 0.5227, gen loss = 1.9577
Iteration 52350: dis loss = 0.4482, gen loss = 1.6835
Iteration 52400: dis loss = 0.4018, gen loss = 1.9591
Iteration 52450: dis loss = 0.2912, gen loss = 2.8276
Iteration 52500: dis loss = 0.7162, gen loss = 1.5385
Iteration 52550: dis loss = 0.9461, gen loss = 1.1599
Iteration 52600: dis loss = 0.4718, gen loss = 2.4582
Iteration 52650: dis loss = 0.6002, gen loss = 4.3030
Iteration 52700: dis loss = 0.5135, gen loss = 2.3121
Iteration 52750: dis loss = 0.3872, gen loss = 3.0133
Iteration 52800: dis loss = 0.6810, gen loss = 3.1425
Iteration 52850: dis loss = 0.2293, gen loss = 3.2866
Iteration 52900: dis loss = 0.4442, gen loss = 1.5108
Iteration 52950: dis loss = 0.5079, gen loss = 3.5151
Iteration 53000: dis loss = 0.9337, gen loss = 3.8699
Iteration 53050: dis loss = 0.3805, gen loss = 1.6377
Iteration 53100: dis loss = 0.5159, gen loss = 3.6627






    












    












    












    



Iteration 53150: dis loss = 0.6955, gen loss = 1.7891
Iteration 53200: dis loss = 0.6892, gen loss = 3.6880
Iteration 53250: dis loss = 0.8720, gen loss = 4.4184
Iteration 53300: dis loss = 0.4660, gen loss = 3.5059
Iteration 53350: dis loss = 0.5325, gen loss = 1.9510
Iteration 53400: dis loss = 0.5534, gen loss = 1.7136
Iteration 53450: dis loss = 0.2264, gen loss = 3.3748
Iteration 53500: dis loss = 0.2358, gen loss = 3.3320
Iteration 53550: dis loss = 0.1900, gen loss = 3.7510
Iteration 53600: dis loss = 0.2724, gen loss = 3.0246
Iteration 53650: dis loss = 1.1793, gen loss = 4.9368
Iteration 53700: dis loss = 0.2093, gen loss = 3.3222
Iteration 53750: dis loss = 0.8214, gen loss = 4.8371
Iteration 53800: dis loss = 0.5209, gen loss = 2.4715
Iteration 53850: dis loss = 0.3594, gen loss = 4.7633
Iteration 53900: dis loss = 0.4072, gen loss = 3.6181
Iteration 53950: dis loss = 0.8068, gen loss = 0.8014
Iteration 54000: dis loss = 0.4486, gen loss = 4.0140
Iteration 54050: dis loss = 0.6699, gen loss = 4.5394
Iteration 54100: dis loss = 0.2282, gen loss = 3.8019
Iteration 54150: dis loss = 0.6467, gen loss = 4.1830
Iteration 54200: dis loss = 0.6133, gen loss = 2.9607
Iteration 54250: dis loss = 1.0482, gen loss = 0.9669
Iteration 54300: dis loss = 0.4241, gen loss = 1.8523
Iteration 54350: dis loss = 0.6121, gen loss = 4.2295
Iteration 54400: dis loss = 0.4517, gen loss = 2.4504
Iteration 54450: dis loss = 0.4265, gen loss = 3.8570
Iteration 54500: dis loss = 0.5752, gen loss = 3.4553
Iteration 54550: dis loss = 0.3975, gen loss = 2.1106
Iteration 54600: dis loss = 0.4801, gen loss = 1.8699
Iteration 54650: dis loss = 0.5682, gen loss = 2.0015






    












    












    












    



Iteration 54700: dis loss = 0.8560, gen loss = 3.6965
Iteration 54750: dis loss = 0.4034, gen loss = 3.4928
Iteration 54800: dis loss = 0.6505, gen loss = 4.3243
Iteration 54850: dis loss = 0.6977, gen loss = 2.1438
Iteration 54900: dis loss = 0.4388, gen loss = 2.6226
Iteration 54950: dis loss = 0.7877, gen loss = 1.0653
Iteration 55000: dis loss = 0.1740, gen loss = 3.7662
Iteration 55050: dis loss = 1.2141, gen loss = 3.7848
Iteration 55100: dis loss = 0.3633, gen loss = 3.0901
Iteration 55150: dis loss = 0.3951, gen loss = 1.8971
Iteration 55200: dis loss = 0.6563, gen loss = 3.9314
Iteration 55250: dis loss = 0.2723, gen loss = 2.5816
Iteration 55300: dis loss = 1.3404, gen loss = 4.8837
Iteration 55350: dis loss = 0.4050, gen loss = 3.8263
Iteration 55400: dis loss = 0.5152, gen loss = 3.3438
Iteration 55450: dis loss = 0.3280, gen loss = 3.2819
Iteration 55500: dis loss = 0.2777, gen loss = 2.5996
Iteration 55550: dis loss = 0.4878, gen loss = 3.7217
Iteration 55600: dis loss = 0.7176, gen loss = 1.5013
Iteration 55650: dis loss = 0.6119, gen loss = 4.2168
Iteration 55700: dis loss = 0.4875, gen loss = 2.2276
Iteration 55750: dis loss = 0.3036, gen loss = 3.2656
Iteration 55800: dis loss = 0.5570, gen loss = 4.2883
Iteration 55850: dis loss = 1.0331, gen loss = 2.8669
Iteration 55900: dis loss = 0.2251, gen loss = 4.2229
Iteration 55950: dis loss = 0.2247, gen loss = 3.3566
Iteration 56000: dis loss = 0.3810, gen loss = 2.8899
Iteration 56050: dis loss = 0.3812, gen loss = 2.5788
Iteration 56100: dis loss = 0.7285, gen loss = 3.8526
Iteration 56150: dis loss = 0.4655, gen loss = 3.5495
Iteration 56200: dis loss = 0.2783, gen loss = 3.0439






    












    












    












    



Iteration 56250: dis loss = 0.9516, gen loss = 5.8629
Iteration 56300: dis loss = 0.2306, gen loss = 3.1154
Iteration 56350: dis loss = 0.1870, gen loss = 3.4354
Iteration 56400: dis loss = 0.5259, gen loss = 1.8695
Iteration 56450: dis loss = 0.8309, gen loss = 4.9715
Iteration 56500: dis loss = 0.4138, gen loss = 1.9464
Iteration 56550: dis loss = 0.5709, gen loss = 3.8609
Iteration 56600: dis loss = 0.3699, gen loss = 4.0936
Iteration 56650: dis loss = 0.2376, gen loss = 3.5526
Iteration 56700: dis loss = 0.1519, gen loss = 3.6462
Iteration 56750: dis loss = 0.1076, gen loss = 3.6836
Iteration 56800: dis loss = 0.7571, gen loss = 4.5913
Iteration 56850: dis loss = 0.3123, gen loss = 3.3129
Iteration 56900: dis loss = 0.3605, gen loss = 1.4887
Iteration 56950: dis loss = 0.6447, gen loss = 2.6644
Iteration 57000: dis loss = 0.3954, gen loss = 2.7332
Iteration 57050: dis loss = 0.4173, gen loss = 2.9578
Iteration 57100: dis loss = 1.0197, gen loss = 3.8802
Iteration 57150: dis loss = 0.8029, gen loss = 3.8914
Iteration 57200: dis loss = 0.6765, gen loss = 1.7139
Iteration 57250: dis loss = 1.0406, gen loss = 0.7601
Iteration 57300: dis loss = 0.5829, gen loss = 2.3132
Iteration 57350: dis loss = 0.2015, gen loss = 2.8287
Iteration 57400: dis loss = 0.3732, gen loss = 2.0800
Iteration 57450: dis loss = 0.2941, gen loss = 4.2030
Iteration 57500: dis loss = 0.6856, gen loss = 1.2138
Iteration 57550: dis loss = 0.2253, gen loss = 3.0207
Iteration 57600: dis loss = 0.5537, gen loss = 2.0679
Iteration 57650: dis loss = 0.2973, gen loss = 2.6365
Iteration 57700: dis loss = 0.3794, gen loss = 2.3380
Iteration 57750: dis loss = 0.2715, gen loss = 2.8271






    












    












    












    



Iteration 57800: dis loss = 0.4462, gen loss = 3.2204
Iteration 57850: dis loss = 0.5852, gen loss = 3.6221
Iteration 57900: dis loss = 0.3505, gen loss = 2.3985
Iteration 57950: dis loss = 1.1109, gen loss = 2.1907
Iteration 58000: dis loss = 0.4652, gen loss = 3.9978
Iteration 58050: dis loss = 0.6254, gen loss = 2.9599
Iteration 58100: dis loss = 0.3354, gen loss = 4.2306
Iteration 58150: dis loss = 0.5811, gen loss = 3.6505
Iteration 58200: dis loss = 0.7258, gen loss = 1.3823
Iteration 58250: dis loss = 0.9474, gen loss = 4.0454
Iteration 58300: dis loss = 0.3035, gen loss = 2.7208
Iteration 58350: dis loss = 0.5195, gen loss = 4.9010
Iteration 58400: dis loss = 0.2333, gen loss = 3.1943
Iteration 58450: dis loss = 0.5337, gen loss = 1.1075
Iteration 58500: dis loss = 0.7638, gen loss = 1.2821
Iteration 58550: dis loss = 0.3772, gen loss = 2.5599
Iteration 58600: dis loss = 0.3406, gen loss = 2.5635
Iteration 58650: dis loss = 0.6495, gen loss = 3.2624
Iteration 58700: dis loss = 0.6885, gen loss = 2.0233
Iteration 58750: dis loss = 0.1789, gen loss = 3.9403
Iteration 58800: dis loss = 0.3792, gen loss = 2.2258
Iteration 58850: dis loss = 0.6262, gen loss = 1.4584
Iteration 58900: dis loss = 0.1781, gen loss = 3.2222
Iteration 58950: dis loss = 0.4778, gen loss = 1.7732
Iteration 59000: dis loss = 0.2176, gen loss = 3.0142
Iteration 59050: dis loss = 0.4670, gen loss = 2.1971
Iteration 59100: dis loss = 0.5537, gen loss = 3.5669
Iteration 59150: dis loss = 0.6359, gen loss = 1.7728
Iteration 59200: dis loss = 0.3044, gen loss = 3.4853
Iteration 59250: dis loss = 1.2276, gen loss = 4.9396
Iteration 59300: dis loss = 0.4243, gen loss = 2.4679
Iteration 59350: dis loss = 0.9676, gen loss = 3.7337






    












    












    












    



Iteration 59400: dis loss = 0.2537, gen loss = 2.7097
Iteration 59450: dis loss = 0.4913, gen loss = 2.0791
Iteration 59500: dis loss = 0.8753, gen loss = 1.8001
Iteration 59550: dis loss = 0.2740, gen loss = 3.5355
Iteration 59600: dis loss = 0.3490, gen loss = 3.2658
Iteration 59650: dis loss = 0.3319, gen loss = 3.4129
Iteration 59700: dis loss = 0.2591, gen loss = 4.2442
Iteration 59750: dis loss = 0.3320, gen loss = 2.9716
Iteration 59800: dis loss = 0.3431, gen loss = 3.6487
Iteration 59850: dis loss = 0.5045, gen loss = 2.4192
Iteration 59900: dis loss = 0.2312, gen loss = 2.7304
Iteration 59950: dis loss = 0.7002, gen loss = 1.5954
Iteration 60000: dis loss = 0.3605, gen loss = 3.3748
Iteration 60050: dis loss = 0.1077, gen loss = 4.1298
Iteration 60100: dis loss = 0.4633, gen loss = 2.1914
Iteration 60150: dis loss = 0.3112, gen loss = 2.7204
Iteration 60200: dis loss = 0.2503, gen loss = 3.6330
Iteration 60250: dis loss = 0.7194, gen loss = 2.0199
Iteration 60300: dis loss = 0.5045, gen loss = 2.0374
Iteration 60350: dis loss = 0.4406, gen loss = 3.7544
Iteration 60400: dis loss = 0.5057, gen loss = 4.7819
Iteration 60450: dis loss = 0.1118, gen loss = 3.9219
Iteration 60500: dis loss = 0.5303, gen loss = 2.0424
Iteration 60550: dis loss = 0.2608, gen loss = 3.2758
Iteration 60600: dis loss = 0.4649, gen loss = 3.6250
Iteration 60650: dis loss = 0.4559, gen loss = 4.2488
Iteration 60700: dis loss = 0.4409, gen loss = 4.0080
Iteration 60750: dis loss = 0.7043, gen loss = 2.4153
Iteration 60800: dis loss = 1.4723, gen loss = 4.7295
Iteration 60850: dis loss = 0.4056, gen loss = 1.9919
Iteration 60900: dis loss = 0.3214, gen loss = 3.1921






    












    












    












    



Iteration 60950: dis loss = 0.4699, gen loss = 4.1366
Iteration 61000: dis loss = 0.4889, gen loss = 4.0755
Iteration 61050: dis loss = 1.0902, gen loss = 4.8986
Iteration 61100: dis loss = 0.8682, gen loss = 1.9338
Iteration 61150: dis loss = 2.6897, gen loss = 1.0543
Iteration 61200: dis loss = 0.3711, gen loss = 1.9328
Iteration 61250: dis loss = 0.5157, gen loss = 3.3573
Iteration 61300: dis loss = 0.7257, gen loss = 2.3974
Iteration 61350: dis loss = 0.4949, gen loss = 2.9339
Iteration 61400: dis loss = 0.6760, gen loss = 3.4028
Iteration 61450: dis loss = 0.2780, gen loss = 3.9726
Iteration 61500: dis loss = 0.3501, gen loss = 2.9128
Iteration 61550: dis loss = 0.5527, gen loss = 3.8549
Iteration 61600: dis loss = 0.4046, gen loss = 4.9189
Iteration 61650: dis loss = 0.3594, gen loss = 3.6664
Iteration 61700: dis loss = 0.4538, gen loss = 5.2141
Iteration 61750: dis loss = 0.3651, gen loss = 2.3980
Iteration 61800: dis loss = 0.3944, gen loss = 3.2837
Iteration 61850: dis loss = 0.4954, gen loss = 2.3921
Iteration 61900: dis loss = 0.6659, gen loss = 1.7212
Iteration 61950: dis loss = 0.3796, gen loss = 2.8958
Iteration 62000: dis loss = 1.2061, gen loss = 5.9542
Iteration 62050: dis loss = 0.4020, gen loss = 3.0888
Iteration 62100: dis loss = 0.3889, gen loss = 3.1304
Iteration 62150: dis loss = 0.3217, gen loss = 2.2960
Iteration 62200: dis loss = 0.2435, gen loss = 3.2211
Iteration 62250: dis loss = 0.2404, gen loss = 3.0079
Iteration 62300: dis loss = 0.4459, gen loss = 2.6723
Iteration 62350: dis loss = 0.7923, gen loss = 1.2359
Iteration 62400: dis loss = 0.5167, gen loss = 3.1169
Iteration 62450: dis loss = 0.6370, gen loss = 1.3903






    












    












    












    



Iteration 62500: dis loss = 0.2816, gen loss = 3.0939
Iteration 62550: dis loss = 0.4737, gen loss = 2.4564
Iteration 62600: dis loss = 0.3784, gen loss = 4.3519
Iteration 62650: dis loss = 0.3233, gen loss = 2.9698
Iteration 62700: dis loss = 0.4790, gen loss = 4.5098
Iteration 62750: dis loss = 0.3774, gen loss = 2.8604
Iteration 62800: dis loss = 0.9191, gen loss = 1.2236
Iteration 62850: dis loss = 0.7288, gen loss = 1.1794
Iteration 62900: dis loss = 0.6212, gen loss = 4.3141
Iteration 62950: dis loss = 0.4198, gen loss = 4.4032
Iteration 63000: dis loss = 0.4540, gen loss = 2.8848
Iteration 63050: dis loss = 0.1643, gen loss = 3.5319
Iteration 63100: dis loss = 0.1973, gen loss = 3.6012
Iteration 63150: dis loss = 0.3324, gen loss = 1.6204
Iteration 63200: dis loss = 0.3988, gen loss = 1.9296
Iteration 63250: dis loss = 0.3393, gen loss = 1.8548
Iteration 63300: dis loss = 0.3768, gen loss = 3.7236
Iteration 63350: dis loss = 0.3339, gen loss = 3.3092
Iteration 63400: dis loss = 0.2388, gen loss = 2.5591
Iteration 63450: dis loss = 0.3531, gen loss = 2.5106
Iteration 63500: dis loss = 0.7515, gen loss = 1.1762
Iteration 63550: dis loss = 0.2525, gen loss = 4.3551
Iteration 63600: dis loss = 0.3844, gen loss = 2.6986
Iteration 63650: dis loss = 0.3525, gen loss = 5.1339
Iteration 63700: dis loss = 0.3716, gen loss = 5.2769
Iteration 63750: dis loss = 0.6106, gen loss = 1.1747
Iteration 63800: dis loss = 0.4872, gen loss = 2.0248
Iteration 63850: dis loss = 0.4223, gen loss = 4.9903
Iteration 63900: dis loss = 0.2754, gen loss = 4.1082
Iteration 63950: dis loss = 0.4622, gen loss = 3.7177
Iteration 64000: dis loss = 0.3011, gen loss = 2.7235






    












    












    












    



Iteration 64050: dis loss = 0.5623, gen loss = 3.8894
Iteration 64100: dis loss = 0.2648, gen loss = 3.5576
Iteration 64150: dis loss = 0.7474, gen loss = 1.4760
Iteration 64200: dis loss = 0.3374, gen loss = 2.6207
Iteration 64250: dis loss = 0.4058, gen loss = 1.8924
Iteration 64300: dis loss = 0.7621, gen loss = 4.8059
Iteration 64350: dis loss = 0.7101, gen loss = 1.9938
Iteration 64400: dis loss = 0.5142, gen loss = 2.0769
Iteration 64450: dis loss = 0.5077, gen loss = 2.8426
Iteration 64500: dis loss = 0.4380, gen loss = 2.4971
Iteration 64550: dis loss = 0.3021, gen loss = 3.9110
Iteration 64600: dis loss = 0.3331, gen loss = 2.9327
Iteration 64650: dis loss = 0.4178, gen loss = 3.7542
Iteration 64700: dis loss = 0.5455, gen loss = 2.4600
Iteration 64750: dis loss = 0.7524, gen loss = 2.4932
Iteration 64800: dis loss = 0.2590, gen loss = 3.3853
Iteration 64850: dis loss = 0.2043, gen loss = 3.8437
Iteration 64900: dis loss = 0.3348, gen loss = 3.7339
Iteration 64950: dis loss = 0.2063, gen loss = 3.5072
Iteration 65000: dis loss = 0.3366, gen loss = 3.7100
Iteration 65050: dis loss = 0.3694, gen loss = 2.7115
Iteration 65100: dis loss = 0.4308, gen loss = 2.6670
Iteration 65150: dis loss = 0.4091, gen loss = 1.6881
Iteration 65200: dis loss = 0.1909, gen loss = 3.3675
Iteration 65250: dis loss = 0.2328, gen loss = 3.9639
Iteration 65300: dis loss = 0.4037, gen loss = 2.4774
Iteration 65350: dis loss = 0.3336, gen loss = 3.8636
Iteration 65400: dis loss = 0.3595, gen loss = 4.3156
Iteration 65450: dis loss = 0.1700, gen loss = 3.3347
Iteration 65500: dis loss = 0.4320, gen loss = 2.5073
Iteration 65550: dis loss = 0.3902, gen loss = 3.9932
Iteration 65600: dis loss = 0.6423, gen loss = 1.9524






    












    












    












    



Iteration 65650: dis loss = 0.6128, gen loss = 1.5040
Iteration 65700: dis loss = 0.3462, gen loss = 2.4010
Iteration 65750: dis loss = 0.2208, gen loss = 4.1784
Iteration 65800: dis loss = 0.5164, gen loss = 4.0391
Iteration 65850: dis loss = 0.2372, gen loss = 3.4523
Iteration 65900: dis loss = 1.2230, gen loss = 5.2918
Iteration 65950: dis loss = 0.2918, gen loss = 4.1470
Iteration 66000: dis loss = 0.3760, gen loss = 2.2822
Iteration 66050: dis loss = 0.5938, gen loss = 3.3146
Iteration 66100: dis loss = 0.3303, gen loss = 2.8023
Iteration 66150: dis loss = 0.3424, gen loss = 4.4036
Iteration 66200: dis loss = 0.2460, gen loss = 3.5148
Iteration 66250: dis loss = 0.5477, gen loss = 1.5286
Iteration 66300: dis loss = 0.3951, gen loss = 2.7743
Iteration 66350: dis loss = 0.3040, gen loss = 2.0408
Iteration 66400: dis loss = 1.2091, gen loss = 5.6310
Iteration 66450: dis loss = 0.4295, gen loss = 3.6106
Iteration 66500: dis loss = 1.2468, gen loss = 0.8152
Iteration 66550: dis loss = 1.0848, gen loss = 5.9577
Iteration 66600: dis loss = 0.4546, gen loss = 3.0884
Iteration 66650: dis loss = 0.6714, gen loss = 1.2962
Iteration 66700: dis loss = 0.3213, gen loss = 2.9611
Iteration 66750: dis loss = 0.7359, gen loss = 4.6813
Iteration 66800: dis loss = 0.7506, gen loss = 1.4197
Iteration 66850: dis loss = 0.4923, gen loss = 1.7024
Iteration 66900: dis loss = 0.2293, gen loss = 3.9311
Iteration 66950: dis loss = 0.6470, gen loss = 1.4435
Iteration 67000: dis loss = 0.5103, gen loss = 2.5413
Iteration 67050: dis loss = 0.3315, gen loss = 2.9318
Iteration 67100: dis loss = 0.1734, gen loss = 3.0214
Iteration 67150: dis loss = 0.5072, gen loss = 2.0334






    












    












    












    



Iteration 67200: dis loss = 0.5002, gen loss = 3.4712
Iteration 67250: dis loss = 0.2594, gen loss = 4.1688
Iteration 67300: dis loss = 0.2554, gen loss = 3.2563
Iteration 67350: dis loss = 0.3682, gen loss = 2.6011
Iteration 67400: dis loss = 0.7767, gen loss = 5.0074
Iteration 67450: dis loss = 0.2846, gen loss = 4.0855
Iteration 67500: dis loss = 1.4636, gen loss = 5.4323
Iteration 67550: dis loss = 0.3586, gen loss = 4.0530
Iteration 67600: dis loss = 0.2233, gen loss = 2.9492
Iteration 67650: dis loss = 0.2156, gen loss = 3.7358
Iteration 67700: dis loss = 0.4372, gen loss = 2.5419
Iteration 67750: dis loss = 0.2268, gen loss = 3.8505
Iteration 67800: dis loss = 0.4124, gen loss = 2.1182
Iteration 67850: dis loss = 1.6709, gen loss = 4.9643
Iteration 67900: dis loss = 0.2011, gen loss = 3.0637
Iteration 67950: dis loss = 0.4420, gen loss = 3.2932
Iteration 68000: dis loss = 0.4495, gen loss = 1.8511
Iteration 68050: dis loss = 0.4923, gen loss = 2.2068
Iteration 68100: dis loss = 0.8766, gen loss = 4.3682
Iteration 68150: dis loss = 0.7458, gen loss = 1.8754
Iteration 68200: dis loss = 0.4691, gen loss = 2.6113
Iteration 68250: dis loss = 0.5689, gen loss = 4.1459
Iteration 68300: dis loss = 0.3424, gen loss = 2.5340
Iteration 68350: dis loss = 0.4831, gen loss = 0.7491
Iteration 68400: dis loss = 0.4434, gen loss = 2.5304
Iteration 68450: dis loss = 0.3101, gen loss = 3.9152
Iteration 68500: dis loss = 0.3953, gen loss = 3.5204
Iteration 68550: dis loss = 0.4264, gen loss = 4.1503
Iteration 68600: dis loss = 0.4983, gen loss = 2.7845
Iteration 68650: dis loss = 0.3711, gen loss = 2.7451
Iteration 68700: dis loss = 0.4511, gen loss = 5.1061






    












    












    












    



Iteration 68750: dis loss = 0.6097, gen loss = 4.7007
Iteration 68800: dis loss = 0.4306, gen loss = 1.7296
Iteration 68850: dis loss = 0.2961, gen loss = 3.3402
Iteration 68900: dis loss = 0.3813, gen loss = 3.9232
Iteration 68950: dis loss = 0.3254, gen loss = 3.0312
Iteration 69000: dis loss = 0.4705, gen loss = 2.2279
Iteration 69050: dis loss = 0.5276, gen loss = 4.1149
Iteration 69100: dis loss = 0.9594, gen loss = 1.6136
Iteration 69150: dis loss = 0.1444, gen loss = 3.5547
Iteration 69200: dis loss = 0.2359, gen loss = 3.1114
Iteration 69250: dis loss = 0.7443, gen loss = 5.0285
Iteration 69300: dis loss = 0.2629, gen loss = 3.8534
Iteration 69350: dis loss = 0.4116, gen loss = 1.8188
Iteration 69400: dis loss = 0.2850, gen loss = 2.2862
Iteration 69450: dis loss = 0.6839, gen loss = 1.5944
Iteration 69500: dis loss = 0.1986, gen loss = 3.6116
Iteration 69550: dis loss = 1.5799, gen loss = 5.3817
Iteration 69600: dis loss = 0.3182, gen loss = 2.5315
Iteration 69650: dis loss = 0.2649, gen loss = 3.0688
Iteration 69700: dis loss = 0.3626, gen loss = 2.8096
Iteration 69750: dis loss = 0.6476, gen loss = 3.9002
Iteration 69800: dis loss = 0.4707, gen loss = 1.5043
Iteration 69850: dis loss = 0.7866, gen loss = 1.6531
Iteration 69900: dis loss = 0.3611, gen loss = 3.5310
Iteration 69950: dis loss = 0.4088, gen loss = 2.4712
Iteration 70000: dis loss = 0.3125, gen loss = 4.1505
Iteration 70050: dis loss = 0.6394, gen loss = 2.6622
Iteration 70100: dis loss = 0.5856, gen loss = 4.1374
Iteration 70150: dis loss = 0.3703, gen loss = 2.3971
Iteration 70200: dis loss = 0.2354, gen loss = 4.5223
Iteration 70250: dis loss = 0.1607, gen loss = 3.5993






    












    












    












    



Iteration 70300: dis loss = 0.5112, gen loss = 1.9615
Iteration 70350: dis loss = 0.5916, gen loss = 5.0690
Iteration 70400: dis loss = 0.3616, gen loss = 2.5587
Iteration 70450: dis loss = 0.2387, gen loss = 3.1570
Iteration 70500: dis loss = 0.1950, gen loss = 3.4366
Iteration 70550: dis loss = 0.6010, gen loss = 3.3327
Iteration 70600: dis loss = 0.3387, gen loss = 3.3663
Iteration 70650: dis loss = 0.3353, gen loss = 2.4931
Iteration 70700: dis loss = 0.5306, gen loss = 4.5159
Iteration 70750: dis loss = 0.6378, gen loss = 0.9371
Iteration 70800: dis loss = 0.3519, gen loss = 3.2597
Iteration 70850: dis loss = 0.7471, gen loss = 1.3825
Iteration 70900: dis loss = 0.2665, gen loss = 4.5454
Iteration 70950: dis loss = 0.1753, gen loss = 3.4352
Iteration 71000: dis loss = 0.4512, gen loss = 4.7021
Iteration 71050: dis loss = 0.2578, gen loss = 3.7628
Iteration 71100: dis loss = 1.2702, gen loss = 6.0265
Iteration 71150: dis loss = 0.3651, gen loss = 2.7010
Iteration 71200: dis loss = 0.5011, gen loss = 3.6850
Iteration 71250: dis loss = 0.3992, gen loss = 2.7104
Iteration 71300: dis loss = 0.5832, gen loss = 5.6346
Iteration 71350: dis loss = 0.2790, gen loss = 3.8485
Iteration 71400: dis loss = 0.4793, gen loss = 1.6895
Iteration 71450: dis loss = 0.2219, gen loss = 3.7888
Iteration 71500: dis loss = 0.4468, gen loss = 4.3275
Iteration 71550: dis loss = 0.5387, gen loss = 2.3872
Iteration 71600: dis loss = 0.2055, gen loss = 3.6054
Iteration 71650: dis loss = 0.2263, gen loss = 3.3328
Iteration 71700: dis loss = 0.7059, gen loss = 1.2525
Iteration 71750: dis loss = 0.2255, gen loss = 3.2804
Iteration 71800: dis loss = 0.3178, gen loss = 2.4206
Iteration 71850: dis loss = 0.2910, gen loss = 2.7285






    












    












    












    



Iteration 71900: dis loss = 0.3089, gen loss = 3.4248
Iteration 71950: dis loss = 0.2459, gen loss = 4.2817
Iteration 72000: dis loss = 0.3913, gen loss = 2.8672
Iteration 72050: dis loss = 0.2235, gen loss = 3.6361
Iteration 72100: dis loss = 0.2266, gen loss = 2.8553
Iteration 72150: dis loss = 0.3648, gen loss = 3.6326
Iteration 72200: dis loss = 0.7705, gen loss = 4.6190
Iteration 72250: dis loss = 0.4410, gen loss = 4.0870
Iteration 72300: dis loss = 0.4806, gen loss = 2.1604
Iteration 72350: dis loss = 0.3156, gen loss = 2.4006
Iteration 72400: dis loss = 0.5647, gen loss = 5.1618
Iteration 72450: dis loss = 0.5018, gen loss = 2.7906
Iteration 72500: dis loss = 0.1330, gen loss = 3.7583
Iteration 72550: dis loss = 0.3227, gen loss = 4.4414
Iteration 72600: dis loss = 0.2688, gen loss = 3.1575
Iteration 72650: dis loss = 0.4400, gen loss = 1.9782
Iteration 72700: dis loss = 0.6853, gen loss = 3.8172
Iteration 72750: dis loss = 0.3964, gen loss = 2.6776
Iteration 72800: dis loss = 0.3136, gen loss = 4.2289
Iteration 72850: dis loss = 0.4726, gen loss = 2.4924
Iteration 72900: dis loss = 0.3297, gen loss = 4.1558
Iteration 72950: dis loss = 0.2885, gen loss = 3.6915
Iteration 73000: dis loss = 0.7421, gen loss = 1.3523
Iteration 73050: dis loss = 0.1263, gen loss = 3.9055
Iteration 73100: dis loss = 0.3592, gen loss = 3.6330
Iteration 73150: dis loss = 0.1410, gen loss = 3.6495
Iteration 73200: dis loss = 0.2979, gen loss = 4.5936
Iteration 73250: dis loss = 0.7361, gen loss = 1.4800
Iteration 73300: dis loss = 0.4838, gen loss = 2.1607
Iteration 73350: dis loss = 0.4363, gen loss = 3.7492
Iteration 73400: dis loss = 0.3481, gen loss = 2.9540






    












    












    












    



Iteration 73450: dis loss = 0.6029, gen loss = 1.3097
Iteration 73500: dis loss = 0.4126, gen loss = 4.2107
Iteration 73550: dis loss = 0.2376, gen loss = 3.7922
Iteration 73600: dis loss = 0.2911, gen loss = 4.2860
Iteration 73650: dis loss = 0.1396, gen loss = 3.4438
Iteration 73700: dis loss = 0.3524, gen loss = 4.3028
Iteration 73750: dis loss = 0.2026, gen loss = 3.6244
Iteration 73800: dis loss = 0.2556, gen loss = 3.7536
Iteration 73850: dis loss = 0.3826, gen loss = 2.6610
Iteration 73900: dis loss = 0.8244, gen loss = 3.9474
Iteration 73950: dis loss = 0.2391, gen loss = 3.7142
Iteration 74000: dis loss = 0.6836, gen loss = 4.6957
Iteration 74050: dis loss = 0.3775, gen loss = 2.7628
Iteration 74100: dis loss = 0.3243, gen loss = 2.7452
Iteration 74150: dis loss = 0.3731, gen loss = 4.9832
Iteration 74200: dis loss = 0.6719, gen loss = 1.4365
Iteration 74250: dis loss = 0.2243, gen loss = 2.4179
Iteration 74300: dis loss = 0.5354, gen loss = 2.1263
Iteration 74350: dis loss = 0.5944, gen loss = 1.4814
Iteration 74400: dis loss = 0.4419, gen loss = 2.2353
Iteration 74450: dis loss = 0.1465, gen loss = 4.0733
Iteration 74500: dis loss = 0.1869, gen loss = 4.2082
Iteration 74550: dis loss = 0.3426, gen loss = 2.6396
Iteration 74600: dis loss = 0.6553, gen loss = 2.0721
Iteration 74650: dis loss = 0.4395, gen loss = 4.1152
Iteration 74700: dis loss = 0.3631, gen loss = 2.6262
Iteration 74750: dis loss = 0.1419, gen loss = 3.6995
Iteration 74800: dis loss = 0.6730, gen loss = 1.2266
Iteration 74850: dis loss = 0.1982, gen loss = 3.9896
Iteration 74900: dis loss = 0.4467, gen loss = 2.1533
Iteration 74950: dis loss = 0.8123, gen loss = 4.5835






    












    












    












    



Iteration 75000: dis loss = 0.7504, gen loss = 1.0798
Iteration 75050: dis loss = 0.1777, gen loss = 3.0811
Iteration 75100: dis loss = 0.7847, gen loss = 1.0592
Iteration 75150: dis loss = 0.3141, gen loss = 3.4036
Iteration 75200: dis loss = 0.3855, gen loss = 2.0016
Iteration 75250: dis loss = 0.4447, gen loss = 4.8625
Iteration 75300: dis loss = 0.9110, gen loss = 5.5852
Iteration 75350: dis loss = 0.2417, gen loss = 3.5754
Iteration 75400: dis loss = 0.3353, gen loss = 2.8973
Iteration 75450: dis loss = 0.3584, gen loss = 3.9748
Iteration 75500: dis loss = 0.2206, gen loss = 3.4254
Iteration 75550: dis loss = 0.4011, gen loss = 3.5605
Iteration 75600: dis loss = 0.4212, gen loss = 4.0327
Iteration 75650: dis loss = 0.4715, gen loss = 2.5252
Iteration 75700: dis loss = 0.4013, gen loss = 3.4274
Iteration 75750: dis loss = 0.5947, gen loss = 6.3892
Iteration 75800: dis loss = 1.0929, gen loss = 1.0603
Iteration 75850: dis loss = 0.4228, gen loss = 4.1557
Iteration 75900: dis loss = 0.3144, gen loss = 3.1943
Iteration 75950: dis loss = 0.1661, gen loss = 4.6082
Iteration 76000: dis loss = 0.7993, gen loss = 1.5598
Iteration 76050: dis loss = 0.2992, gen loss = 2.9387
Iteration 76100: dis loss = 0.2367, gen loss = 3.6518
Iteration 76150: dis loss = 0.4869, gen loss = 1.9253
Iteration 76200: dis loss = 0.3833, gen loss = 1.8332
Iteration 76250: dis loss = 0.1810, gen loss = 4.2149
Iteration 76300: dis loss = 0.8010, gen loss = 5.1322
Iteration 76350: dis loss = 0.6143, gen loss = 1.9846
Iteration 76400: dis loss = 0.4178, gen loss = 4.6461
Iteration 76450: dis loss = 0.1682, gen loss = 3.2902
Iteration 76500: dis loss = 0.4871, gen loss = 3.7839






    












    












    












    



Iteration 76550: dis loss = 0.2099, gen loss = 3.1023
Iteration 76600: dis loss = 0.1991, gen loss = 3.7490
Iteration 76650: dis loss = 0.2680, gen loss = 2.8631
Iteration 76700: dis loss = 0.2918, gen loss = 3.6909
Iteration 76750: dis loss = 0.3660, gen loss = 2.5287
Iteration 76800: dis loss = 0.5628, gen loss = 1.8170
Iteration 76850: dis loss = 0.6184, gen loss = 3.3639
Iteration 76900: dis loss = 0.1895, gen loss = 3.7657
Iteration 76950: dis loss = 0.9409, gen loss = 5.5208
Iteration 77000: dis loss = 0.2542, gen loss = 3.8980
Iteration 77050: dis loss = 0.2724, gen loss = 2.3369
Iteration 77100: dis loss = 0.2330, gen loss = 3.9209
Iteration 77150: dis loss = 0.2191, gen loss = 3.2222
Iteration 77200: dis loss = 0.2622, gen loss = 2.1835
Iteration 77250: dis loss = 0.2546, gen loss = 3.0998
Iteration 77300: dis loss = 0.2378, gen loss = 4.7511
Iteration 77350: dis loss = 0.3489, gen loss = 3.5259
Iteration 77400: dis loss = 0.9171, gen loss = 4.4246
Iteration 77450: dis loss = 0.4109, gen loss = 3.9423
Iteration 77500: dis loss = 0.3001, gen loss = 3.4352
Iteration 77550: dis loss = 0.4378, gen loss = 2.4712
Iteration 77600: dis loss = 0.1758, gen loss = 3.9899
Iteration 77650: dis loss = 0.6889, gen loss = 1.5057
Iteration 77700: dis loss = 0.2889, gen loss = 3.1034
Iteration 77750: dis loss = 0.6635, gen loss = 1.4290
Iteration 77800: dis loss = 0.8571, gen loss = 1.7361
Iteration 77850: dis loss = 0.1552, gen loss = 3.4527
Iteration 77900: dis loss = 0.4039, gen loss = 3.5536
Iteration 77950: dis loss = 0.3088, gen loss = 2.9073
Iteration 78000: dis loss = 0.2480, gen loss = 3.2319
Iteration 78050: dis loss = 0.5017, gen loss = 3.7572
Iteration 78100: dis loss = 0.4929, gen loss = 2.2804

Problem 2-2: The forger versus the police, revisited

In the forger versus police story, we made part of it hand-wavy to hide a flaw that makes the story improbable to actually happen and makes it a bad analogy of how the training works in a GAN. Now that you have implemented a GAN, can you spot the flaw?

Specifically, when we consider one of the two parties, the other is treated as a black box. They know their opponent's result but not how they works. What is wrong here?

Your answer below:

In the first round, the police gets the fake bill and learns that the forgeries are white while the real bills are green. The forger then finds out that white papers can no longer fool the police and starts to produce green papers.

There is an inherent assumption that the forger (the generator) will always tend to improve upon the previous iteration by somehow correctly guessing what features to produce. So the fact that the white papers do no work anymore is known, but the fact that the forger will know that producing green papers is closer to the real bills, is not coherent.

The Tao of GANs: They might be adversaries, yet they are also cooperative.

Problem 2-3: The Batch Normalization dilemma

Here are three questions related to the use of Batch Normalization in GANs. The first two will not be graded and their answers are provided. But you should attempt to solve them before looking at the answer.

We made separate batches for real samples and fake samples when training the discriminator. Is this just an arbitrary design decision made by the inventor that later becomes the common practice, or is it critical to the correctness of the algorithm?

Select text below to see answer:

When we are training the generator, the input batch to the discriminator will always consist of only fake samples. If we separate real and fake batches when training the discriminator, then the fake samples are normalized in the same way when we are training the discriminator and when we are training the generator. If we mix real and fake samples in the same batch when training the discriminator, then the fake samples are not normalized in the same way when we train the two networks, which causes the generator to fail to learn the correct distribution.

Look at the construction of the discriminator carefully. You will find that between dis_conv1 and dis_lrelu1 there is no batch normalization. This is not a mistake. What could go wrong if there were a batch normalization layer there? Why do you think that omitting this batch normalization layer solves the problem practically if not theoretically?

Select text below to see answer:

Since we put real samples and fake samples in separate batches, if we add a batch normalization layer between dis_conv1 and dis_lrelu1, the discriminator would not be able to distinguish two distributions if one can be obtained by applying an isotropic scaling and a translation in color space to the other.

By removing the first batch normalization layer, for two different distributions to get confused with each other they must produce two distributions after dis_lrelu1 such that one can be obtained by applying an isotropic scaling and a translation to the other. Such a case is still possible but extremely unlikely to happen.

Propose a different way of feeding the samples to solve the problem in the second question without omitting any batch normalization layers or changing their mode of operation.

Your answer below:

Take-aways from this problem: always excercise extreme caution when using batch normalization in your network!

Advertisement: read the TA's paper to find out more about why Batch Normalization might be bad for your GANs: On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Problem 2-4: Activation Maximization

Activation Maximization is a visualization technique to see what a particular neuron has learned, by finding the input that maximizes the activation of that neuron. Here we use methods similar to Synthesizing the preferred inputs for neurons in neural networks via deep generator networks.

In short, what we want to do is to find the samples that the discriminator considers most real, among all possible outputs of the generator, which is to say, we want to find the codes (i.e. a point in the input space of the generator) from which the generated images, if labelled as real, would minimize the classification loss of the discriminator:

$$\min_{z} L(D_{\theta}(G_{\phi}(z)), 1)$$

Compare this to the objective when we were training the generator:

$$\min_{\phi} \mathrm{E}_{z \sim q(z)}[L(D_{\theta}(G_{\phi}(z)), 1)]$$

The function to minimize is the same, with the difference being that when training the network we fix a set of input data and find the optimal model parameters, while in activation maximization we fix the model parameters and find the optimal input.

So, similar to the training, we use gradient descent to solve for the optimal input. Starting from a random code drawn from a standard normal distribution, we perform a fixed step of Adam optimization algorithm on the code.

The batch normalization layers should work in evaluation mode.

We provide the code for this part, as a reference for solving the next part. You may want to go back to the code above and check the following:

Build the computation graph for the loss: Check the definition of these operations in _init_ops()
- actmax_sample_op: generate famples from actmax_code
- actmax_loss_op: compute discriminator's loss on samples generated from actmax_code
Define the optimizer: Check the definition of actmax_op, which updates actmax_code
Feed the data: Check the function actmax_one_sample()



In [6]:

    
tf.reset_default_graph()

with tf.Session() as sess:
    with tf.device('/cpu:0'):
        dcgan = DCGAN()
        sess.run(tf.global_variables_initializer())
        dis_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'dis')
        gen_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'gen')
        saver = tf.train.Saver(dis_var_list + gen_var_list)
        saver.restore(sess, 'model/dcgan')
        actmax_results = dcgan.actmax(np.random.random([64, dcgan.code_size]))
        fig = plt.figure(figsize = (8, 8))   
        ax1 = plt.subplot(111)
        ax1.imshow(viz_grid(actmax_results, 1))
        plt.show()









    



INFO:tensorflow:Restoring parameters from model/dcgan

The output should have less variety than those generated from random code. While it is reasonable that the samples that are "most real" makes up only a small portion of the sample space, this also gives us a hint that the so-called "mode collapse", in which the GAN simply fails to model a majority part of the data distribution, is a real problem.

A similar technique can be used to reconstruct a test sample, that is, to find the code that most closely approximates the test sample. To achieve this, we only need to change the loss function from discriminator's loss to the squared L2-distance between the generated image and the terget image:

$$\min_{z} \left|\left|G_{\phi}(z)-x\right|\right|_2^2$$

This time, we always start from a zero vector.

For this part, you need to complete code blocks marked with "Prob 2-4":

Build the computation graph for the loss: Complete the definition of recon_loss_op in _init_ops(), which computes the squared L2-distance between recon_sample and the sample generated from actmax_code.
Define the optimizer: Modify the definition of reconstruct_op so that it updates actmax_code rather than the parameters of the networks.
Feed the data: Set the proper initial value and feed the proper data in reconstruct_one_sample()



In [7]:

    
tf.reset_default_graph()

with tf.Session() as sess:
    with tf.device('/cpu:0'):
        dcgan = DCGAN()
        sess.run(tf.global_variables_initializer())
        dis_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'dis')
        gen_var_list = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'gen')
        saver = tf.train.Saver(dis_var_list + gen_var_list)
        saver.restore(sess, 'model/dcgan')

        avg_loss, reconstructions = dcgan.reconstruct(test_samples[0:64])
        print('average reconstruction loss = {0:.4f}'.format(avg_loss))
        fig = plt.figure(figsize = (8, 8))   
        ax1 = plt.subplot(111)
        ax1.imshow(viz_grid(test_samples[0:64], 1))
        plt.show()
        fig = plt.figure(figsize = (8, 8))   
        ax1 = plt.subplot(111)
        ax1.imshow(viz_grid(reconstructions, 1))
        plt.show()









    



INFO:tensorflow:Restoring parameters from model/dcgan
average reconstruction loss = 3.9316

Now you can increase the training epochs. You should be able to achieve a reconstruction loss lower than 30.