B `@sdZddlmZddlmZddlmZddlmmZddl m Z ddl m Z dd d d gZ dd dZ dd d Zddd Zddd ZdddZdddZddZddZdS)z1Utilities for initializing neural network layers.)absolute_import)division)print_functionN) dtype_util) prefer_static glorot_normalglorot_uniform he_normal he_uniformcsdfdd S)aPThe Glorot normal initializer, aka Xavier normal initializer. It draws samples from a truncated normal distribution centered on 0 with standard deviation (after truncation) given by `stddev = sqrt(2 / (fan_in + fan_out))` where `fan_in` is the number of input units in the weight tensor and `fan_out` is the number of output units in the weight tensor. Args: seed: A Python integer. Used to create random seeds. Default value: `None`. Returns: init_fn: A python `callable` which takes a shape `Tensor`, dtype and an optional scalar `int` number of batch dims and returns a randomly initialized `Tensor` with the specified shape and dtype. References: [Glorot et al., 2010](http://proceedings.mlr.press/v9/glorot10a.html) ([pdf](http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf)) rc st|||ddddS)Ng?fan_avgtruncated_normal)scalemode distributionseed) _initialize)shapedtype batch_ndims)rv/home/dcms/DCMS/lib/python3.7/site-packages/tensorflow_probability/python/experimental/nn/initializers/initializers.py8szglorot_normal..)rr)rr)rrr"scsdfdd S)a$The Glorot uniform initializer, aka Xavier uniform initializer. It draws samples from a uniform distribution within [-limit, limit] where `limit` is `sqrt(6 / (fan_in + fan_out))` where `fan_in` is the number of input units in the weight tensor and `fan_out` is the number of output units in the weight tensor. Args: seed: A Python integer. Used to create random seeds. Default value: `None`. Returns: init_fn: A python `callable` which takes a shape `Tensor`, dtype and an optional scalar `int` number of batch dims and returns a randomly initialized `Tensor` with the specified shape and dtype. References: [Glorot et al., 2010](http://proceedings.mlr.press/v9/glorot10a.html) ([pdf](http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf)) rc st|||ddddS)Ng?r uniform)r rrr)r)rrr)rrrrRsz glorot_uniform..)rr)rr)rrr=scsdfdd S)a>He normal initializer. It draws samples from a truncated normal distribution centered on 0 with standard deviation (after truncation) given by `stddev = sqrt(2 / fan_in)` where `fan_in` is the number of input units in the weight tensor. Args: seed: A Python integer. Used to create random seeds. Default value: `None`. Returns: init_fn: A python `callable` which takes a shape `Tensor`, dtype and an optional scalar `int` number of batch dims and returns a randomly initialized `Tensor` with the specified shape and dtype. References: [He et al., 2015](https://www.cv-foundation.org/openaccess/content_iccv_2015/html/He_Delving_Deep_into_ICCV_2015_paper.html) ([pdf](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf)) rc st|||ddddS)Ng@fan_inr )r rrr)r)rrr)rrrrnszhe_normal..)rr)rr)rrr Wscsdfdd S)a'He uniform variance scaling initializer. It draws samples from a uniform distribution within [-limit, limit] where `limit` is `sqrt(6 / fan_in)` where `fan_in` is the number of input units in the weight tensor. Args: seed: A Python integer. Used to create random seeds. Default value: `None`. Returns: init_fn: A python `callable` which takes a shape `Tensor`, dtype and an optional scalar `int` number of batch dims and returns a randomly initialized `Tensor` with the specified shape and dtype. References: [He et al., 2015] (https://www.cv-foundation.org/openaccess/content_iccv_2015/html/He_Delving_Deep_into_ICCV_2015_paper.html) ([pdf](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf)) rc st|||ddddS)Ng@rr)r rrr)r)rrr)rrrrszhe_uniform..)rr)rr)rrr ssc Csdt|std|tj|dgd}t||\}}t||||} t||}t ||| |||S)z-Samples a random `Tensor` per specified args.z0Argument `dtype` must be float type (saw: "{}").)r) rZ is_floating TypeErrorformatrZreshape_compute_fans_from_shape_summarize_fanscast_sample_distribution) rrrr rrrrfan_outZfansrrrrs  rcCsrtddt|}tj|d|ggdd}t||dddg\}}}}t|}|d|}|d|}||fS)z9Extracts `fan_in, fan_out` from specified shape `Tensor`.r)ZpaddingsZconstant_valuesr)rmaximumsizepadsplitZ reduce_prod)rrZnum_padZ batch_shapeZ extra_shaperr!Zreceptive_field_sizerrrrs   rcCsbt||}t||}t|}|dkr0|S|dkr<|S|dkrP||dStd|dS)z2Combines `fan_in`, `fan_out` per specified `mode`.rr!r g@zUnrecognized mode: "{}".N)rrstrlower ValueErrorr)rr!rrrrrrs    rcCst|}|dkr8t|d}tjj|d|||dS|dkrftd|}tjj|| |||dS|dkrt|}tjj|d|||dSt d |d S) zHSamples from specified distribution (by appropriately scaling `var` arg.r g۶%?g)rrg@Zuntruncated_normalz Unrecognized distribution: "{}".N) r(r)rsqrttfrandomr rnormalr*r)rvarrrrstddevlimitrrrr s  r )N)N)N)N)N)r)__doc__ __future__rrrZtensorflow.compat.v2compatZv2r,Z&tensorflow_probability.python.internalrr__all__rrr r rrrr rrrrs$