B `L@s~dZddlmZddlmZddlmZddlZddlZddlmm Z ddl mm Z ddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZdddgZej ddddGdddej!e"dddddddddgZ#Gd d!d!ej!e"d!d"gZ$e%d#d$d/d'dZ&Gd(ddej'Z(Gd)ddej'Z)d0d*d+Z*d1d-d.Z+dS)2z9Hamiltonian Monte Carlo, a gradient-based MCMC algorithm.)absolute_import)division)print_functionN) dtype_util) prefer_static)samplers)kernel)metropolis_hastings)leapfrog_integrator)util) SeedStream) deprecationHamiltonianMonteCarlo!UncalibratedHamiltonianMonteCarlo#make_simple_step_size_update_policyalwaysztensorflow_probability.*hmcT)moduleappendc@seZdZdZdZdS).UncalibratedHamiltonianMonteCarloKernelResultsz4Internal state and diagnostics for Uncalibrated HMC.N)__name__ __module__ __qualname____doc__ __slots__rrrU/home/dcms/DCMS/lib/python3.7/site-packages/tensorflow_probability/python/mcmc/hmc.pyr4srlog_acceptance_correctiontarget_log_probgrads_target_log_probinitial_momentumfinal_momentum step_sizenum_leapfrog_stepsseedc@seZdZdZdS)'HamiltonianMonteCarloExtraKernelResultsrN)rrrrrrrrr$Hsr$step_size_assignz 2019-05-22z.Use tfp.mcmc.SimpleStepSizeAdaptation instead.?{Gz?csJdkr2dk r2tjdtjdtjdtjdddfdd }|S) a Create a function implementing a step-size update policy. The simple policy increases or decreases the `step_size_var` based on the average of `exp(minimum(0., log_accept_ratio))`. It is based on [Section 4.2 of Andrieu and Thoms (2008)]( https://people.eecs.berkeley.edu/~jordan/sail/readings/andrieu-thoms.pdf). The `num_adaptation_steps` argument is set independently of any burnin for the overall chain. In general, adaptation prevents the chain from reaching a stationary distribution, so obtaining consistent samples requires `num_adaptation_steps` be set to a value [somewhat smaller]( http://andrewgelman.com/2017/12/15/burn-vs-warm-iterative-simulation-algorithms/#comment-627745) than the number of burnin steps. However, it may sometimes be helpful to set `num_adaptation_steps` to a larger value during development in order to inspect the behavior of the chain during adaptation. Args: num_adaptation_steps: Scalar `int` `Tensor` number of initial steps to during which to adjust the step size. This may be greater, less than, or equal to the number of burnin steps. If `None`, the step size is adapted on every step (note this breaks stationarity of the chain!). target_rate: Scalar `Tensor` representing desired `accept_ratio`. Default value: `0.75` (i.e., [center of asymptotically optimal rate](https://arxiv.org/abs/1411.6669)). decrement_multiplier: `Tensor` representing amount to downscale current `step_size`. Default value: `0.01`. increment_multiplier: `Tensor` representing amount to upscale current `step_size`. Default value: `0.01`. step_counter: Scalar `int` `Variable` specifying the current step. The step size is adapted iff `step_counter < num_adaptation_steps`. Default value: if `None`, an internal variable `step_size_adaptation_step_counter` is created and initialized to `-1`. Returns: step_size_simple_update_fn: Callable that takes args `step_size_var, kernel_results` and returns updated step size(s). NZ!step_size_adaptation_step_counter)dtypeFT)nameZ initializerr)Z trainableZ use_resourcec s|dkr*tr ddDStStjtt|j|jj }t t |jd|}t |ttj|j k dfdd}dkr|St dgtjk|fd d d SQRXdS) aUpdates (list of) `step_size` using a standard adaptive MCMC procedure. Args: step_size_var: (List of) `tf.Variable`s representing the per `state_part` HMC `step_size`. kernel_results: `collections.namedtuple` containing `Tensor`s representing values from most recent call to `one_step`. Returns: step_size_assign: (List of) `Tensor`(s) representing updated `step_size_var`(s). NcSsg|]}t|qSr)tfidentity).0ssrrr sz[make_simple_step_size_update_policy..step_size_simple_update_fn..gg?cs4trfddDStjS)Nc s$g|]}||t|jqSr) assign_addr+castr))r-r.) adjustmentrrr/sztmake_simple_step_size_update_policy..step_size_simple_update_fn..build_assign_op..) mcmc_util is_list_liker0r+r1r)r)r2 step_size_varrrbuild_assign_ops   z`make_simple_step_size_update_policy..step_size_simple_update_fn..build_assign_opcsS)Nrr)r5rrzYmake_simple_step_size_update_policy..step_size_simple_update_fn..)predZtrue_fnZfalse_fn)r3r4r+r,mathlogr1sizeZlog_accept_ratior)Zreduce_logsumexpZminimumwherecontrol_dependenciesr0Zcond)r5kernel_resultsZlog_nZlog_mean_accept_ratior6)decrement_multiplierincrement_multipliernum_adaptation_steps step_counter target_rate)r2r5rstep_size_simple_update_fns.      zGmake_simple_step_size_update_policy..step_size_simple_update_fn)tf1Z get_variabler+Zconstantint32)rCrErArBrDrFr)rArBrCrDrErrPs.0c@seZdZdZedddedddd"d d Zed d ZeddZ eddZ eddZ eddZ eddZ eddZeddZeddZd#ddZd d!Zd S)$ra Runs one step of Hamiltonian Monte Carlo. Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) algorithm that takes a series of gradient-informed steps to produce a Metropolis proposal. This class implements one random HMC step from a given `current_state`. Mathematical details and derivations can be found in [Neal (2011)][1]. The `one_step` function can update multiple chains in parallel. It assumes that all leftmost dimensions of `current_state` index independent chain states (and are therefore updated independently). The output of `target_log_prob_fn(*current_state)` should sum log-probabilities across all event dimensions. Slices along the rightmost dimensions may have different target distributions; for example, `current_state[0, :]` could have a different target distribution from `current_state[1, :]`. These semantics are governed by `target_log_prob_fn(*current_state)`. (The number of independent chains is `tf.size(target_log_prob_fn(*current_state))`.) #### Examples: ##### Simple chain with warm-up. In this example we sample from a standard univariate normal distribution using HMC with adaptive step size. ```python import tensorflow as tf import tensorflow_probability as tfp tf.enable_eager_execution() # Target distribution is proportional to: `exp(-x (1 + x))`. def unnormalized_log_prob(x): return -x - x**2. # Initialize the HMC transition kernel. num_results = int(10e3) num_burnin_steps = int(1e3) adaptive_hmc = tfp.mcmc.SimpleStepSizeAdaptation( tfp.mcmc.HamiltonianMonteCarlo( target_log_prob_fn=unnormalized_log_prob, num_leapfrog_steps=3, step_size=1.), num_adaptation_steps=int(num_burnin_steps * 0.8)) # Run the chain (with burn-in). @tf.function def run_chain(): # Run the chain (with burn-in). samples, is_accepted = tfp.mcmc.sample_chain( num_results=num_results, num_burnin_steps=num_burnin_steps, current_state=1., kernel=adaptive_hmc, trace_fn=lambda _, pkr: pkr.inner_results.is_accepted) sample_mean = tf.reduce_mean(samples) sample_stddev = tf.math.reduce_std(samples) is_accepted = tf.reduce_mean(tf.cast(is_accepted, dtype=tf.float32)) return sample_mean, sample_stddev, is_accepted sample_mean, sample_stddev, is_accepted = run_chain() print('mean:{:.4f} stddev:{:.4f} acceptance:{:.4f}'.format( sample_mean.numpy(), sample_stddev.numpy(), is_accepted.numpy())) ``` ##### Estimate parameters of a more complicated posterior. In this example, we'll use Monte-Carlo EM to find best-fit parameters. See [_Convergence of a stochastic approximation version of the EM algorithm_][2] for more details. More precisely, we use HMC to form a chain conditioned on parameter `sigma` and training data `{ (x[i], y[i]) : i=1...n }`. Then we use one gradient step of maximum-likelihood to improve the `sigma` estimate. Then repeat the process until convergence. (This procedure is a [Robbins--Monro algorithm]( https://en.wikipedia.org/wiki/Stochastic_approximation).) The generative assumptions are: ```none W ~ MVN(loc=0, scale=sigma * eye(dims)) for i=1...num_samples: X[i] ~ MVN(loc=0, scale=eye(dims)) eps[i] ~ Normal(loc=0, scale=1) Y[i] = X[i].T * W + eps[i] ``` We now implement a stochastic approximation of Expectation Maximization (SAEM) using `tensorflow_probability` intrinsics. [Bernard (1999)][2] ```python import tensorflow as tf import tensorflow_probability as tfp import numpy as np tf.enable_eager_execution() tfd = tfp.distributions def make_training_data(num_samples, dims, sigma): dt = np.asarray(sigma).dtype x = np.random.randn(dims, num_samples).astype(dt) w = sigma * np.random.randn(1, dims).astype(dt) noise = np.random.randn(num_samples).astype(dt) y = w.dot(x) + noise return y[0], x, w[0] def make_weights_prior(dims, log_sigma): return tfd.MultivariateNormalDiag( loc=tf.zeros([dims], dtype=log_sigma.dtype), scale_identity_multiplier=tf.math.exp(log_sigma)) def make_response_likelihood(w, x): if w.shape.ndims == 1: y_bar = tf.matmul(w[tf.newaxis], x)[0] else: y_bar = tf.matmul(w, x) return tfd.Normal(loc=y_bar, scale=tf.ones_like(y_bar)) # [n] # Setup assumptions. dtype = np.float32 num_samples = 500 dims = 10 tf.random.set_seed(10014) np.random.seed(10014) weights_prior_true_scale = np.array(0.3, dtype) y, x, _ = make_training_data( num_samples, dims, weights_prior_true_scale) log_sigma = tf.Variable(0., dtype=dtype, name='log_sigma') optimizer = tf.optimizers.SGD(learning_rate=0.01) @tf.function def mcem_iter(weights_chain_start, step_size): with tf.GradientTape() as tape: tape.watch(log_sigma) prior = make_weights_prior(dims, log_sigma) def unnormalized_posterior_log_prob(w): likelihood = make_response_likelihood(w, x) return ( prior.log_prob(w) + tf.reduce_sum(likelihood.log_prob(y), axis=-1)) # [m] def trace_fn(_, pkr): return ( pkr.inner_results.log_accept_ratio, pkr.inner_results.accepted_results.target_log_prob, pkr.inner_results.accepted_results.step_size) num_results = 2 weights, ( log_accept_ratio, target_log_prob, step_size) = tfp.mcmc.sample_chain( num_results=num_results, num_burnin_steps=0, current_state=weights_chain_start, kernel=tfp.mcmc.SimpleStepSizeAdaptation( tfp.mcmc.HamiltonianMonteCarlo( target_log_prob_fn=unnormalized_posterior_log_prob, num_leapfrog_steps=2, step_size=step_size, state_gradients_are_stopped=True, ), # Adapt for the entirety of the trajectory. num_adaptation_steps=2), trace_fn=trace_fn, seed=123) # We do an optimization step to propagate `log_sigma` after two HMC # steps to propagate `weights`. loss = -tf.reduce_mean(target_log_prob) avg_acceptance_ratio = tf.math.exp( tfp.math.reduce_logmeanexp(tf.minimum(log_accept_ratio, 0.))) optimizer.apply_gradients( [[tape.gradient(loss, log_sigma), log_sigma]]) weights_prior_estimated_scale = tf.math.exp(log_sigma) return (weights_prior_estimated_scale, weights[-1], loss, step_size[-1], avg_acceptance_ratio) num_iters = int(40) weights_prior_estimated_scale_ = np.zeros(num_iters, dtype) weights_ = np.zeros([num_iters + 1, dims], dtype) loss_ = np.zeros([num_iters], dtype) weights_[0] = np.random.randn(dims).astype(dtype) step_size_ = 0.03 for iter_ in range(num_iters): [ weights_prior_estimated_scale_[iter_], weights_[iter_ + 1], loss_[iter_], step_size_, avg_acceptance_ratio_, ] = mcem_iter(weights_[iter_], step_size_) tf.compat.v1.logging.vlog( 1, ('iter:{:>2} loss:{: 9.3f} scale:{:.3f} ' 'step_size:{:.4f} avg_acceptance_ratio:{:.4f}').format( iter_, loss_[iter_], weights_prior_estimated_scale_[iter_], step_size_, avg_acceptance_ratio_)) # Should converge to ~0.22. import matplotlib.pyplot as plt plt.plot(weights_prior_estimated_scale_) plt.ylabel('weights_prior_estimated_scale') plt.xlabel('iteration') ``` #### References [1]: Radford Neal. MCMC Using Hamiltonian Dynamics. _Handbook of Markov Chain Monte Carlo_, 2011. https://arxiv.org/abs/1206.1901 [2]: Bernard Delyon, Marc Lavielle, Eric, Moulines. _Convergence of a stochastic approximation version of the EM algorithm_, Ann. Statist. 27 (1999), no. 1, 94--128. https://projecteuclid.org/euclid.aos/1018031103 z 2019-05-22zbThe `step_size_update_fn` argument is deprecated. Use `tfp.mcmc.SimpleStepSizeAdaptation` instead.step_size_update_fnz 2020-09-20znThe `seed` argument is deprecated (but will work until removed). Pass seed to `tfp.mcmc.sample_chain` instead.r#FNc Cs|r|rtdt|dd|_|dkr*in t|d} |dkrDin t|d} tjfdtf|||||pld|d| i| |_|jjj |_ ||j d <||j d <dS) ajInitializes this transition kernel. Args: target_log_prob_fn: Python callable which takes an argument like `current_state` (or `*current_state` if it's a list) and returns its (possibly unnormalized) log-density under the target distribution. step_size: `Tensor` or Python `list` of `Tensor`s representing the step size for the leapfrog integrator. Must broadcast with the shape of `current_state`. Larger step sizes lead to faster progress, but too-large step sizes make rejection exponentially more likely. When possible, it's often helpful to match per-variable step sizes to the standard deviations of the target distribution in each variable. num_leapfrog_steps: Integer number of steps to run the leapfrog integrator for. Total progress per HMC step is roughly proportional to `step_size * num_leapfrog_steps`. state_gradients_are_stopped: Python `bool` indicating that the proposed new state be run through `tf.stop_gradient`. This is particularly useful when combining optimization over samples from the HMC chain. Default value: `False` (i.e., do not apply `stop_gradient`). step_size_update_fn: Python `callable` taking current `step_size` (typically a `tf.Variable`) and `kernel_results` (typically `collections.namedtuple`) and returns updated step_size (`Tensor`s). Default value: `None` (i.e., do not update `step_size` automatically). seed: Python integer to seed the random number generator. Deprecated, pass seed to `tfp.mcmc.sample_chain`. store_parameters_in_results: If `True`, then `step_size` and `num_leapfrog_steps` are written to and read from eponymous fields in the kernel results objects returned from `one_step` and `bootstrap_results`. This allows wrapper kernels to adjust those parameters on the fly. This is incompatible with `step_size_update_fn`, which must be set to `None`. name: Python `str` name prefixed to Ops created by this function. Default value: `None` (i.e., 'hmc_kernel'). znIt is invalid to simultaneously specify `step_size_update_fn` and set `store_parameters_in_results` to `True`.hmc)saltN)r# inner_kernelZ hmc_kernel)target_log_prob_fnr!r"state_gradients_are_stoppedr*store_parameters_in_resultsrIr#) ValueErrorr _seed_streamdictr ZMetropolisHastingsr_implrL parameterscopy _parameters) selfrMr!r"rNrIr#rOr*Z uhmc_kwargsZ mh_kwargsrrr__init__s$1  zHamiltonianMonteCarlo.__init__cCs |jjjS)N)rSrLrM)rWrrrrMsz(HamiltonianMonteCarlo.target_log_prob_fncCs |jjjS)aReturns the step_size parameter. If `store_parameters_in_results` argument to the initializer was set to `True`, this only returns the value of the `step_size` placed in the kernel results by the `bootstrap_results` method. The actual step size in that situation is governed by the `previous_kernel_results` argument to `one_step` method. Returns: step_size: A floating point `Tensor` or a list of such `Tensors`. )rSrLr!)rWrrrr!s zHamiltonianMonteCarlo.step_sizecCs |jjjS)aReturns the num_leapfrog_steps parameter. If `store_parameters_in_results` argument to the initializer was set to `True`, this only returns the value of the `num_leapfrog_steps` placed in the kernel results by the `bootstrap_results` method. The actual `num_leapfrog_steps` in that situation is governed by the `previous_kernel_results` argument to `one_step` method. Returns: num_leapfrog_steps: An integer `Tensor`. )rSrLr")rWrrrr"s z(HamiltonianMonteCarlo.num_leapfrog_stepscCs |jjjS)N)rSrLrN)rWrrrrNsz1HamiltonianMonteCarlo.state_gradients_are_stoppedcCs |jdS)NrI)rV)rWrrrrI sz)HamiltonianMonteCarlo.step_size_update_fncCs |jjjS)N)rSrLr#)rWrrrr# szHamiltonianMonteCarlo.seedcCs |jjjS)N)rSrLr*)rWrrrr*szHamiltonianMonteCarlo.namecCs|jS)z9Return `dict` of ``__init__`` arguments and their values.)rV)rWrrrrTsz HamiltonianMonteCarlo.parameterscCsdS)NTr)rWrrr is_calibratedsz#HamiltonianMonteCarlo.is_calibratedc Cs|jdkrgnt|jjr$|jjn|jjg}t|J|jj|||d\}}|jdk rz||j |}|j t |dd}||fSQRXdS)aRuns one iteration of Hamiltonian Monte Carlo. Args: current_state: `Tensor` or Python `list` of `Tensor`s representing the current state(s) of the Markov chain(s). The first `r` dimensions index independent chains, `r = tf.rank(target_log_prob_fn(*current_state))`. previous_kernel_results: `collections.namedtuple` containing `Tensor`s representing values from previous calls to this function (or from the `bootstrap_results` function.) seed: Optional, a seed for reproducible sampling. Returns: next_state: Tensor or Python list of `Tensor`s representing the state(s) of the Markov chain(s) after taking exactly one step. Has same type and shape as `current_state`. kernel_results: `collections.namedtuple` of internal calculations used to advance the chain. Raises: ValueError: if there isn't one `step_size` or a list with same length as `current_state`. N)r#)r%)extra) rIr3r4rZr%r+r?rSone_stepr!_replacer$)rW current_stateprevious_kernel_resultsr#Zprevious_step_size_assign next_stater@r%rrrr[s     zHamiltonianMonteCarlo.one_stepcCs:|j|}|jdk r6||jd}|jt|dd}|S)zCCreates initial `previous_kernel_results` using a supplied `state`.N)r%)rZ)rSbootstrap_resultsrIr!r\r$)rW init_stater@r%rrrr`Gs   z'HamiltonianMonteCarlo.bootstrap_results)FNNFN)N)rrrrr deprecated_argsrXpropertyrMr!r"rNrIr#r*rTrYr[r`rrrrrs0a8          )c@seZdZdZeddddddZed d Zed d Z ed dZ eddZ eddZ eddZ eddZeddZeddZeejjd ddZeejjddZdS)!raqRuns one step of Uncalibrated Hamiltonian Monte Carlo. Warning: this kernel will not result in a chain which converges to the `target_log_prob`. To get a convergent MCMC, use `HamiltonianMonteCarlo(...)` or `MetropolisHastings(UncalibratedHamiltonianMonteCarlo(...))`. For more details on `UncalibratedHamiltonianMonteCarlo`, see `HamiltonianMonteCarlo`. z 2020-09-20znThe `seed` argument is deprecated (but will work until removed). Pass seed to `tfp.mcmc.sample_chain` instead.r#FNc Cs^|dk rtrtd|s.tt||dt|dd|_t|||||||d|_d|_ dS)aInitializes this transition kernel. Args: target_log_prob_fn: Python callable which takes an argument like `current_state` (or `*current_state` if it's a list) and returns its (possibly unnormalized) log-density under the target distribution. step_size: `Tensor` or Python `list` of `Tensor`s representing the step size for the leapfrog integrator. Must broadcast with the shape of `current_state`. Larger step sizes lead to faster progress, but too-large step sizes make rejection exponentially more likely. When possible, it's often helpful to match per-variable step sizes to the standard deviations of the target distribution in each variable. num_leapfrog_steps: Integer number of steps to run the leapfrog integrator for. Total progress per HMC step is roughly proportional to `step_size * num_leapfrog_steps`. state_gradients_are_stopped: Python `bool` indicating that the proposed new state be run through `tf.stop_gradient`. This is particularly useful when combining optimization over samples from the HMC chain. Default value: `False` (i.e., do not apply `stop_gradient`). seed: Python integer to seed the random number generator. Deprecated, pass seed to `tfp.mcmc.sample_chain`. store_parameters_in_results: If `True`, then `step_size` and `num_leapfrog_steps` are written to and read from eponymous fields in the kernel results objects returned from `one_step` and `bootstrap_results`. This allows wrapper kernels to adjust those parameters on the fly. name: Python `str` name prefixed to Ops created by this function. Default value: `None` (i.e., 'hmc_kernel'). NzSpecifying a `seed` when running eagerly is not currently supported. To run in Eager mode with a seed, pass the seed to `tfp.mcmc.sample_chain`.)r!r"Zuncalibrated_hmc_one_step)rK)rMr!r"rNr#r*rO) r+Zexecuting_eagerlyNotImplementedErrorr3Z)warn_if_parameters_are_not_simple_tensorsrRr rQrV_momentum_dtype)rWrMr!r"rNr#rOr*rrrrX]s( z*UncalibratedHamiltonianMonteCarlo.__init__cCs |jdS)NrM)rV)rWrrrrMsz4UncalibratedHamiltonianMonteCarlo.target_log_prob_fncCs |jdS)aReturns the step_size parameter. If `store_parameters_in_results` argument to the initializer was set to `True`, this only returns the value of the `step_size` placed in the kernel results by the `bootstrap_results` method. The actual step size in that situation is governed by the `previous_kernel_results` argument to `one_step` method. Returns: step_size: A floating point `Tensor` or a list of such `Tensors`. r!)rV)rWrrrr!s z+UncalibratedHamiltonianMonteCarlo.step_sizecCs |jdS)aReturns the num_leapfrog_steps parameter. If `store_parameters_in_results` argument to the initializer was set to `True`, this only returns the value of the `num_leapfrog_steps` placed in the kernel results by the `bootstrap_results` method. The actual `num_leapfrog_steps` in that situation is governed by the `previous_kernel_results` argument to `one_step` method. Returns: num_leapfrog_steps: An integer `Tensor`. r")rV)rWrrrr"s z4UncalibratedHamiltonianMonteCarlo.num_leapfrog_stepscCs |jdS)NrN)rV)rWrrrrNsz=UncalibratedHamiltonianMonteCarlo.state_gradients_are_stoppedcCs |jdS)Nr#)rV)rWrrrr#sz&UncalibratedHamiltonianMonteCarlo.seedcCs |jdS)Nr*)rV)rWrrrr*sz&UncalibratedHamiltonianMonteCarlo.namecCs|jS)z9Return `dict` of ``__init__`` arguments and their values.)rV)rWrrrrTsz,UncalibratedHamiltonianMonteCarlo.parameterscCsdS)NFr)rWrrrrYsz/UncalibratedHamiltonianMonteCarlo.is_calibratedcCs |jdS)NrO)rV)rWrrr_store_parameters_in_resultssz>UncalibratedHamiltonianMonteCarlo._store_parameters_in_resultsc sxtt|jddV|jr.|j}|j}n |j}|j}t|j ||j |j d|j d\}}}} |dk rtt |}n&|jjdk rttjt |}t j|t|d} g} x@t| |D]2\} } | t jt| |jpt| j| dqWt|j ||}|| ||| \}}}}|j r.dd|D}fd d }t !|}|j"t#| ||||| ||d }|||fSQRXdS) NrJr[T) maybe_expandrN)n)shaper)r#cSsg|]}t|qSr)r+ stop_gradient)r-xrrrr/ sz>UncalibratedHamiltonianMonteCarlo.one_step..cstr|S|dS)Nr)r3r4)rk)r]rr maybe_flatten szAUncalibratedHamiltonianMonteCarlo.one_step..maybe_flatten)rrrrr r#)$r+ name_scoper3 make_namer*rfr!r" _prepare_argsrMrrrNrZ sanitize_seedrQZ original_seedwarningswarnZSEED_CTOR_ARG_DEPRECATION_MSGZ split_seedlenziprnormalrirerZ base_dtyper) leapfrog_implZSimpleLeapfrogIntegratorrrankr\"_compute_log_acceptance_correction)rWr]r^r#r!r"Zcurrent_state_parts step_sizesZcurrent_target_log_probZ"current_target_log_prob_grad_partsZseedsZcurrent_momentum_partsZ part_seedrkZ integratorZnext_momentum_partsZnext_state_partsZnext_target_log_probZnext_target_log_prob_grad_partsrlindependent_chain_ndimsZnew_kernel_resultsr)r]rr[s^      z*UncalibratedHamiltonianMonteCarlo.one_stepc stt|jddt|\}}|jr:dd|D}t|j|\}t t |tj tj |tj tj |ggt d}|jr|jtj fdd|jtj|jtjdd d }|SQRXdS) NrJr`cSsg|]}t|qSr)r+rj)r-rkrrrr/%szGUncalibratedHamiltonianMonteCarlo.bootstrap_results..)rrrrr r!r"r#cstj|jddS)Nr!)r)r*)r+convert_to_tensorr))rk)init_target_log_probrrr8=szEUncalibratedHamiltonianMonteCarlo.bootstrap_results..r")r)r*)r!r")r+rmr3rnr*prepare_state_partsrNmaybe_call_fn_and_gradsrMrZ zeros_likeZnestZ map_structurerZ zeros_seedrfr\r!rzr"rH)rWra_Zinit_grads_target_log_probresultr)r{rr`s2   z3UncalibratedHamiltonianMonteCarlo.bootstrap_results)FNFN)N)rrrrr rbrXrcrMr!r"rNr#r*rTrYrfr3Zset_docrr[r`rrrrrRs(  3           Ic sjt|p dRfddtfdd|D}tfdd|D}dt|| gSQRXdS) a: Helper to `kernel` which computes the log acceptance-correction. A sufficient but not necessary condition for the existence of a stationary distribution, `p(x)`, is "detailed balance", i.e.: ```none p(x'|x) p(x) = p(x|x') p(x') ``` In the Metropolis-Hastings algorithm, a state is proposed according to `g(x'|x)` and accepted according to `a(x'|x)`, hence `p(x'|x) = g(x'|x) a(x'|x)`. Inserting this into the detailed balance equation implies: ```none g(x'|x) a(x'|x) p(x) = g(x|x') a(x|x') p(x') ==> a(x'|x) / a(x|x') = p(x') / p(x) [g(x|x') / g(x'|x)] (*) ``` One definition of `a(x'|x)` which satisfies (*) is: ```none a(x'|x) = min(1, p(x') / p(x) [g(x|x') / g(x'|x)]) ``` (To see that this satisfies (*), notice that under this definition only at most one `a(x'|x)` and `a(x|x') can be other than one.) We call the bracketed term the "acceptance correction". In the case of UncalibratedHMC, the log acceptance-correction is not the log proposal-ratio. UncalibratedHMC augments the state-space with momentum, z. Assuming a standard Gaussian distribution for momentums, the chain eventually converges to: ```none p([x, z]) propto= target_prob(x) exp(-0.5 z**2) ``` Relating this back to Metropolis-Hastings parlance, for HMC we have: ```none p([x, z]) propto= target_prob(x) exp(-0.5 z**2) g([x, z] | [x', z']) = g([x', z'] | [x, z]) ``` In other words, the MH bracketed term is `1`. However, because we desire to use a general MH framework, we can place the momentum probability ratio inside the metropolis-correction factor thus getting an acceptance probability: ```none target_prob(x') accept_prob(x'|x) = ----------------- [exp(-0.5 z**2) / exp(-0.5 z'**2)] target_prob(x) ``` (Note: we actually need to handle the kinetic energy change at each leapfrog step, but this is the idea.) Args: current_momentums: `Tensor` representing the value(s) of the current momentum(s) of the state (parts). proposed_momentums: `Tensor` representing the value(s) of the proposed momentum(s) of the state (parts). independent_chain_ndims: Scalar `int` `Tensor` representing the number of leftmost `Tensor` dimensions which index independent chains. name: Python `str` name prefixed to Ops created by this function. Default value: `None` (i.e., 'compute_log_acceptance_correction'). Returns: log_acceptance_correction: `Tensor` representing the `log` acceptance-correction. (See docstring for mathematical definition.) Z!compute_log_acceptance_correctioncs tj|dtt|dS)Ng@)Zaxis)r+Z reduce_sumrrangerv)v)ryrrr8sz4_compute_log_acceptance_correction..csg|] }|qSrr)r-r)sum_sqrrr/sz6_compute_log_acceptance_correction..csg|] }|qSrr)r-r)rrrr/sg?N)r+rmZadd_nr3Zsafe_sum)Zcurrent_momentumsZproposed_momentumsryr*Zcurrent_kineticZproposed_kineticr)ryrrrwIs N rwFc stjdd\}}|r$dd|D}t||||\}}tj||jdd\} }t| dkrf| t|9} t|t| kr~tdfd d } | || | ||gS) z@Helper which processes input args to meet list-like assumptions.r])r*cSsg|]}t|qSr)r+rj)r-rkrrrr/sz!_prepare_args..r!)r)r*r7zYThere should be exactly one `step_size` or it should have same length as `current_state`.csstr|S|dS)Nr)r3r4)rk)rgstaterrrlsz$_prepare_args..maybe_flatten)r3r|r}r)rrrP) rMrr!rrrgrNZ state_partsr~rxrlr)rgrrros   ro)r&r'r'N)N)NNFF),r __future__rrr collectionsrpZtensorflow.compat.v1compatZv1rGZtensorflow.compat.v2Zv2r+Z&tensorflow_probability.python.internalrrrZ"tensorflow_probability.python.mcmcrZ kernel_baser Z+tensorflow_probability.python.mcmc.internalr rur r3Z.tensorflow_probability.python.util.seed_streamr Ztensorflow.python.utilr __all__filterwarningsZPrettyNamedTupleMixin namedtuplerr$ deprecatedrZTransitionKernelrrrwrorrrrsn                 e{ V