B /`\@sddlZddlZddlZddlmZddlmZddl m Z eeej dddZ eeeeej dd d Z eeeeej d d d Zeeeej dddZd,eeee eej dddZeeeej dddZeeeej dddZeeeeej dddZeeej dddZGdd d ejjjZeeeeej dd!d"Zeeej d#d$d%Zeeeej d&d'd(Zeeeeej d)d*d+ZdS)-N) TensorLike) typechecked)Optional)inputs tag_bitmapreturnc Cs.t||tt|ttd|j}|S)aConstrains the inputs to filter out certain tags at each time step. tag_bitmap limits the allowed tags at each input time step. This is useful when an observed output at a given time step needs to be constrained to a selected set of tags. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer. tag_bitmap: A [batch_size, max_seq_len, num_tags] boolean tensor representing all active tags at each index for which to calculate the unnormalized score. Returns: filtered_inputs: A [batch_size] vector of unnormalized sequence scores. z-inf)tfwherefillshapecastfloatdtype)rrfiltered_inputsrI/home/dcms/DCMS/lib/python3.7/site-packages/tensorflow_addons/text/crf.pycrf_filtered_inputss "r)r tag_indicessequence_lengthstransition_paramsrcsbtjtjdtjtjdfdd}fdd}tttdd||S)a Computes the unnormalized score for a tag sequence. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer. tag_indices: A [batch_size, max_seq_len] matrix of tag indices for which we compute the unnormalized score. sequence_lengths: A [batch_size] vector of true sequence lengths. transition_params: A [num_tags, num_tags] transition matrix. Returns: sequence_scores: A [batch_size] vector of unnormalized sequence scores. )rcstjtjdd}tt|ddg}tj|t|gdd}t|}t|ddg}tj||gdd}t|}tt dt||}|S)N)Zout_typer)axis) rr int32reshaperangeconcat zeros_like gather_ndr less_equal) batch_sizeZ batch_indsindicesZtag_indssequence_scores)rrrrr_single_seq_fnMs   z*crf_sequence_score.._single_seq_fncs$t}t}||}|S)N)crf_unary_scorecrf_binary_score) unary_scores binary_scoresr")rrrrrr _multi_seq_fn_s   z)crf_sequence_score.._multi_seq_fnr)rr rcondequalr )rrrrr#r(r)rrrrrcrf_sequence_score6s  r+)rrrrrcsftj|tjd}tjtjdt||fdd}fdd}ttt|dd||S)aComputes the unnormalized score of all tag sequences matching tag_bitmap. tag_bitmap enables more than one tag to be considered correct at each time step. This is useful when an observed output at a given time step is consistent with more than one tag, and thus the log likelihood of that observation must take into account all possible consistent tags. Using one-hot vectors in tag_bitmap gives results identical to crf_sequence_score. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer. tag_bitmap: A [batch_size, max_seq_len, num_tags] boolean tensor representing all active tags at each index for which to calculate the unnormalized score. sequence_lengths: A [batch_size] vector of true sequence lengths. transition_params: A [num_tags, num_tags] transition matrix. Returns: sequence_scores: A [batch_size] vector of unnormalized sequence scores. )rcstjddgddS)NrF)rZkeepdims)rreduce_logsumexpr)rrrr#sz3crf_multitag_sequence_score.._single_seq_fncstdS)N)rrr) crf_log_normr)rrrrrr(sz2crf_multitag_sequence_score.._multi_seq_fnr)rr boolrrr)r*r )rrrrr#r(r)rrrrcrf_multitag_sequence_scoreks    r0)rrrrcsxtjtjdtdddgdddgtdgfdd}fdd}tttdd||S) aComputes the normalization for a CRF. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer. sequence_lengths: A [batch_size] vector of true sequence lengths. transition_params: A [num_tags, num_tags] transition matrix. Returns: log_norm: A [batch_size] vector of normalizers for a CRF. )rrrrcs.tdg}ttdt||}|S)Nrr)rr-r rr)log_norm) first_inputrrrr#sz$crf_log_norm.._single_seq_fncsVtdddgdddg}t|}t|dg}ttdt||}|S)z$Forward computation of alpha values.rrr)rslice crf_forwardr-r rr)Z rest_of_inputalphasr1)r2rrrrrr(s z#crf_log_norm.._multi_seq_fn)rr rr3squeezer)r*r )rrrr#r(r)r2rrrrr.s r.c Cst|}|jd}tj|tjd}tj|tjd}|dkr\tjj}t|||gd}t||j }t ||||}t |||}||}||fS)aComputes the log-likelihood of tag sequences in a CRF. Args: inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials to use as input to the CRF layer. tag_indices: A [batch_size, max_seq_len] matrix of tag indices for which we compute the log-likelihood. sequence_lengths: A [batch_size] vector of true sequence lengths. transition_params: A [num_tags, num_tags] transition matrix, if available. Returns: log_likelihood: A [batch_size] `Tensor` containing the log-likelihood of each example, given the sequence of tag indices. transition_params: A [num_tags, num_tags] transition matrix. This is either provided by the caller or created in this function. r,)rNZ transitions) rZconvert_to_tensorr r rkerasZ initializersZ GlorotUniformVariablerr+r.) rrrrnum_tagsZ initializerr"r1Zlog_likelihoodrrrcrf_log_likelihoods     r:)rrrrc Cstj|tjd}tj|tjd}t|d}t|d}t|d}t|dg}tt|||d}|tt||d7}|jtjkrt|tj}t||dg}tt ||||g} tj |t|d| jd} t | | d} | S)aaComputes the unary scores of tag sequences. Args: tag_indices: A [batch_size, max_seq_len] matrix of tag indices. sequence_lengths: A [batch_size] vector of true sequence lengths. inputs: A [batch_size, max_seq_len, num_tags] tensor of unary potentials. Returns: unary_scores: A [batch_size] vector of unary scores. )rrrr,r)maxlenr) rr rr r expand_dimsrrint64gather sequence_mask reduce_sum) rrrr Z max_seq_lenr9Zflattened_inputsoffsetsZflattened_tag_indicesr&masksrrrr$s"  r$)rrrrc Cstj|tjd}tj|tjd}t|d}t|dd}t|ddgd|g}t|ddgd|g}|||}t|dg}t||} tj|t|d| jd} t| ddgddg} t | | d} | S)aaComputes the binary scores of tag sequences. Args: tag_indices: A [batch_size, max_seq_len] matrix of tag indices. sequence_lengths: A [batch_size] vector of true sequence lengths. transition_params: A [num_tags, num_tags] matrix of binary potentials. Returns: binary_scores: A [batch_size] vector of binary scores. )rrrr)r;r) rr rr r3rr>r?rr@) rrrr9Znum_transitionsZstart_tag_indicesZend_tag_indicesZflattened_transition_indicesZflattened_transition_paramsr'rBZtruncated_masksrrrr%#s   r%)rstaterrrcstj|tjd}ttjd|jd|d}t|dddg}tdfdd}tt|||dddg}t t|d|gd}tj t t |d|gdd}t ||S)aComputes the alpha values in a linear-chain CRF. See http://www.cs.columbia.edu/~mcollins/fb.pdf for reference. Args: inputs: A [batch_size, num_tags] matrix of unary potentials. state: A [batch_size, num_tags] matrix containing the previous alpha values. transition_params: A [num_tags, num_tags] matrix of binary potentials. This matrix is expanded into a [1, num_tags, num_tags] in preparation for the broadcast summation occurring within the cell. sequence_lengths: A [batch_size] vector of true sequence lengths. Returns: new_alphas: A [batch_size, num_tags] matrix containing the new alpha values. )rrrr,cs*t|d}|}|t|dg}|S)Nr,r)rr<r-)_stateZ_inputstransition_scoresZ new_alphas)rrr_scan_fnhs zcrf_forward.._scan_fn)r)rr rmaximumconstantr transposer<scanrstackrr r)rrCrr last_indexrFZ all_alphasidxsr)rrr4Is  "r4)scorerrc Cst|}tj|tjd}|d|d<xXtd|jdD]D}t||dd|}||t|d||<t|d||<q8Wt|dg}x*t|ddD]}| ||dqW| t|d}||fS)aDecode the highest scoring sequence of tags outside of TensorFlow. This should only be used at test time. Args: score: A [seq_len, num_tags] matrix of unary potentials. transition_params: A [num_tags, num_tags] matrix of binary potentials. Returns: viterbi: A [seq_len] list of integers containing the highest scoring tag indices. viterbi_score: A float containing the score for the Viterbi sequence. )rrrrN) nprrrr r<maxargmaxreversedappendreverse) rNrZtrellis backpointerstvZviterbiZbpZ viterbi_scorerrrviterbi_decodevs  rXcs|eZdZdZeedfdd ZeddZeddZ fd d Z d d Z e d fdd Z ee ddddZZS)CrfDecodeForwardRnnCellz4Computes the forward decoding in a linear-chain CRF.)rc s,tjf|t|d|_|jd|_dS)a1Initialize the CrfDecodeForwardRnnCell. Args: transition_params: A [num_tags, num_tags] matrix of binary potentials. This matrix is expanded into a [1, num_tags, num_tags] in preparation for the broadcast summation occurring within the cell. rN)super__init__rr<_transition_paramsr _num_tags)selfrkwargs) __class__rrr[s z CrfDecodeForwardRnnCell.__init__cCs|jS)N)r])r^rrr state_sizesz"CrfDecodeForwardRnnCell.state_sizecCs|jS)N)r])r^rrr output_sizesz#CrfDecodeForwardRnnCell.output_sizecst|dS)N)rZbuild)r^Z input_shape)r`rrrcszCrfDecodeForwardRnnCell.buildcCsZt|dd}|t|j|j}|t|dg}t|d}tj|tjd}||fS)aBuild the CrfDecodeForwardRnnCell. Args: inputs: A [batch_size, num_tags] matrix of unary potentials. state: A [batch_size, num_tags] matrix containing the previous step's score values. Returns: backpointers: A [batch_size, num_tags] matrix of backpointers. new_state: A [batch_size, num_tags] matrix of new score values. rr,r)r)rr<r r\Z_compute_dtype reduce_maxrQr)r^rrCrEZ new_staterUrrrcalls  zCrfDecodeForwardRnnCell.call)rcsDdt|jdi}tt|}tt | t | S)Nrr) rr6r\numpytolistrZrY get_configdictlistitems)r^configZ base_config)r`rrrhsz"CrfDecodeForwardRnnCell.get_config)rlrcCs"tj|dtjd|d<|f|S)Nr)r)rOarrayfloat32)clsrlrrr from_configsz#CrfDecodeForwardRnnCell.from_config)__name__ __module__ __qualname____doc__rrr[propertyrarbrcrerirh classmethodrp __classcell__rr)r`rrYs   rYcCsZtj|tjd}t|t|d}t||jd}tjjj |dd|jd}||||dS)aComputes forward decoding in a linear-chain CRF. Args: inputs: A [batch_size, num_tags] matrix of unary potentials. state: A [batch_size, num_tags] matrix containing the previous step's score values. transition_params: A [num_tags, num_tags] matrix of binary potentials. sequence_lengths: A [batch_size] vector of true sequence lengths. Returns: backpointers: A [batch_size, num_tags] matrix of backpointers. new_state: A [batch_size, num_tags] matrix of new score values. )rrT)Zreturn_sequencesZ return_stater)mask) rr rr?r rYrr7layersZRNN)rrCrrrxZ crf_fwd_cellZ crf_fwd_layerrrrcrf_decode_forwards rz)rrCrcCs6t|dddg}dd}tt|||dddgS)aOComputes backward decoding in a linear-chain CRF. Args: inputs: A [batch_size, num_tags] matrix of backpointer of next step (in time order). state: A [batch_size, 1] matrix of tag index of next step. Returns: new_tags: A [batch_size, num_tags] tensor containing the new tag indices. rrr,cSsLtj|dgd}tjtt|d|gdd}tjt||dd}|S)Nr)rrr)rr6rKrr r<r)rCrrMZnew_tagsrrrrFs"z%crf_decode_backward.._scan_fn)rrIrJ)rrCrFrrrcrf_decode_backwards r{) potentialsrsequence_lengthrcstjdddkrtdtjtjdfdd}fdd }jd dk rrjd d krj|S|Sn tttd d ||SdS) a,Decode the highest scoring sequence of tags. Args: potentials: A [batch_size, max_seq_len, num_tags] tensor of unary potentials. transition_params: A [num_tags, num_tags] matrix of binary potentials. sequence_length: A [batch_size] vector of true sequence lengths. Returns: decode_tags: A [batch_size, max_seq_len] matrix, with dtype `tf.int32`. Contains the highest scoring tag indices. best_score: A [batch_size] vector, containing the score of `decode_tags`. Nz2.4zsCRF Decoding does not work with KerasTensors in TF2.4. The bug has since been fixed in tensorflow/tensorflow##45534)rcs<tjtjddtjd}tjtjdddgd}||fS)Nr,)r)rr)r )rr rQrrrd) decode_tags best_score)r|rrr#"sz"crf_decode.._single_seq_fncstdddgdddg}tj|dgd}tdddgdddg}ttjdtjdd}t|||\}}tj||dd}tjtj |ddtjd}tj |dd}t ||}tj|dgd}tj ||gdd}tj|dd}tj |dd}||fS)Nrrr)r)r)Zseq_axisr,)rr3r6rGrHrrzZreverse_sequencer rQr<r{rrd)Z initial_staterZsequence_length_less_onerUZ last_scorerr)r|r}rrrr('s"  z!crf_decode.._multi_seq_fnr) r __version__warningswarnr rr r)r*)r|rr}r#r(r)r|r}rr crf_decode s r)r|rrr}rcCst||}t|||S)aDecode the highest scoring sequence of tags under constraints. This is a function for tensor. Args: potentials: A [batch_size, max_seq_len, num_tags] tensor of unary potentials. tag_bitmap: A [batch_size, max_seq_len, num_tags] boolean tensor representing all active tags at each index for which to calculate the unnormalized score. transition_params: A [num_tags, num_tags] matrix of binary potentials. sequence_length: A [batch_size] vector of true sequence lengths. Returns: decode_tags: A [batch_size, max_seq_len] matrix, with dtype `tf.int32`. Contains the highest scoring tag indices. best_score: A [batch_size] vector, containing the score of `decode_tags`. )rr)r|rrr}Zfiltered_potentialsrrrcrf_constrained_decodeQs r)N)rrfrOZ tensorflowrZtensorflow_addons.utils.typesrZ typeguardrtypingrZTensorrr+r0r.r:r$r%r4rXr7ryZAbstractRNNCellrYrzr{rrrrrrsX   1-2+'%( AG