B !`a@stdZddlmZddlmZddlmZddlZddlmZdZdZ d Z d d Z d d Z ddZ GdddeZdS)zCA wrapper around DebugDataReader used for retrieving tfdbg v2 data.)absolute_import)division)print_functionN)errorsZ__default_debugger_run__ics6tfdd}tj|dd}||fS)aRun a target task repeatedly in the background. In the context of this module, `target` is the `update()` method of the underlying reader for tfdbg2-format data. This method is mocked by unit tests for deterministic behaviors during testing. Args: target: The target task to run in the background, a callable with no args. interval_sec: Time interval between repeats, in seconds. Returns: - A `threading.Event` object that can be used to interrupt an ongoing waiting interval between successive runs of `target`. To interrupt the interval, call the `set()` method of the object. - The `threading.Thread` object on which `target` is run repeatedly. cs"xqWdS)N)waitclear)event interval_sectargetr e/home/dcms/DCMS/lib/python3.7/site-packages/tensorboard/plugins/debugger_v2/debug_data_multiplexer.py_run_repeatedly?s z5run_repeatedly_in_background.._run_repeatedlyT)r daemon) threadingEventThreadstart)r r rthreadr )r r r r run_repeatedly_in_background+s rc CsTddlm}t||jr@d|j|j|j|j|j|j |j |j d St dt |dS)Nr)debug_events_monitors InfNanAlert) alert_typeop_type output_slotsize num_neg_inf num_pos_infnum_nanexecution_indexgraph_execution_trace_indexzUnrecognized alert subtype: %s)tensorflow.python.debug.libr isinstancerrrrrrrrr TypeErrortype)alertrr r r _alert_to_jsonKs   r&cCs0d}d|kr$|d\}}t|}n|}||fS)z@Helper function that extracts op name and slot from tensor name.r:)splitint)Z tensor_namerop_namer r r parse_tensor_namebs  r+c@seZdZdZddZddZddZdd Zd d Zd d Z ddZ d)ddZ ddZ ddZ d*ddZd+ddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(ZdS),DebuggerV2EventMultiplexeraA class used for accessing tfdbg v2 DebugEvent data on local filesystem. This class is a short-term hack, mirroring the EventMultiplexer for the main TensorBoard plugins (e.g., scalar, histogram and graphs.) As such, it only implements the methods relevant to the Debugger V2 pluggin. TODO(cais): Integrate it with EventMultiplexer and use the integrated class from MultiplexerDataProvider for a single path of accessing debugger and non-debugger data. cCs(||_d|_t|_d|_|dS)zConstructor for the `DebugEventMultiplexer`. Args: logdir: Path to the directory to load the tfdbg v2 data from. N)_logdir_readerrLock _reader_lock_reload_needed_event_tryCreateReader)selfZlogdirr r r __init__ys  z#DebuggerV2EventMultiplexer.__init__c Cs|jr dS|j|jsyddlm}ddlm}Wntk rHdSXy||j|_Wn&tk rpdSt k rdSX|j |jt dg|_ t |jjt\|_}WdQRXdS)a*Try creating reader for tfdbg2 data in the logdir. If the reader has already been created, a new one will not be created and this function is a no-op. If a reader has not been created, create it and start periodic calls to `update()` on a separate thread. Nr)debug_events_reader)r)limit)r.r0r!r5r ImportErrorZDebugDataReaderr-AttributeError ValueError InfNanMonitorDEFAULT_PER_TYPE_ALERT_LIMIT _monitorsrupdateDEFAULT_RELOAD_INTERVAL_SECr1)r3r5r_r r r r2s(  z+DebuggerV2EventMultiplexer._tryCreateReadercCs|jr|jdS)a(If a reader exists and has started period updating, unblock the update. The updates are performed periodically with a sleep interval between successive calls to the reader's update() method. Calling this method interrupts the sleep immediately if one is ongoing. N)r1set)r3r r r _reloadReadersz(DebuggerV2EventMultiplexer._reloadReadercCs4|jdkrtd|tkr*tdt|f|jS)aWReturn the timestamp of the first DebugEvent of the given run. This may perform I/O if no events have been loaded yet for the run. Args: run: A string name of the run for which the timestamp is retrieved. This currently must be hardcoded as `DEFAULT_DEBUGGER_RUN_NAME`, as each logdir contains at most one DebugEvent file set (i.e., a run of a tfdbg2-instrumented TensorFlow program.) Returns: The wall_time of the first event of the run, which will be in seconds since the epoch as a `float`. NzNo tfdbg2 runs exists.z&Expected run name to be %s, but got %s)r.r9DEFAULT_DEBUGGER_RUN_NAMEZstarting_wall_time)r3runr r r FirstEventTimestamps  z.DebuggerV2EventMultiplexer.FirstEventTimestampcCs tddS)NzLDebugDataMultiplexer.PluginRunToTagToContent() has not been implemented yet.)NotImplementedError)r3Z plugin_namer r r PluginRunToTagToContentsz2DebuggerV2EventMultiplexer.PluginRunToTagToContentcCs*||jr"|tdgiiSiSdS)aReturn all the tfdbg2 run names in the logdir watched by this instance. The `Run()` method of this class is specialized for the tfdbg2-format DebugEvent files. As a side effect, this method unblocks the underlying reader's period reloading if a reader exists. This lets the reader update at a higher frequency than the default one with 30-second sleeping period between reloading when data is being queried actively from this instance. Note that this `Runs()` method is used by all other public data-access methods of this class (e.g., `ExecutionData()`, `GraphExecutionData()`). Hence calls to those methods will lead to accelerated data reloading of the reader. Returns: If tfdbg2-format data exists in the `logdir` of this object, returns: ``` {runName: { "debugger-v2": [tag1, tag2, tag3] } } ``` where `runName` is the hard-coded string `DEFAULT_DEBUGGER_RUN_NAME` string. This is related to the fact that tfdbg2 currently contains at most one DebugEvent file set per directory. If no tfdbg2-format data exists in the `logdir`, an empty `dict`. z debugger-v2N)r2r.rArB)r3r r r Runss  zDebuggerV2EventMultiplexer.RunscCsb|dkrtd|||kr0td||f|dkrR||krRtd||f|dkr^|}|S)NrzInvalid begin index (%d)z!end index (%d) out of bounds (%d)z9end index (%d) is unexpectedly less than begin index (%d))rInvalidArgumentError)r3beginendZ total_countr r r _checkBeginEndIndices s  z0DebuggerV2EventMultiplexer._checkBeginEndIndicesNc Csddlm}|}||kr dSg}t}t} xP|jD]F} | } | sJq8|| t| |jrfd} nd} t | || <| | | <q8Wt |} |dk r||krt d|| |}| ||t |}|||| |t dd|||Dd S) aGet alerts from the debugged TensorFlow program. Args: run: The tfdbg2 run to get Alerts from. begin: Beginning alert index. end: Ending alert index. alert_type_filter: Optional filter string for alert type, used to restrict retrieved alerts data to a single type. If used, `begin` and `end` refer to the beginning and ending indices within the filtered alert type. r)rNrZ__MiscellaneousAlert__z8Filtering of alerts failed: alert type %s does not existcSsg|] }t|qSr )r&).0r%r r r Qsz5DebuggerV2EventMultiplexer.Alerts..)rIrJr num_alertsalerts_breakdownZper_type_alert_limitalerts)r!rrGdictr<rPextendr"r:lenrrHrKr;)r3rCrIrJZalert_type_filterrrunsrPrOZalerts_by_typeZmonitorZmonitor_alertsrrNr r r Alertss@      z!DebuggerV2EventMultiplexer.AlertscCsX|}||krdS|jjdd}|||t|}||t|dd|||DdS)a>Get ExecutionDigests. Args: run: The tfdbg2 run to get `ExecutionDigest`s from. begin: Beginning execution index. end: Ending execution index. Returns: A JSON-serializable object containing the `ExecutionDigest`s and related meta-information NT)digestcSsg|] }|qSr )to_json)rLrVr r r rMlsz?DebuggerV2EventMultiplexer.ExecutionDigests..)rIrJ num_digestsexecution_digests)rGr. executionsrKrS)r3rCrIrJrTrYr r r ExecutionDigestsTs z+DebuggerV2EventMultiplexer.ExecutionDigestscCsh|}||krdS|jjdd}|||t|}|||}|jjd||d}||dd|DdS) a`Get Execution data objects (Detailed, non-digest form). Args: run: The tfdbg2 run to get `ExecutionDigest`s from. begin: Beginning execution index. end: Ending execution index. Returns: A JSON-serializable object containing the `ExecutionDigest`s and related meta-information NT)rVF)rVrIrJcSsg|] }|qSr )rW)rLZ executionr r r rMsz.)rIrJrZ)rGr.rZrKrS)r3rCrIrJrTrYrZr r r ExecutionDataps  z(DebuggerV2EventMultiplexer.ExecutionDatacCsh|}||krdS|dk r$td|jjdd}|||t|}||t|dd|||DdS)a`Get `GraphExecutionTraceDigest`s. Args: run: The tfdbg2 run to get `GraphExecutionTraceDigest`s from. begin: Beginning graph-execution index. end: Ending graph-execution index. Returns: A JSON-serializable object containing the `ExecutionDigest`s and related meta-information NzFtrace_id support for GraphExecutionTraceDigest is not implemented yet.T)rVcSsg|] }|qSr )rW)rLrVr r r rMszDDebuggerV2EventMultiplexer.GraphExecutionDigests..)rIrJrXZgraph_execution_digests)rGrEr.graph_execution_tracesrKrS)r3rCrIrJtrace_idrTZgraph_exec_digestsr r r GraphExecutionDigestss z0DebuggerV2EventMultiplexer.GraphExecutionDigestscCsl|}||krdS|dk r$td|jjdd}|||t|}|jjd||d}||dd|Dd S) aTGet `GraphExecutionTrace`s. Args: run: The tfdbg2 run to get `GraphExecutionTrace`s from. begin: Beginning graph-execution index. end: Ending graph-execution index. Returns: A JSON-serializable object containing the `ExecutionDigest`s and related meta-information NzDtrace_id support for GraphExecutionTraceData is not implemented yet.T)rVF)rVrIrJcSsg|] }|qSr )rW)rLZ graph_execr r r rMszADebuggerV2EventMultiplexer.GraphExecutionData..)rIrJgraph_executions)rGrEr.r]rKrS)r3rCrIrJr^rTdigestsr`r r r GraphExecutionDatas  z-DebuggerV2EventMultiplexer.GraphExecutionDatacCsP|}||krdSy|j|}Wn"tk rFtd|YnX|S)aGet the information regarding a TensorFlow graph. Args: run: Name of the run. graph_id: Debugger-generated ID of the graph in question. This information is available in the return values of `GraphOpInfo`, `GraphExecution`, etc. Returns: A JSON-serializable object containing the information regarding the TensorFlow graph. Raises: NotFoundError if the graph_id is not known to the debugger. NzThere is no graph with ID "%s")rGr. graph_by_idKeyErrorr NotFoundErrorrW)r3rCgraph_idrTgraphr r r GraphInfosz$DebuggerV2EventMultiplexer.GraphInfoc CsB|}||krdSy|j|}Wn"tk rFtd|YnXy||}Wn&tk r|td||fYnX|||}xP|dD]D}y||d} Wntk rd} YnX| r|| ||d<qWx`|dD]T} xN| D]F} y|| d} Wntk r d} YnX| r|| || d<qWqW|S)abGet the information regarding a graph op's creation. Args: run: Name of the run. graph_id: Debugger-generated ID of the graph that contains the op in question. This ID is available from other methods of this class, e.g., the return value of `GraphExecutionDigests()`. op_name: Name of the op. Returns: A JSON-serializable object containing the information regarding the op's creation and its immediate inputs and consumers. Raises: NotFoundError if the graph_id or op_name does not exist. NzThere is no graph with ID "%s"z/There is no op named "%s" in graph with ID "%s"inputsr*data consumers)rGr.rcrdrreZget_op_creation_digest_opCreationDigestToDataObject) r3rCrfr*rTrgop_creation_digestZ data_objectZ input_specZinput_op_digestZslot_consumer_specsZ consumer_specrVr r r GraphOpInfosH   z&DebuggerV2EventMultiplexer.GraphOpInfoc Cs|dkr dS|}|d=||j|d<|j|d<|d=g|d<x2|jpLgD]$}t|\}}|d||dqNWg|d<x"t|dD]}|dgqWx4||dD]"\}} } |d|| | d qW|S) Nrf graph_ids num_outputs input_namesri)r*rrkr*)r*Z input_slot) rW_getGraphStackIdsrfrprqr+appendrangeZget_op_consumers) r3rmrgZ json_objectZinput_tensor_nameZ input_op_namerr?Zsrc_slotZconsumer_op_nameZdst_slotr r r rl's*    z8DebuggerV2EventMultiplexer._opCreationDigestToDataObjectcCs>|g}|j|}x&|jr8|d|j|j|j}qW|S)aLRetrieve the IDs of all outer graphs of a graph. Args: graph_id: Id of the graph being queried with respect to its outer graphs context. Returns: A list of graph_ids, ordered from outermost to innermost, including the input `graph_id` argument as the last item. r)r.rcZouter_graph_idinsert)r3rfrorgr r r rrDs  z,DebuggerV2EventMultiplexer._getGraphStackIdscCs|}||krdS|jS)N)rGr.source_file_list)r3rCrTr r r SourceFileListVsz)DebuggerV2EventMultiplexer.SourceFileListcCsd|}||krdSy|j|\}}Wn"tk rLtd|YnX|||j||dS)Nz(There is no source-code file at index %d) host_name file_pathlines)rGr.rv IndexErrorrreZ source_lines)r3rCindexrTrxryr r r SourceLines\sz&DebuggerV2EventMultiplexer.SourceLinescCsZ|}||krdSg}x8|D]0}||jjkrrr&r+objectr,r r r r s