tj#dZddlZddlZddlmZdgZejeZde fdZ edZ e j dZ e jd ZdS) aQCompile-safe backward gradient logging for multiple tensors. ``debug_grad_log`` logs gradient norms during backward for one or more tensors. It is a leaf function with a ``register_multi_grad_hook`` that fires exactly once when all requires_grad tensor inputs have their gradients computed. Example:: import torch from torch.utils.debug_log import debug_grad_log x = torch.randn(4, requires_grad=True) y = torch.randn(4, requires_grad=True) z = x * 2 + y * 3 debug_grad_log(x, y) z.sum().backward() # Logs: [rank 0][bwd] t0_grad_norm=... t1_grad_norm=... N) leaf_functiondebug_grad_logreturnctjsdSddlm}|r|ndS)Nr)torch distributed is_availabletorch.distributedis_initializedget_rank)dists S/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/utils/debug_log.py _get_rankr!sS   ) ) + +q$$$$$$"1133 :4==???:cdS)aLog gradient norms of multiple tensors during backward. This is a no-op in the forward pass. During backward, the hook fires exactly once when all requires_grad tensor inputs have their gradients computed, and logs ``[rank R][bwd] t0_grad_norm=... t1_grad_norm=...``. Args: *tensors: One or more tensors to monitor. Returns: None. Call without assignment: ``debug_grad_log(x, y)``. Ntensorss rrr)s  4rcdS)Nrrs r_debug_grad_log_faker:s 4rcddt|D}tdt |dS)N c3vK|]4\}}d|d|dV5dS)tz _grad_norm=z.4fN)normitem).0igs r z'_debug_grad_log_hook..As[48Aq/A//!&&((--/////rz[rank %d][bwd] %s)join enumerateloginfor)gradsnormss r_debug_grad_log_hookr'?s\ HHr1s* 222222  g!!;3;;;;    (66)(666r