tjB <ddlZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z m Z e r,ddlZddlmZdZejrddlZd ZndZe jeZd Zejd Zd ed efdZdZ d"dedzdededzfdZd"dedzdededzfdZ dej!fdZ"dZ#ejdZ$dZ%dZ&dedzfdZ' d#dededefdZ(e d e d$dedzdedefd!Z)dS)%N)contextmanagerredirect_stdout)StringIO)logging)is_torch_availablerequires) save_fileFTctrtjsdStjdkS)z7Return True if rank=0 or we aren't running distributed.Tr)_torch_distributed_availabletorch distributedis_initializedget_rank`/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/model_debugging_utils.py _is_rank_zeror,s? (U->-M-M-O-Ot   % % ' '1 ,,rzobject at 0x[0-9A-Fa-f]+x_strreturnc8td|S)z Replace memory addresses in an object's repr with a stable placeholder so that beautiful JSON diffs won't be ruined by ephemeral addresses. zobject at 0xXXXXXXXX)MEMORY_ADDRESS_REGEXsub)rs r_sanitize_repr_for_diffr6s # #$:E B BBrcPtrdt|jSdS)z@Return a stable string representation for a DTensor-like object.zDTensor (rank0) -> zDTensor(non-rank0))rrepr _local_tensor)xs r _dtensor_reprr>s,=Es$ D)))) T!%(( T%%n55 , ^ +M>H[27<< M:::m65++--4466::<<=xHHH((( RHRRMRRRSSSek""ek""  C  {u}emU^DDD /UZZ\\0B0BCC.tEIIKK/@/@AA.tEIIKK/@/@AA.tEIIKK/@/@AA       Jrct|ttfrfdt|DSt|tr"fd|DSt |drt|jSt|tj rt|Stt|S)a Recursively build a JSON-serializable Python structure from `value`. Tensors and DTensors become either sanitized repr strings, or are saved to disk as SafeTensors files and their relative paths are recorded in the returned Python structure. Lists/tuples/dicts are recursed into. All memory addresses are replaced with a stable placeholder. Args: value: Any Python object, often including torch Tensors, lists, dicts, etc. debug_path (`str`, *optional*, defaults to `None`): Directory to dump debug JSON and SafeTensors files. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the asscoiated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. path_to_value (`str`, *optional*, defaults to `None`): The file name for the SafeTensors file holding the full tensor value if `use_repr=False`. Returns: A nested Python structure (list, dict, or sanitized string) that is safe to json.dump. c Fg|]\}}t|d|S_r r!r" _serialize_io).0ivr r"r!s r z!_serialize_io..sN   1 ! XXeUkUkhiUkUk l l l   rc Hi|]\}}|t|d|SrArD)rFkrHr r"r!s r z!_serialize_io..sP   1 }Q:[hXnXnklXnXnooo   rrrC) isinstancelisttuple enumeratedictitemshasattrr>rr Tensorrr)r(r r!r"s ```rrErEvs'(%$''       !%((    %            uo&& (  JYf    %&&w(:PXhuvvvv "4;; / //rr(cPtjddt5}t|5t ||}dddn #1swxYwYdddn #1swxYwYt |S)z Converts a tensor into a sanitized multi-line string representation. Args: value (`torch.Tensor`): The tensor to represent. Returns: `list[str]`: List of string lines representing the tensor. Tx)r$ linewidthN)r r-rrprintgetvaluer splitlines)r(bufraws rr.r.s  DC8888 sOC00 e llnn #3 ' ' 2 2 4 44s4A<$A% A<%A) )A<,A) -A<<BBc|dr0|dd|dD]}t|dSdS)Nchildrenoutputs)getpopprune_outputs_if_children)nodechilds rrbrbse xx - D!!!*% - -E %e , , , ,-- - -rz (.*)\.(\d+)$ct|dd}|r|dsdS|dt fd|dDS)z Checks whether a node represents a layer block with submodules. Args: node (`dict`): A node from the call tree. Returns: `bool`: Whether the node is a layer block. module_pathr^Fc3PK|] }dd|ddvV!dS).rfrgNr`)rFrdnumbers r z!is_layer_block..s>[[}6}}} - < <<[[[[[[r)LAYER_SUFFIX_REmatchr`groupany)rcrorls @ris_layer_blockrrs}  ! !$((=""="= > >E ,,u [[^^F [[[[$zJZ[[[ [ [[rc>|dsdSdt|dD}t|dkr8d|ddDfdt|dD|d<|dD]}t|dS) z Recursively removes intermediate layers from the tree to improve readability. Keeps at least the first and last layers if many consecutive layers are present. Args: node (`dict`): The root or subnode to prune recursively. r^Nc:g|]\}}t|||fSr)rr)rFrGrds rrIz-prune_intermediate_layers..s.ddd81en]bNcNcdQJdddrrhcg|]\}}|Srr)rFrGrBs rrIz-prune_intermediate_layers..s66641aQ666rrc"g|] \}}|v | Srr)rFrGrd to_removes rrIz-prune_intermediate_layers..s)dddhaQRZcQcQcEQcQcQcr)r`rPlenprune_intermediate_layers)rc layer_blocksrdrxs @rrzrzs 88J  ddyj9I/J/JdddL <166<"#5666 dddd)D.strip_values..cleans#t$$ &&&AE!HHHHC&&   DE$KKKK    rinputsr_r^rk)rcrdr strip_valuess @rrz+log_model_debug_trace..strip_valuess      dhhx$$%%% dhhy"%%&&&XXj"--  E L      r)r0makedirsr1r2_debugger_module_dump_name Exceptionr6loggerinforb _call_treeopenjsondumploadsdumps) r modelbasee full_path summary_pathf tree_copyrs @rlog_model_debug_tracers$@ X K T 2 2 2 27<< E,L},\]]DD X X XO*OOOPPVW W X/-? KK5$555666++I/)Le./// i  1 %"Aa0000111111111111111      4:e&67788IL lC *A )Qq))))******************s;>A A'A""A'C))C-0C-?E$$E(+E(rjdo_prune_layersc jj ddgd_g_ _r> t jdn&#t$r}tdd|d}~wwxYwfd} D]\}}|dkr || d|j tj   fd }|_ dS) a Attaches a debugging wrapper to every module in the model. This records structured inputs and outputs during the forward pass into a call tree. Args: model (`PreTrainedModel`, `nn.Module`): Model to wrap. debug_path (`str`): Optional directory to dump debug JSON files. do_prune_layers (`bool`, *optional*, defaults to `True`): Whether to prune intermediate layers. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the associated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. Nrfrr_r^Tr}rrjcrjtjfd}|_dS)NctrH||dfdDt ddgd}j|t j5 |i|}dddn #1swxYwYtrt d Ddkrd|d<nt| d |d<j}|d s|d jr&jd d ||S) NargskwargscTi|]$}t|dk||%S)r)ry)rFrK dict_inputss rrLzY_attach_debugger_logic..wrap_forward..wrapped_forward..5s7aaaQ[YZ^I\I\_`I`I`q+a.I`I`I`r_inputsrCrc3K|]}dVdS)rNr)rFrBs rrmzX_attach_debugger_logic..wrap_forward..wrapped_forward..Fs"::Qq::::::rrr__outputsr^rv) rrE_debugger_model_call_stackappendr no_gradsumnamed_childrenra) inpskwsrcr=finishedrr rrmodule orig_forwardr!s @rwrapped_forwardzE_attach_debugger_logic..wrap_forward..wrapped_forward1s >'+s;; aaaa+aaa #,+##-!))2&;&;&;  $ "  077=== 1 1"lD0C00 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 V::&"7"7"9"9:::::Q>>&*DOO&3#-!))2&<&<&< '''DO!;??AA +-LL,,,3V4R8DKKHUUUJs, BBB)forward functoolswraps)rrrrr rr!s`` @r wrap_forwardz,_attach_debugger_logic..wrap_forward.sb~  & &% % % % % % % % %  ' &% N)rrgcJtr;dt||d ddgd}j| |i|}tr‰jrt| d|d<j}|djd<|djd<|d jd <fd t jDrtjt |S) Nz (top-level)rrrCrrr_rr^c`g|]*}j|j|d+S)N)rra)rFrKrs rrIzG_attach_debugger_logic..top_wrapped_forward..s; m m mqY^YijkYl mU  ! !!T * * m m mr)r r) rrErrrarrNkeysrzr) rrtop_noder=r class_namer rrreal_top_forwardr!s rtop_wrapped_forwardz3_attach_debugger_logic..top_wrapped_forwardds ?? >",:::'!S11)%%/"8"8"8     H  , 3 3H = = =,,, ?? Fu? F"/%!!+555 ###HY  7;;==H)1();E X &*29*=E Y '+3J+?E Z ( m m m mD9I9N9N9P9P4Q4Q m m m m <)%*:;;; !Zu E E E E r) __class____name__rrrr0rrr6 named_modulesrrr) rr rr!rrname submodulerrrs ```` @@r_attach_debugger_logicrsv&)J(2Td`bccE')E$'1E$X X K T 2 2 2 2 2 X X XO*OOOPPVW W X+)+)+)+)+)+)+)\!..0088i 2::  Y: 6 6 6 67777}_%&&#########'&#J(EMMMsA A*A%%A*)r )backendsc#Kd|D}|j||<t|||| |V|D] \}}||_ dS#|D] \}}||_ wxYw)a # Model addition debugger - context manager for model adders This context manager is a power user tool intended for model adders. It tracks all forward calls within a model forward and logs a slice of each input and output on a nested JSON file. If `use_repr=True` (the default), the JSON file will record a `repr()`-ized version of the tensors as a list of strings. If `use_repr=False`, the full tensors will be stored in separate SafeTensors files and the JSON file will provide a relative path to that file. To note, this context manager enforces `torch.no_grad()`. ## Usage add the context manager to a model to debug ```python import torch from PIL import Image from transformers import LlavaProcessor, LlavaForConditionalGeneration, model_addition_debugger_context torch.random.manual_seed(673) # load pretrained model and processor model_id = "llava-hf/llava-1.5-7b-hf" processor = LlavaProcessor.from_pretrained(model_id) model = LlavaForConditionalGeneration.from_pretrained(model_id) # create random image input random_image = Image.fromarray(torch.randint(0, 256, (224, 224, 3), dtype=torch.uint8).numpy()) # prompt prompt = "Describe this image." # process inputs inputs = processor(text=prompt, images=random_image, return_tensors="pt") # call forward method (not .generate!) with model_addition_debugger_context(model, debug_path="Your_debug_path", do_prune_layers=False): output = model.forward(**inputs) ``` c$i|] \}}||jSr)r)rFrBms rrLz3model_addition_debugger_context..s DDDdaQ DDDrN)rrrrR)rr rr! orig_forwardsmodule_instanceforward_methods rmodel_addition_debugger_contextrsfEDe.A.A.C.CDDDM =M%5*oxHHH5 /rErTr.rbrnrrrzrrrrrrrs   66666666<<<<<<<< )LLL++++++#(  %%'','''''+$#(   H % %---"rz"=>>C3C3CCCC   ^b..Tz.48.PSVZPZ....b(0(0S4Z(0$(0^adh^h(0(0(0(0V55555"---"*_--\\\")))((*cDj(*(*(*(*Z  |(|(|(|( |(|(|(|(~ :"  8585d 8585 858585858585r