tjۇhUdZddlmZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZejeZerddlZdd lmZdd lmZmZmZmZmZmZiZd ed <iZ d ed<iZ!d ed<ej"d`dZ#ej"dadZ$ej"dbdZ%dbdZ&dbdZ'dcdZ(ddd!Z)ded#Z*dcd$Z+e,fZ-d%ed&<ere-ej.ej/ej0ej1fz Z-dfd)Z2dgdhd,Z3did.Z4djd/Z5dkd4Z6dld8Z7dmd:Z8d;Z9dnd?Z:dodCZ;iZe=dKdqdLZ?e=dMdNdqdOZ@e=dMdPdqdQZAdqdRZBej"drdTZCdsdWZDdXZEdYZFdtd[ZGdud]ZHdsd^ZIdsd_ZJdS)vuShared export utilities used by all exporter backends. Organised into five sections (search for the `# ── Name ──` banners): - **Patch and fix registries** — backend-keyed `_PATCHES` / `_FX_NODE_FIXES` / `_FX_PROGRAM_FIXES` populated via `@register_patch(backend, *paths)` / `@register_fx_node_fix` / `@register_fx_program_fix`, applied via `apply_patches` / `apply_fx_node_fixes` / `apply_fx_program_fixes`. - **Recursive structure traversal** — internal helpers (`_map_leaf_tensors`, `_iter_leaf_tensors`) that drive every other tensor utility. - **Public tensor utilities** — `get_leaf_tensors`, `duplicate_leaf_tensors`, `cast_leaf_tensors`, and `prepare_for_export` (sets attention/experts impl, patches non-exportable patterns, strips output flags). - **Export input preparers** — `@register_export_input_preparer(marker)` registry that precomputes the per-encoder kwargs (`cu_seqlens`, `position_ids`, audio chunks, …) the model would otherwise need data-dependent ops for. - **Decomposition** — `decompose_prefill_decode` (split a generative forward into prefill + decode) and `decompose_multimodal` + `is_multimodal` (split a multimodal forward into one entry per submodule), backed by `_capture_forward`. ) annotationsN)MutableMapping)Any)logging)is_torch_available)PreTrainedModel)'get_vision_bilinear_indices_and_weightsget_vision_cu_seqlensget_vision_merged_shapeget_vision_nearest_position_idsget_vision_position_idsget_vision_window_indexz*dict[str, list[tuple[Any, str, callable]]]_PATCHESzdict[str, list[callable]]_FX_NODE_FIXES_FX_PROGRAM_FIXESobjr attributestrfactoryc#Kt||}t|||| dVt|||dS#t|||wxYw)zNSwap `obj.` with `factory(original)` for the duration of the block.N)getattrsetattr)rrroriginals Z/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/exporters/utils.pypatch_attributerQsqsI&&H CGGH--...* Y)))))Y))))s AApatcheslist[tuple[Any, str, callable]]c #Ktj5}|D]*\}}}|t|||+dVddddS#1swxYwYdS)uInstall `(obj, attribute, factory)` patches for the duration of the block. Plural form of `patch_attribute` — each `factory(original)` returns the replacement callable. Originals are restored on exit, even if the body raises. N) contextlib ExitStack enter_contextr)rstackrrrs rpatch_attributesr$\s    5'. J J #CG   Y H H I I I I s2AAAbackendc#Ktt|g5dVddddS#1swxYwYdS)z:Install `_PATCHES[backend]` for the duration of the block.N)r$rget)r%s r apply_patchesr(is (,,w33 4 4 s=AAcfd}|S)zKAppend the decorated `(gm, node) -> bool` fix to `_FX_NODE_FIXES[backend]`.cdtg||SN)r setdefaultappendfnr%s r decoratorz'register_fx_node_fix..decoratorss,!!'2..55b999 r%r0s` rregister_fx_node_fixr4ps$ r1cfd}|S)uAppend the decorated `(exported_program) -> None` fix to `_FX_PROGRAM_FIXES[backend]`. Use this for fixes that need program-level context (range_constraints, graph_signature, state_dict) — the per-node `_FX_NODE_FIXES` shape only sees one node at a time. cdtg||Sr+)rr,r-r.s rr0z*register_fx_program_fix..decorators,$$Wb1188<<< r1r2r3s` rregister_fx_program_fixr7zs$ r1returnNonecXt|gD] }||dS)zDApply `_FX_PROGRAM_FIXES[backend]` to `exported_program` (in place).N)rr')r%exported_programfixs rapply_fx_program_fixesr=s? $$Wb11  r1pathscfd}|S)umAppend the decorated `factory(original)` to `_PATCHES[backend]`, once per `path`. Each `path` is a dotted Python path like `"torch.where"`, `"torch.Tensor.unsqueeze"`, or `"transformers.models.nllb_moe.modeling_nllb_moe.NllbMoeTop2Router._cast_classifier"`. The rightmost segment is the attribute to swap; the rest is the object that owns it. Paths are resolved at decoration time — submodules are imported as needed, falling back to `getattr` for class attributes. A path that fails to resolve (e.g. the backend isn't installed) is silently skipped so the module still imports. Passing multiple paths registers the SAME factory against each — useful for swapping the same method or torch op across several call sites (e.g. ``torch.unsqueeze`` + ``torch.Tensor.unsqueeze``, or one vision-attention forward across N model classes). cD]^}|d\}}}t|}|-tg|||f_|S)N.) rpartition_resolve_dotted_pathrr,r-)r/pathobj_path_rrr%r>s rr0z!register_patch..decoratorst J JD%)__S%9%9 "Ha&x00C{    , , 3 3S)R4H I I I I r1r2)r%r>r0s`` rregister_patchrGs* r1rDcHddl}|d} ||d}|ddD]I} ||jd|}##tt f$rt ||}YFwxYw|S#tt f$rYdSwxYw)uResolve a dotted Python path to the actual object — importing submodules where possible, falling back to `getattr` for class attributes (e.g. `torch.Tensor`). Returns `None` if the path can't be resolved (e.g. the backend isn't installed).rNrA) importlibsplit import_module__name__ ImportErrorAttributeErrorr)rDrJpartsrparts rrCrCs JJsOOE %%eAh//!""I ) )D )--.F.F.F.FGG0 ) ) )c4(( )  (tts5'B A#"B #!BB BB B! B!ct|g}|D]}t|tjjs"t|jj D]!}|j dkr|D]}|||rn" |j | #ttf$rYwxYwdS)uWalk every call_function node and apply the first matching `_FX_NODE_FIXES[backend]` fix, then DCE. Each fix has signature `(gm, node) -> bool`. Returning `True` means the fix consumed the node — no further fixes run against it. Fixes are expected to be disjoint by `node.target`; if multiple could apply, list order decides. After the walk, `Graph.eliminate_dead_code` runs on every sub-GraphModule and `gm.recompile()` is called once. PyTorch DCE occasionally raises `SystemError` / `KeyError` from `erase_node._update_args_kwargs` on orphaned symbolic-size nodes — we swallow both; any survivors are handled by the downstream backend optimizer. call_functionN)rr'modules isinstancetorchfx GraphModulelistgraphnodesopeliminate_dead_code recompile SystemErrorKeyError)r% graph_modulefixesgmnoder<s rapply_fx_node_fixesres   w + +E""$$  "eh233  ((  Dw/))  3r4==E  H ( ( * * * LLNNNNX&    D   s-B<<CCztuple[type, ...]_LEAF_SKIP_TYPESr/callablec <t|tr|St|tjr |St|tt t fr$t|fd|DSt|tr-t |D]}t||||<|St|drFt| D]$\}}t||t|%|S)uApply `fn` to every tensor in a nested structure, preserving container types. Mutates dicts and `__dict__`-bearing objects in place (preserving identity — callers rely on this so downstream pops/mutations propagate back to the original mapping); rebuilds lists/tuples/sets/frozensets (immutable or order-sensitive containers). Skips non-traversable leaf types (enum, SymInt, etc.). c38K|]}t|VdSr+_map_leaf_tensors).0itemr/s r z$_map_leaf_tensors..s.EE*444EEEEEEr1__dict__)rUrfrVTensorrYtuplesettypedictrkhasattrvarsitemsr)rr/kattrattr_vals ` rrkrks%#'(( #u|$$r#ww#eS)**FtCyyEEEEEEEEEE#tc 3 3A&s1vr22CFF sJ@"3iioo// @ @ND( C02>> ? ? ? ? Jr1prefixc#\Kt|trdSt|tjr |pd|fVdSt|tt t frEt|D]3\}}|r|d|nt|}t||Ed{V4dSt|tr=| D]&\}}|r|d|n|}t||Ed{V'dSt|dr%tt||Ed{VdSdS)zEYield `(dotted_path, tensor)` for every tensor in a nested structure.NoutputrAro)rUrfrVrprYrqrr enumerater_iter_leaf_tensorsrtrwrurv)rr|indexrmrDkeyvalues rrrs#'((#u|$$ 9 #%%%%%% C$s+ , , 9$S>> 6 6KE4*0@f&&u&&&c%jjD)$55 5 5 5 5 5 5 5 5 6 6 C  9))++ 7 7JC(.7f$$s$$$CD)%66 6 6 6 6 6 6 6 6 7 7 j ! !9%d3ii8888888888899r1dict[str, torch.Tensor]c:tt|S)aRecursively retrieve all leaf tensors from a potentially nested structure. Args: obj (`Any`): A tensor, dataclass, dict, list, tuple, or any nesting thereof. Returns: `dict[str, torch.Tensor]`: Flat mapping from dotted path strings to tensors. )rtr)rs rget_leaf_tensorsrs "3'' ( ((r1cLtdfd }t||S)aClone tensors that appear more than once in an output structure. When a model returns the same tensor under two output names (e.g. `last_hidden_state` and `hidden_states[0]`), the ONNX optimizer deduplicates the two output nodes and renames one, breaking the expected name mapping. Cloning duplicates gives each output leaf a distinct identity so the optimizer has nothing to merge. tensor torch.Tensorr8ct|vr|St||Sr+)idcloneadd)rseens r_dedupz&duplicate_leaf_tensors.._dedup+s? f::  <<>> ! F r1rrr8r)rrrk)rrrs @rduplicate_leaf_tensorsr!s> 55D S& ) ))r1dtype torch.dtypedevice torch.devicec4dfd }t||S)zJRecursively cast all floating-point tensors to the given dtype and device.rrr8c|r|n|S)Nrr)r)is_floating_pointto)rrrs r_castz cast_leaf_tensors.._cast7sB8>8P8P8R8RpvyyuVy444X^XaXaioXaXpXppr1rrj)rrrrs `` rcast_leaf_tensorsr4s@qqqqqqq S% ( ((r1model!PreTrainedModel | torch.nn.Moduletorch.device | Nonect|dr|jS t|jS#t$rYdSwxYw)a#`.device` for any `nn.Module`. `PreTrainedModel` exposes it directly via `ModuleUtilsMixin`; for plain submodules (e.g. a `Linear` or `MultiModalProjector` from a decomposed multimodal model) we fall back to the first parameter. Returns `None` if the module has no parameters at all.rN)rurnext parameters StopIterationrs r module_devicer=sbuh|E$$&&''.. tt%? A  A torch.dtype | Nonect|dr|jS t|jS#t$rYdSwxYw)zE`.dtype` for any `nn.Module`. Same fallback story as `module_device`.rN)rurrrrrs r module_dtyperIs`ug{E$$&&''-- ttr) use_cacheoutput_attentionsoutput_hidden_states return_dict return_lossinputsMutableMapping[str, Any]Rtuple[PreTrainedModel | torch.nn.Module, MutableMapping[str, Any], dict[str, Any]]c,dD]0}|d}|td|d|d1t|dr%t|jddrtd ddrtd fd t D}tj5t|dddn #1swxYwYt|}t|}||t|| ||fS) u6Configure model and inputs for export. Mutates both `model` and `inputs` in place, returning `(model, inputs, output_flags)` where `output_flags` holds the values popped from `inputs` for `use_cache`, `return_dict`, etc. (to be applied reversibly onto `model.config` by `patch_model_config` during the trace). - Strips label inputs (`labels`, `future_values`) — loss computation is unsupported. - Pops output flags (`use_cache`, `return_dict`, …) from `inputs` so they don't appear as traced kwargs; the values are returned for the trace block to apply onto `model.config`. - Pre-computes data-dependent vision/audio kwargs registered via `@register_export_input_preparer` and writes them into `inputs`. - Casts input tensors to match the model's `dtype` / `device`. )labels future_valuesNzFound 'zM' in inputs. Loss computation is not supported during export. Please remove 'z+' from your inputs before calling export().configrFzFound 'model.config.return_loss=True'. Loss computation is not supported during export. Please set 'model.config.return_loss=False' before calling export().zFound 'return_loss=True' in inputs. Loss computation is not supported during export. Please remove 'return_loss' from your inputs or set it to False.cDi|]}|v||Sr2)pop)rlflagrs r z&prepare_for_export..|s-WWWtPVD&**T**r1r) r ValueErrorrurrr' _OUTPUT_FLAGSrVno_gradprecompute_export_inputsrrr)rr label_keyr output_flagsrrs ` rprepare_for_exportrWs"1  9d++  Y)YY"+YYY   uh GEL-$O$O  S   zz-''  O   XWWW}WWWL 00 ///000000000000000   E 5 ! !F F."6vFFF &, &&s5CCCtorch.nn.Modulename Any | Nonecb|D]}t||dx}|cSdS)zTReturn the first non-None value of `name` found on `model` or any of its submodules.N)rTr)rrmodulers r_find_submodule_attrrsC--//VT400 0E =LLL > 4r1zdict[tuple[str, ...], callable]_EXPORT_INPUT_PREPARERSmarkerscfd}|S)u4Register `fn(model, inputs) -> None`. Dispatched when every `marker` is a key in `inputs` with a non-`None` value — no model_type list to maintain. Use multiple markers to narrow the match when a single kwarg is too ambiguous (e.g. `("input_features", "feature_lens")` for omni audio encoders).c|t<|Sr+)r)r/rs rr0z1register_export_input_preparer..decorators+-( r1r2)rr0s` rregister_export_input_preparerrs$  r1grid_thwdict[str, Any]c|d}t|d}||dd}t||d<t|ddu}t||||d <t|d }t|d }||t ||||\|d <|d <t|d}|t |||\|d<|d<dSdS)aPrecompute helpers driven by `grid_thw`: `cu_seqlens`, `position_ids`, plus optional `window_index`/`cu_window_seqlens` (XNet-style window attn) and `bilinear_indices`/`bilinear_weights` (interpolation-based merging). Optional helpers are gated by the presence of their config attribute on the encoder (`window_size`+`patch_size` for window attention, `num_grid_per_side` for bilinear), so a model that doesn't use that feature won't get its kwarg injected. rspatial_merge_sizeN merge_sizesrI cu_seqlensaxis_dim)include_temporal position_ids window_size patch_size window_indexcu_window_seqlensnum_grid_per_sidebilinear_indicesbilinear_weights)rr'r rrr )rrrrrrrrs r_prepare_grid_thw_vision_inputsrs#j!H-e5IJJ!$ZZ q990::F<,E:>>dJ4X?QdtuuuF>&um<U (+z? ? ;~': ;-U4GHH$Ah ');B B >!"F+=$>$>$>%$r1 target_sizesc@|d}t|d}|t|||d<t|d}|^tjj|dd}t |d|d d \|d <|d <t|||d <dSdS)a NaViT-style packed encoders carry per-image `(h, w)` as `target_sizes` instead of `grid_thw`. Synthesise `grid_thw = [1, h, w]` and run the nearest-position-id / window-index / merged-shape helpers so the per-image Python loops move outside the traced graph.rnum_patches_per_sideNrwindow_kernel_size)rIrrI)rr)rrrrr merged_shape)rr rVnn functionalpadrr )rrrrrrs r_prepare_navit_vision_inputsrs .)L/7MNN'!@Oc!d!d~-e5IJJ%8&**<q*II>U 8J18MZ[? ? ? ;~': ;"9GY!Z!Z~ &%r1input_features feature_lensc|d}|d}tjt|j}t |d}t |d}t |d}||||j\}} ||d<| |d<t |dr1|| ||j|j|d <|| |j|d <d S|| |d <|| |d <t |d ||d <d S)uReplace `input_features`/`feature_lens` with precomputed `padded_feature`, `chunk_lengths`, `cu_seqlens`, `valid_indices` (+ `pool_indices` on Qwen2.5-Omni-style encoders) so the encoder's `.split(.tolist(), dim=0)` and related data-dependent ops happen outside the traced graph. The helpers (`chunk_and_pad_features`, `get_audio_cu_seqlens`, …) all live in the model's own ``modeling_*.py`` module, so we resolve them via ``type(model).__module__`` rather than hard-coding one Omni variant. ``n_window_infer`` selects the Qwen3-Omni-style four-arg ``get_audio_cu_seqlens`` over the Qwen2.5-Omni-style single-arg form. rrchunk_and_pad_featuresget_audio_cu_seqlensget_valid_indicespadded_feature chunk_lengthsn_window_inferr valid_indicesget_pool_indices pool_indicesN)sysrTrs __module__rn_windowrur) rrrrrrrrrrs r_prepare_omni_audio_inputsrs8.)L,-N [e/ 0F$V-EFF"6+ABB(;<<$:$:>"R"R33MBB|"3"3M"B"B!D1C!D!D\!R!R~r1input_features_maskcddlm}t|d}t|d}||dS|d}|j\}}||dzz}|dt j} |||dd dt j} || | |||d <dS) u4Precompute `cu_seqlens` for Qwen3-ASR — the encoder's call to ``get_audio_cu_seqlens`` has a data-dependent Python loop that we evaluate here so the encoder pops the result from ``kwargs``. Mirrors the few lines that build ``feature_lens``/``chunk_lengths`` in ``Qwen3ASREncoder.forward``. r)rrrNr)dimr) #models.qwen3_asr.modeling_qwen3_asrrrshapesumrrVlongviewreshape) rrrrrr batch_sizepadded_feature_length num_chunksrrs r_prepare_qwen3_asr_audio_inputsrsKJJJJJ#E:66H)%1ABBN>1 !67(;(A%J%&8a<8J&**2..11%*==L',,ZRHHLLQSLTT\\]_``ccdidnooM// |^]effF<r1cdt|drd}d}|dup|dup|jd|jdk}|rNtt j|jj}fd|D}|jd i|\}}|d<t D],\} } tfd| Dr | |-dS) uInject precomputed tensors for data-dependent ops the model would otherwise hit during tracing. Two layers: - Outer LLM rope index (`get_rope_index`) — generic `hasattr` probe; covers Qwen-VL / GLM-4V etc. - Per-encoder preparer dispatched by marker kwargs present in `inputs` (e.g. `grid_thw`, `target_sizes`, `(input_features, feature_lens)`) — see `register_export_input_preparer`. A preparer fires only when every one of its markers is present in `inputs`. rNget_rope_index input_idsattention_maskrIc*i|]}|v||Sr2r2)rlrxrs rrz,precompute_export_inputs..,s$LLLAV 1fQi r1c3FK|]}|duVdSr+)r')rlmrs rrnz+precompute_export_inputs..3s2::Qvzz!}}D(::::::r1r2) r'rurrrinspect signaturerrrrwall) rrr attn_mask is_prefill rope_params rope_inputsrrFrpreparers ` rrrsBzz.!!)ge=M.N.N)JJ{++ JJ/00 $&g)t*;gyq?QU^UdefUg?g  2g/0DEEPQQKLLLLLLLK2e2AA[AAOL!%1F> "5::<<$$ ::::'::: : : $ HUF # # #$$r1rc#Kg|jtjtjfd}||_ V|_dS#|_wxYw)zCapture forward call kwargs into a list (one dict per call). Positional args are normalised to kwargs via `inspect.signature` so the captured dicts can be passed directly as `kwargs=inputs` to `torch.export`. ci} j|i|}|jD]\}} j|}|jt jjkr(|tj |T|jt jj krtj |||< ||i|Sr+) bind argumentsrwrkindr  Parameter VAR_KEYWORDupdatecopydeepcopyVAR_POSITIONALr-) argskwargscapturedboundrrparamcallsrsigs rwrapperz!_capture_forward..wrapperKs$)&)) ?0022 6 6KD%N4(EzW.::: e 4 45555w0???!%u!5!5 Xx((((r1N)forwardr r  functoolswraps)rr%r#rr$s @@@r_capture_forwardr)?sE~H  H % %C_X ) ) ) ) ) ) )FN" !!!!!s A A"r 'dict[str, tuple[torch.nn.Module, dict]]c R t|5}|jd itj|ddddddn #1swxYwYnZ#t$rM}t dt |jdt| d|d}~wwxYwt|dkr5t dt |jdt|d tj||d ftj||d fd S)uRun `model.generate()` for 2 tokens and capture prefill and decode inputs. Reuses the full generation machinery so every architecture (decoder-only, SSM, encoder-decoder, multi-modal, …) gets correct inputs without reimplementing the loop. Returns: `dict[str, tuple[torch.nn.Module, dict]]`: `{"prefill": (model, prefill_inputs), "decode": (model, decode_inputs)}` r)max_new_tokensmin_new_tokensNz$decompose_prefill_decode failed for . Inputs passed: z<. Make sure the inputs are compatible with model.generate().z6decompose_prefill_decode expected at least 2 calls to z;.forward() during generate(max_new_tokens=2), but captured z. This likely means generate() bypasses the top-level forward() (e.g. delegates to an inner model), so prefill/decode decomposition is not supported for this architecture.rrI)prefilldecoder2) r)generaterr Exception RuntimeErrorrsrMrYkeyslen)rrr#es rdecompose_prefill_decoder7_s e $ $ X EN W WT]622 W1UV W W W W W X X X X X X X X X X X X X X X  J4;;3G J J"6;;==11 J J J     5zzA~~ VT%[[EY V V?B5zz V V V   Ie$$eAh/9U##U1X.  s:A%A AAA A A B%AB  B%)multi_modal_projector connector embed_vision embed_audio)lm_headdict[str, torch.nn.Module]c>i}d}dD](}||}|||ur |||d<d})|}| ||ur||d<||jhD]<}ttzD]*}||vr$t ||dt ||||<+=|rd|vriS|S)uReturn `{attr_name: module}` for multi-modal submodules found on `model`. Uses the canonical `PreTrainedModel.get_encoder("image"/"audio")` and `get_decoder()` accessors for encoders and the language model. Projectors and `lm_head` are looked up by name on `model` and its `base_model` (e.g. `LlavaModel` under `LlavaForConditionalGeneration`). Only returns results when at least one modal encoder AND a language model are found — otherwise the model is not multi-modal and should be exported as a single unit. F)imageaudio)modalityN_encoderTlanguage_model) get_encoder get_decoder base_model_MULTIMODAL_PROJECTOR_NAMES_MULTIMODAL_LM_HEAD_NAMESr)rfound has_encoderrAencoderdecoderrootrs r_find_multimodal_submodulesrNs )+EK&##X#66  7%#7#7+2EX''' (K!!Gwe33")()22/2KK 2 2D5  WT4%>%>%J%dD11d  2 *%77 Lr1boolc:tt|S)zTReturns `True` if the model is multi-modal with modal encoders and a language model.)rOrNrs r is_multimodalrQs +E22 3 33r1c t|}|s%tdt|jd t j5t j5fd|D|d itj |dddn #1swxYwYdddn #1swxYwYnZ#t$rM}tdt|jdt|d|d}~wwxYwfd|DS) uCapture inputs to each multi-modal submodule via a single forward pass. Detects all known multi-modal submodules by attribute name (vision tower, projector, language model, lm_head, …) and captures their forward kwargs during one `model(**inputs)` call. Each submodule is returned as a separate `name: (module, inputs)` entry for independent export. The token-merge step (e.g. `masked_scatter` for multi-modal models) is intentionally left outside the exported graphs — it is the caller's responsibility to assemble `inputs_embeds` from the encoder outputs before running the decoder. Returns: `dict[str, tuple[torch.nn.Module, dict]]`: One `name: (module, inputs)` entry per detected submodule (image/audio encoder, projector, language model, lm_head). Raises: `ValueError`: if no known multi-modal submodules are found on the model. z8decompose_multimodal found no multi-modal submodules on zB. Expected an image/audio encoder + language model, found neither.c\i|](\}}|t|)Sr2)r"r))rlrrr#s rrz(decompose_multimodal..sC   HTfe))*:6*B*BCC   r1Nz decompose_multimodal failed for r.rAcHi|]\}}| |||dfS)rr2)rlrrsubmodule_inputss rrz(decompose_multimodal..sJ    D& D !  v'-b12   r1r2)rNrrsrMr r!rVrrwrrr2r3rYr4)rr submodulesr6r#rUs @@rdecompose_multimodalrWs&-U33J   PtE{{G[ P P P     ! # # +uemoo + +    XbXhXhXjXj     E * *DM&)) * * *  + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  ltE{{/C l lVZ[a[f[f[h[hViVi l l l        &,,..   s`C B?!;B( B?(B, ,B?/B, 0B?3 C ?CC CC D"ADD"ct||}|d\}}t|s|St||}|d|d<|S)uDecompose a generative model into independently exportable `(model, forward_inputs)` pairs. Runs `decompose_prefill_decode` to capture prefill and decode forward kwargs from a real `model.generate(**inputs, max_new_tokens=2)`. If the prefill is multi-modal (per `is_multimodal`), further splits it into one entry per submodule (vision/audio encoder, projector, language model, `lm_head`) via `decompose_multimodal`. Args: model: Generative model. Must support `model.generate(**inputs)`. inputs: **Generate** kwargs — what you'd pass to `model.generate(**inputs)`. Returns: `{component_name: (submodel, forward_inputs)}`. Keys are `"prefill"` / `"decode"` for plain generative models and `"_encoder"` / `"multi_modal_projector"` / `"language_model"` / `"lm_head"` / `"decode"` for multi-modal generative models. r/r0)r7rQrW)rrstages prefill_modelprefill_inputs componentss rdecompose_for_generationr]s[&&eV 4 4F$*9$5!M>  ' ' %m^DDJ!(+Jx r1)rrrrrr)rr)r%r)r%rr8r9)r%rr>r)rDr)rrr/rgr8r)r{)rrr|r)rrr8r)rrr8r)rrrrrrr8r)rrr8r)rrr8r)rrrrr8r)rrrrr8r)rr)rrrrr8r9)rr)rr rrr8r*)rr r8r=)rr r8rO)K__doc__ __future__rr renumr'r rcollections.abcrtypingrutilsrutils.import_utilsr get_loggerrMloggerrVmodeling_utilsr vision_utilsr r r r rrr__annotations__rrcontextmanagerrr$r(r4r7r=rGrCrersrfEnumSymIntSymFloatSymBoolrkrrrrrrrrrrrrrrrrr)r7rGrHrNrQrWr]r2r1rros *#"""""  ******333333  H % % LLL000000"8:9999,...../11111 ****            6&J'+W,,,,QEL%.%-PP099999, ) ) ) )****&))))    i 4'4'4'4'|<>====      ++! ! ! ,+! H //[[[0/[$  0.AASSSBAS>  02GHHgggIHg*$$$$H """">""""Nd(""""J4444 ))))Xr1