tj^"UdZddlmZddlZddlZddlmZmZmZm Z ddl Z ddl m Z m Z erddlmZmZddlmZgdZd ZdZdad ed <ejZd*dZGdde Zd+dZGddeZGddZGddZGddZ Gdd Z!ee ze"zdzZ#d!ed"<e#ezZ$d!ed#<e$eze ze!zZ%d!ed$<e%e&e%ze'e(e%fzZ)d!ed%<Gd&d'Z*Gd(d)Z+dS),zxSpec types for controlling what is dynamic in compiled/exported code. Currently only supports unbacked dynamic shapes. ) annotationsN)Anycast TYPE_CHECKING TypeAlias)SymBoolSymInt)IteratorSequenceShapeEnv) ShapeVarIntVarSTATIC TensorSpec ObjectSpecDictSpecSeqSpec ParamsSpec ShapesSpecLeafSpec LeafIntSpecIntermediateSpecz zShapeEnv | None_SPEC_SHAPE_ENVreturnr cttSt5ttcdddSddlm}Gdd|}|atcdddS#1swxYwYdS)z7Lazily build the singleton spec ShapeEnv (thread-safe).Nrr cVeZdZUdZehdZded<d fd Zd fd ZxZ S)*_get_spec_shape_env.._SpecShapeEnvuSpecial ShapeEnv used only at spec-definition time. Attribute access is blocked by default via ``__getattribute__``; only private (``_``-prefix) names and the explicit allowlist in ``_ALLOWED_PUBLIC`` pass through. SymInt arithmetic paths read a handful of public fields transparently (via the ``@record_shapeenv_event`` decorator and FX-cache machinery), so those are allowlisted. Everything else — evaluation, guard recording, deferred asserts, etc. — is blocked. During ``ShapeEnv.__init__`` itself, access is unrestricted because the base class touches its own public fields while bootstrapping. > bound_sympy is_recording var_to_range fx_node_cacheshould_record_eventszfrozenset[str]_ALLOWED_PUBLICrNonect|ddtt|dddS)N _init_doneFT)object __setattr__super__init__)self __class__s `/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/fx/experimental/dynamic_spec.pyr+z3_get_spec_shape_env.._SpecShapeEnv.__init__msO""4u===  """""4t<<<<<namestrrc|dr!t|Stds!t|S|t|jvr!t|St d|d)N_r'z_SpecShapeEnv: 'zd' is not allowed at spec-definition time. Use IntVar / ShapeVar only inside TensorSpec / ShapesSpec.) startswithr*__getattribute__typer$ TypeError)r,r0r-s r.r5z;_get_spec_shape_env.._SpecShapeEnv.__getattribute__ss??3'': 7733D999ww// ==: 7733D9994::555 7733D9997t777r/)rr%)r0r1rr) __name__ __module__ __qualname____doc__ frozensetr$__annotations__r+r5 __classcell__r-s@r. _SpecShapeEnvrMs  "/8i    / /O     = = = = = =          r/r@)r_SPEC_SHAPE_ENV_LOCK%torch.fx.experimental.symbolic_shapesr )r r@s r._get_spec_shape_envrCCs" 99  &"99999999 CBBBBB1 1 1 1 1 H1 1 1 f(-//s999999999999999999sA"$A""A&)A&cheZdZdZejZ ddddddfd ZddZddZ ddZ xZ S)raIndicates that a scalar integer argument is dynamic (no implicit range). Unbacked dynamic shapes will represent the underlying value in the compiled graph. IntVar is a ``SymInt`` subclass backed by ``_SpecShapeEnv``, so arithmetic and comparisons compose naturally:: A = IntVar("a") B = IntVar("b") A + 1 # SymInt expr=a#0 + 1 A * B + 1 # SymInt expr=a#0*b#1 + 1 A > 0 # SymBool Such derived ``SymInt`` values may be used directly as leaf specs in ``TensorSpec`` / ``ParamsSpec`` Repr always includes a per-instance uid so two IntVars with the same name (or two anonymous ones) are distinguishable in logs:: IntVar() -> "IntVar(anon#0)" IntVar("offset") -> "IntVar(offset#1)" Example:: IntVar("num_heads") IntVar("offset", min=-100, max=100) IntVar("size", min=1, max=2048, optimization_hint=512) Nminmaxoptimization_hintr0 str | NonerF int | NonerGrHrr%cddlm}m}ddlm}ddlm}||nd|_ttj |_ ||_ ||_ ||_tj|jd|j |_t%} ||j| t&|} t)| |||n| ||n|| j|j<dS)Nr)_NO_HINTSymNode)int_oo) ValueRangesanon#) shape_envpytypehint)torch.fx.experimental.sym_noderLrMtorch.utils._sympy.numbersrNtorch.utils._sympy.value_rangesrOr0nextr _uid_counter_uidrFrGrHsympySymbol sympy_symrCintr*r+r!) r,r0rFrGrHrLrMrNrOenvnoder-s r.r+zIntVar.__init__s! EDDDDDDD555555?????? ,DD& ,-- !2&@&@TY&@&@AA!##w N     ,7;?CC?CC, , (((r/r1cX|jd|jg}|j|d|j|j|d|j|j|d|jt |jdd|dS)NrQzmin=zmax=zoptimization_hint=(z, )) r0rZrFappendrGrHr6r8join)r,partss r.__repr__zIntVar.__repr__sI++ ++, 8  LL*** + + + 8  LL*** + + +  ! - LLFd.DFF G G Gt**%;; %(8(8;;;;r/r^c t|SN)idr,s r.__hash__zIntVar.__hash__s $xxr/dict[str, Any]c^t|j|j|j|j|jdS)N)r6r0rFrGrH)r6r8r0rFrGrHrks r. to_jsonablezIntVar.to_jsonables2JJ'I88!%!7    r/ri) r0rIrFrJrGrJrHrJrr%rr1rr^rrm) r8r9r:r; itertoolscountrYr+rgrlror>r?s@r.rrs<#9?$$L  (,         B < < < <        r/rvrwherer1r%ct|trd}nt|trd}ndS|jjt urt |d|d|ddS)zXIf ``v`` is a SymInt or SymBool, validate it originates from the spec ShapeEnv. r rN: z= spec values must originate from spec IntVar / ShapeVar; got z backed by a different ShapeEnv.) isinstancer rr`rRrCr7)rurvkinds r._validate_spec_symr{s!V Aw  v24444 C C C C C C C   54r/c2eZdZdZ ddddddfdZxZS)rzIndicates that a dimension size is dynamic and is >= 0. Subclass of `IntVar` with ``min=0`` by default. Example:: ShapeVar("batch") ShapeVar("batch", max=64) ShapeVar("batch", max=64, optimization_hint=32) NrrEr0rIrFr^rGrJrHrr%c|dkrtd|dt||||dS)NrzBShapeVar requires min >= 0 (a shape dim is non-negative); got min=z+. Use IntVar(...) for unrestricted scalars.rE) ValueErrorr*r+)r,r0rFrGrHr-s r.r+zShapeVar.__init__se 77LLLL  3CCTUUUUUr/ri) r0rIrFr^rGrJrHrJrr%)r8r9r:r;r+r>r?s@r.rrss    V(, V V V V V V V V V V V Vr/rcBeZdZdZddZdd Zdd Zdd ZddZddZ dS)ra~Per-dimension shape specification for a tensor. A list-like container of ``LeafIntSpec`` with length equal to the tensor's dim. Example:: B = ShapeVar("batch") TensorSpec([B, None]) # rank 2, dim 0 dynamic TensorSpec([B, 10]) # rank 2, dim 0 dynamic, dim 1 static=10 TensorSpec([B * 2 + 1, None]) # rank 2, dim 0 derived from B dimsSequence[LeafIntSpec]rr%c t|D]a\}}|Ft|ttfs*t d|dt |jd|t|d|bt||_ dS)NzTensorSpec dim z: expected LeafIntSpec, got rxrv) enumerateryr^r r7r6r8r{list_specs)r,rids r.r+zTensorSpec.__init__sdOO ? ?DAq}ZC=%A%A}1a11Aww'11+,11 q(=!(=(= > > > > >)-d r/indexr^rc|j|Srir)r,rs r. __getitem__zTensorSpec.__getitem__&s{5!!r/c*t|jSri)lenrrks r.__len__zTensorSpec.__len__)s4;r/Iterator[LeafIntSpec]c*t|jSri)iterrrks r.__iter__zTensorSpec.__iter__,sDK   r/r1cdg}t|jD]&\}}|t|d|'d|S)NzTensor:rx )rrrd_INDENTre)r,linesrspecs r.rgzTensorSpec.__repr__/s`  -- 4 4GAt LLG2Q22$22 3 3 3 3yyr/rmc*dd|jDdS)NrcZg|](}t|dr|n|)Srohasattrro.0rs r. z*TensorSpec.to_jsonable..8sH'.dM&B&BL  """r/)r6rrrks r.rozTensorSpec.to_jsonable5s2  K   r/N)rrrr%)rr^rrrq)rrrprr) r8r9r:r;r+rrrrgror/r.rrs  4444""""    !!!!          r/rcLeZdZdZdddZdd Zdd ZddZddZddZ ddZ dS)raSpec for any Python object's attributes. Constructor:: ObjectSpec({name: IntermediateSpec, ...}) Values may be leaves (``TensorSpec`` / ``IntVar`` / ``int`` / ``None``) or another ``ObjectSpec`` for recursion. Example:: ObjectSpec({"weight": TensorSpec([ShapeVar("h"), None])}) ObjectSpec({"inner": ObjectSpec({"weight": TensorSpec([ShapeVar("h")])})}) Nfields"dict[str, IntermediateSpec] | Nonerr%c6|rt|ni|_dSri)dict_fields)r,rs r.r+zObjectSpec.__init__OsDJ4RDLLLPR r/r0r(boolc||jvSri)r)r,r0s r. __contains__zObjectSpec.__contains__Rst|##r/ Iterator[str]c*t|jSri)rrrks r.rzObjectSpec.__iter__UsDL!!!r/r^c*t|jSri)rrrks r.rzObjectSpec.__len__Xs4<   r/rc4|jSriritemsrks r.rzObjectSpec.items[s|!!###r/r1cdg}|jD]\}}t|}d|vrX|td|d|D]"}|tdz|z#p|td|d|d|S)Nz object_spec:r.:rx)rrreprrdr splitlinesre)r,rr0r spec_reprlines r.rgzObjectSpec.__repr__^s ,,,.. ? ?JD$T Iy   11$111222%002255DLL1t!344445 ==$==)==>>>>yyr/rmcNdd|jDdS)Nrcbi|],\}}|t|dr|n|-Srr)rr0rs r. z*ObjectSpec.to_jsonable..msND$GD-,H,HRd&&(((dr/)r6rrrks r.rozObjectSpec.to_jsonablejs> "&,"4"4"6"6   r/ri)rrrr%)r0r(rr)rrrqrrrprr r8r9r:r;r+rrrrrgrorr/r.rr?s  SSSSS$$$$""""!!!!$$$$           r/rcNeZdZdZ dddZdd Zdd ZddZddZddZ ddZ dS)raTSpec for a Python ``dict``-typed value. Constructor:: DictSpec({key: IntermediateSpec, ...}) Keys may be ``str`` or ``int``. Example:: DictSpec({"x": TensorSpec([ShapeVar("h"), None])}) DictSpec({"config": DictSpec({"batch": IntVar()})}) DictSpec({0: TensorSpec([ShapeVar("h"), None])}) Nentries(dict[str | int, IntermediateSpec] | Nonerr%c|rt|ni|_|jD]E}t|ttfs't dt |jd|FdS)Nz0DictSpec entries must have str or int keys, got rx)r_entriesryr1r^r7r6r8)r,rks r.r+zDictSpec.__init__s% ,DMMM"   Aa#s,, `tAwwGW``[\``   r/keyr(rc||jvSrir)r,rs r.rzDictSpec.__contains__sdm##r/Iterator[str | int]c*t|jSri)rrrks r.rzDictSpec.__iter__sDM"""r/r^c*t|jSrirrrks r.rzDictSpec.__len__4=!!!r/rc4|jSrirrrks r.rzDictSpec.itemss}""$$$r/r1cdg}|jD]\}}t|}d|vrX|td|d|D]"}|tdz|z#p|td|d|d|S)Nz dict_spec:r[]:r]: )rrrrdrrre)r,rrrrrs r.rgzDictSpec.__repr__s,,.. A AICT Iy   33#333444%002255DLL1t!344445 ??#??I??@@@@yyr/rmcNdd|jDdS)Nrc|i|]9\}}t|t|dr|n|:Sr)r1rro)rrrs r.rz(DictSpec.to_jsonable..sUCCm0L0LV$**,,,RVr/r6rrrks r.rozDictSpec.to_jsonables>!%!4!4!6!6   r/ri)rrrr%)rr(rr)rrrqrrprrrrr/r.rrts   CG     $$$$####""""%%%%           r/rc2eZdZdZddZddZdd Zdd Zd S)raSpec for a Python ``list``- or ``tuple``-typed value. Per-position. The spec list may be shorter than the runtime sequence; any positions beyond ``len(self)`` are treated as fully static (i.e. equivalent to an unspecified slot). Constructor:: SeqSpec([IntermediateSpec, ...]) Example:: SeqSpec([TensorSpec([ShapeVar("h"), 10]), 1]) SeqSpec((TensorSpec([ShapeVar("a")]), TensorSpec([ShapeVar("b")]))) rSequence[IntermediateSpec]rr%c.t||_dSri)rr)r,rs r.r+zSeqSpec.__init__s04W  r/r^c*t|jSrirrks r.rzSeqSpec.__len__rr/r1cdg}t|jD]\}}t|}d|vrX|td|d|D]"}|tdz|z#p|td|d|d|S)Nz seq_spec:rrrrr)rrrrdrrre)r,rrrrrs r.rgzSeqSpec.__repr__s  // = =GAtT Iy   //!///000%002255DLL1t!344445 ;;!;; ;;<<<<yyr/rmc*dd|jDdS)NrcZg|](}t|dr|n|)Srrrs r.rz'SeqSpec.to_jsonable..sH'.dM&B&BL  """r/rrrks r.rozSeqSpec.to_jsonables2 M   r/N)rrrr%rqrprr)r8r9r:r;r+rrgrorr/r.rrsn >>>>""""           r/rrrrrParamsSpecValuec6eZdZdZdZdZ ddd Zdd Zdd ZdS)ra"Specification for the arguments of a compiled function. Describes the dynamic shape behavior for named arguments, ``*args``, and ``**kwargs`` of a ``torch.compile``-wrapped function. Takes a single dict keyed by parameter name, with two reserved sentinel keys for the variadic slots:: def func(x, n, *args, **kwargs): ... ParamsSpec( { "x": TensorSpec([ShapeVar("batch"), None]), "n": IntVar("seq"), "*args": [TensorSpec([ShapeVar("a")]), None], "**kwargs": { "foo": TensorSpec([ShapeVar("b"), None]), "bar": TensorSpec([ShapeVar("c"), None]), }, } ) Anything not expressed in ``ParamsSpec`` is STATIC.** z*argsz**kwargsNparams!dict[str, ParamsSpecValue] | Nonerr%c i|_d|_d|_|dS|D]\}}||jkrt |t s,td|jdt|j t|D]\}}t|d|d|dt ||_||j krt |ts,td|j dt|j |D]\}}t|d|d|dt||_(|dr#td |d |jd |j d t|d |dtt ||j|<dS)Nz ParamsSpec z) value must be a list of leaf specs, got z ParamsSpec 'z'[]rz) value must be a dict of leaf specs, got *zUnknown sentinel key z in ParamsSpec; only z and z are reservedz ParamsSpec[) _named_args_varargs_varkwr _VARARGS_KEYryrr~r6r8rr{ _VARKW_KEYrr4rr)r,rrvaluerrurs r.r+zParamsSpec.__init__s]9;7; :> > F ,,.. F FJCd'''!%..$Ed&7EE.25kk.BEE&e,,LLDAq&q0Js0J0Ja0J0J0JKKKKK $U  ''!%..$EdoEE.25kk.BEE"KKMMNNDAq&q0Ls0L0La0L0L0LMMMMM"5kk $$ F RCRR(RR15RRR #50Fc0F0F0FGGGG(,-=u(E(E %%5 F Fr/r1c.tfdddfd fd |jD}|j[d fd t |jD}||jd ||j`d fd|jD}||j d |d |S)Ntextr1prefixrcjdfd|DS)Nrc3"K|] }|zV dSrir)rrrs r. z=ParamsSpec.__repr__.._indent_lines..>s'IItVd]IIIIIIr/)rer)rrs `r. _indent_linesz*ParamsSpec.__repr__.._indent_lines=s499IIIIt7H7HIIIII Ir/rrrcXt|}d|vr|d|S|d|S)Nr: rxr)rrv_reprrs r. _entry_reprz(ParamsSpec.__repr__.._entry_repr@sI%[[Fv~~99--"7"7999%%V%% %r/c.g|]\}}||Srrrrrurs r.rz'ParamsSpec.__repr__..Fs)HHHtq!Q""HHHr/rc3PK|] \}}t||V!dSri)r1)rrrurs r.rz&ParamsSpec.__repr__..HsJ+/1a CFFA&&r/rc36K|]\}}||VdSrirrs r.rz&ParamsSpec.__repr__..Ms3PPDAqkk!Q//PPPPPPr/)rr1rr1rr1)rr1rrrr1) rrrrrerrdrrr)r,rinnerrrs @@r.rgzParamsSpec.__repr__<sb3: J J J J J & & & & & & IHHHt/?/E/E/G/GHHH = $II3.RsN" " " e  )F)FQ%##%%%E" " " r/cZg|](}t|dr|n|)Srr)rrus r.rz*ParamsSpec.to_jsonable..WsC)))$+1m#<#<C !)))r/cbi|],\}}|t|dr|n|-Srrrs r.rz*ParamsSpec.to_jsonable..\sO'''D%WUM-J-JUe'')))PU'''r/r)r6r)rrrrrr)r,rs r.rozParamsSpec.to_jsonableQs" " #/5577" " "  = $)))))F4$ % ; "''#';#4#4#6#6'''F4? # !   r/ri)rrrr%rprr) r8r9r:r;rrr+rgrorr/r.rrsv2LJ59#F#F#F#F#FJ    *      r/rcNeZdZdZ dddddd ZeddZddZddZdS)ruGTop-level shape specification for a ``torch.compile`` call. ``params`` describes the arguments of the compiled callable — for a raw function this is the function's parameters, for an ``nn.Module`` this is the parameters of ``forward`` (excluding ``self``). ``assumptions`` is an optional list of ``SymBool`` expressions built from spec ``IntVar`` / ``ShapeVar`` values. Each assumption is wired into the shape env at compile time and asserted at runtime via the deferred-runtime-assert mechanism:: A = ShapeVar("a") B = ShapeVar("b") ShapesSpec( params={"x": TensorSpec([A, None]), "y": TensorSpec([B, None])}, assumptions=[A + B > 10, A * 2 == B], ) ``globals`` is reserved for future use and will raise ``NotImplementedError`` if set. N)globals assumptionsr.ParamsSpec | dict[str, ParamsSpecValue] | NonerrrSequence[SymBool] | Nonerr%c  d|_g|_|tdt|trt |}n;|9t|t s$t dt|j||_|t|D]u\}}t|ts*t d|dt|jd|t|d|d|j |tdSdS)Nz'ShapesSpec.globals is not supported yetz3shapes_spec must be ParamsSpec, dict, or None, got zShapesSpec.assumptions[z]: expected SymBool, got rxrr) _params _assumptionsNotImplementedErrorryrrr7r6r8rrr{rd)r,rrrras r.r+zShapesSpec.__init__}s\+/ +-  %&OPP P fd # # ''FF   6:(F(F /F||,//   "!+.. , ,1!!W--#9!99#Aww/993499#1,Ja,J,J,JKKKK!((++++ # " , ,r/ list[SymBool]c*t|jSri)rrrks r.rzShapesSpec.assumptionssD%&&&r/r1cdg}|jh|tdt|j}|D]"}|tdz|z#|jrT|td|jD]/}|tdzt|z0d|S)Nz shapes_spec:zparams:rz assumptions:r)rrdrrrrre)r,r param_reprrrs r.rgzShapesSpec.__repr__s  < # LLG,,, - - -dl++J"--// 1 1 Wq[4/0000   4 LLG111 2 2 2& 4 4 Wq[47723333yyr/rmcld|jdn|jd|jDdS)Nrc,g|]}t|Srr)rrs r.rz*ShapesSpec.to_jsonable..s???DGG???r/)r6rr)rrorrks r.rozShapesSpec.to_jsonablesD "l2dd 8P8P8R8R??T->???   r/ri)rrrrrrrr%)rr rprr) r8r9r:r;r+propertyrrgrorr/r.rrfs0BF",04 ",",",",",",H'''X'           r/r)rr )rurrvr1rr%),r; __future__rrs threadingtypingrrrrr[torchrr collections.abcr r rBr __all__rrrr=LockrArCrr{rrrrrr^rrrrrr1rrrrr/r.rs/ #"""""666666666666 !!!!!!!!?22222222>>>>>>   "  $('''' &y~''>>>>BZ Z Z Z Z VZ Z Z z    "VVVVVvVVV8. . . . . . . . b2 2 2 2 2 2 2 2 j; ; ; ; ; ; ; ; |* * * * * * * * p &3.5 5555!J.....'3h>HHHHHt,--S:J5J0KK i i i i i i i i XQ Q Q Q Q Q Q Q Q Q r/