tj/J UdZddlZddlZddlmZddlmZddlZddlmZddl m Z m Z ddl m Z ddlmZd d lmZejZejZd Zeejjd fed <dagaeeegdfeefed<ejZejZ e a!e"ejdr ejj#Z$nde%de%fdZ$e"ejdr ejj&Z'nde%de%fdZ'dZ(dZ)dZ*de+fdZ,dJdZ-Gdde.Z/de+fdZ0de+fdZ1dKdeddfdZ2de%fd Z3de%fd!Z4dKdedefd"Z5dKdedefd#Z6 dLd&edzd'ed(e%ddfd)Z7de8eeffd*Z9d+ee%e%e%e%gdfddfd,Z:dMd.e+fd/Z;dKdedee%e%ffd0Zdeddfd4Z?dKdede8eeffd5Z@Gd6dZAGd7d8ZBd9ZCd2edzdeBfd:ZDdNdedefd<ZEdeefd=ZFdNd>ededdfd?ZGd@eeddfdAZHdBe%ddfdCZIdBe%ddfdDZJde%fdEZKde%fdFZLde%fdGZMd dHlNTd dHlOTgdIZPdS)OzH This package enables an interface for accessing MTIA backend in python N)Callable)Any)Tensor) is_fbcodeis_prod)_LazySeedTracker)Device)_get_device_index.default_generatorsF _queued_calls_mtia_exchangeDevicedevicereturnc0|dkrdStdNrz)PyTorch was compiled without MTIA support RuntimeErrorrs Q/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/mtia/__init__.py_exchange_devicer) A::2FGGG_mtia_maybeExchangeDevicec0|dkrdStdrrrs r_maybe_exchange_devicer3rrc"tdSN) _lazy_initr rrinitr"9sLLLLLrc.tot S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkr rris_initializedr&=s  1 1 111rc t5tr|cdddS|ddr-t|t jnp|ddr-t|t jn-t |t jfddddS#1swxYwYdS)Nseed_allFseed) _initialization_lockr&get_lazy_seed_trackerqueue_seed_all traceback format_stack queue_seedrappend)callablekwargss r _lazy_callr4Bs^  K K    K8:: K K K K K K K K zz*e,, K"11(ItjSr )torch_C_mtia_isInBadForkr rrr%r%Ps 8 % % ' ''rc tsttdrdSt5tr ddddSt rt dt stdtr:tr, ddl m }| n#t$rYnwxYwtjdt_t$dt(D t$D]Z\}} |#t,$r=}dt/|d d |}t3||d}~wwxYw t5tdn#t5tdwxYwdaddddS#1swxYwYdS) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!r)cxx_resource_managerTc3K|]}||V dSr r ).0callss r z_lazy_init..s(XXuRWXUXXXXXXrz6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r&hasattr_tlsr*r%r _is_compiledAssertionErrorrrlibfb.py.cxx_resourcesr;installModuleNotFoundErrorr6r7 _mtia_initr:rextendr, get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr$)r; queued_callorig_tracebackemsgs rr!r!Ts74):;; 77     77777777    I ~~  *  ;; 799  GGGGGG$,,....&      $XX0B0L0L0N0NXXXXXX -/< < <+ ^<KMMMM <<<]QTUVQWQW]]CE77>CZCZ]]044!; < < D+ , , , ,GD+ , , , , o777777777777777777syG AG!B<;G< C GC  A(G2 F+? E  F+ F8F  FF+G+GGGGceZdZdS)rNN)__name__ __module__ __qualname__r rrrNrNsDrrNc>tjS)z*Return true if compiled with MTIA support.)r6r7 _mtia_isBuiltr rrrCrCs 8 ! ! # ##rcFtsdStdkS)z'Return true if MTIA device is availableFr)rC device_countr rr is_availabler\s" >>u >>A rctj|5tjcdddS#1swxYwYdS)z?Waits for all jobs in all streams on a MTIA device to complete.N)r6mtiarr7_mtia_deviceSynchronizers r synchronizer`s   6 " "22x//11222222222222222222sA  AAc>tjS)z,Return the number of MTIA devices available.)r6r7_mtia_getDeviceCountr rrr[r[s 8 ( ( * **rc>tjS)z0Return the index of a currently selected device.)r6r7_mtia_getDevicer rrcurrent_deviceres 8 # # % %%rcbt|d}tj|S)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r r6r7_mtia_getCurrentStream)r device_idxs rcurrent_streamrks,#6D999J 8 * *: 6 66rc^tjt|dS)a=Return the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Trg)r6r7_mtia_getDefaultStreamr rs rdefault_streamrns( 8 * *+tjS)z4Return a dictionary of MTIA memory allocator history)r6r7_mtia_memorySnapshotr rrsnapshotrys 8 ( ( * **robservercDtj|dS)z9Attach an out-of-memory observer to MTIA memory allocatorN)r6r7_mtia_attachOutOfMemoryObserver)rzs rattach_out_of_memory_observerr}s  H,,X66666rTincluding_emulationcdS)zLReturn a bool indicating if the current MTIA device supports dtype bfloat16.Tr )r~s ris_bf16_supportedrs 4rc^tjt|dS)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Trg)r6r7_mtia_getDeviceCapabilityr rs rget_device_capabilityr) 8 - -.?QU.V.V.V W WWrc>tjS)zEmpty the MTIA device cache.)r6r7_mtia_emptyCacher rr empty_cachers 8 $ $ & &&rstreamcL|dStj|dS)aSet the current stream. This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r6r7_mtia_setCurrentStreamrs r set_streamrs*~ H##F+++++rcrt|}|dkr!tj|dSdS)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r6r7_mtia_setDevicers r set_devicers=v & &F {{   ((((({rc^tjt|dS)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Trg)r6r7_mtia_getDevicePropertiesr rs rget_device_propertiesrrrc8eZdZdZdefdZdZdededefdZdS) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. c@t|d|_d|_dS)NTrgr)r idxprev_idx)selfrs r__init__zdevice.__init__0s!$Vd;;; rcXtj|j|_dSr )r6r7rrr)rs r __enter__zdevice.__enter__4s::48DD rtypevaluer.cXtj|j|_dS)NF)r6r7rrr)rrrr.s r__exit__zdevice.__exit__8s 855dmDDurN)rUrVrW__doc__rrrrr rrrr(stsEEESrcPeZdZUdZedzed<dedzfdZdZdeded efd Z dS) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. N cur_streamrcd|_||_tdd|_tjs|jd|_tjsdntjd|_ tjsdntjd|_ dS)NTr) rrr rr6jit is_scriptingr^rnsrc_prev_streamdst_prev_stream)rrs rrzStreamContext.__init__Ls $T400y%%'' x ..00 UDDej6O6OPT6U6U  ..00 UDDej6O6OPT6U6U rc|j}| |jdkrdStjd|_|jj|jkrUt |j5tj|j|_dddn #1swxYwYtj|dSNr) rrr6r^rkrrrr)rrs rrzStreamContext.__enter__[s[  R F$z88>>   &**; ; ; )** T T',z'@'@AR'S'S$ T T T T T T T T T T T T T T T j)))))s$*BB!Brrr.c|j}| |jdkrdS|jj|jkr$tj|jtj|jdSr)rrrrr6r^rr)rrrr.rs rrzStreamContext.__exit__jsl[  R F   &**; ; ; J ! !$"6 7 7 7 d233333r) rUrVrWrStream__annotations__rrrrr rrrr>s     v}      * * * 4S 4 4 4 4 4 4 4 4rrcHtj|||dS)zset stream specified by the stream id, device index and device type Args: stream_id (int): stream id in stream pool device_index (int): device index in topo device_type (int): enum device type N)r6r7_mtia_setStream) stream_id device_index device_types r_set_stream_by_idrxs$ HY kBBBBBrc t|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )rrs rrrs   rr^ctt|d}|t}t|}|S)a[Returns the random number generator state of the specified MTIA device as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). .. warning:: This function eagerly initializes MTIA. Trg)r!r rer get_state)rrdefault_generators r get_rng_statersLLLL FT 2 2 2C {*3/  & & ( ((rcPdttD}|S)zRReturns a list of ByteTensor representing the random number states of all devices.c,g|]}t|Sr )r)r=is r z%get_rng_state_all..s ???A}Q???r)ranger[)resultss rget_rng_state_allrs&??|~~)>)>???G Nr new_statec>tsVtj5tjdddn #1swxYwYt |dtfd}t|dS)a8Sets the random number generator state of the specified MTIA device. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). ) memory_formatNTrgcLt}|dSr )r set_state)rrrs rcbzset_rng_state..cbs(.s3##I.....r) r&r6r7_DisableFuncTorchclonecontiguous_formatr rer4)rrrrs` @r set_rng_staters   O X ' ' ) ) O O"e6MNNI O O O O O O O O O O O O O O O FT 2 2 2C {//////rNNNNNs!AA #A  new_statescPt|D]\}}t||dS)zSets the random number generator state of all devices. Args: new_states (Iterable of torch.ByteTensor): The desired state for each device. N) enumerater)rrstates rset_rng_state_allrs< j))  5eQ  rr)cRtfd}t|ddS)aSets the seed for generating random numbers for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. .. warning:: If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use :func:`manual_seed_all`. cht}t|}|dSr )rer manual_seed)rrr)s rrzmanual_seed..cbs3.s3%%d+++++rTr)Nintr4r)rs` rrrsD t99D,,,,, rrcRtfd}t|ddS)zSets the seed for generating random numbers on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. cttD]$}t|}|%dSr )rr[r r)rrr)s rrzmanual_seed_all..cbsJ|~~&& 0 0A 21 5   ) )$ / / / / 0 0rTr(Nrrs` rmanual_seed_allrsD t99D00000 rD!!!!!!rc*d}t|dS)axSets the seed for generating random numbers to a random number for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. .. warning:: If you are working with a multi-GPU model, this function will only initialize the seed on one GPU. To initialize all GPUs, use :func:`seed_all`. c`t}t|}|Sr )rer r))rrs rrzseed..cbs*.s3 %%'''rTrr4rs rr)r)s&((( bt $ $ $$rc*d}t|dS)zSets the seed for generating random numbers to a random number on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. cd}d}ttD]=}t|}|s|}d}(||>|S)NrFT)rr[r r)r) random_seedseededrrs rrzseed_all..cbso |~~&& ; ;A 21 5  ;/4466 !--k::::rTrrrs rr(r(s&    b4 ( ( ((rcxtt}t|S)zReturns the current random seed of the current MTIA device. .. warning:: This function eagerly initializes MTIA. )r!rer initial_seed)rs rrrs/ LLL   C c " / / 1 11r)*)#r"r\r&r`r[rerkrn memory_statsmax_memory_allocatedmemory_allocatedreset_peak_memory_statsrrrvryr}rrrrrrrrrrrr)r(rr MTIAGraphgraphgraph_pool_handle)rNr )rorpr)T)r^)Qr threadingr.collections.abcrtypingrr6rtorch._environmentrr torch._utilsr torch.typesr _utilsr Eventrr tupler7 Generatorrr$rlistrLlocalrBLockr*r,rArrrrrr"r&r4boolr%r!rKrNrCr\r`r[rerkrnrvdictryr}rrrrrrrrrrrrrrrrr)r(rmemory mtia_graph__all__r rrrs$$$$$$ 11111111))))))%%%%%%   68E%(,c12777 t (2t8 d3i '(y%y~''%%'' 758+,,Hx4HHHHHH  758011H"X?HsHsHHHH 222 K K K(((((;;;;|     I   $d$$$$ d222$2222 +c++++ &&&&& 7 76 7V 7 7 7 7 U U6 UV U U U UMNEE 4ZE),EFIE EEEE&+$sCx.++++ 7S#s+T127 77774 XX&XE#s(OXXXX'''' ,v , , , , )v )$ ) ) ) )XX&XDcNXXXX,7474747474747474tCCC!6D=!]!!!!))&)f))))$4< VV2 $v, 4    cd*"#"$"""""%c%%%%")#))))*2c2222$ $ $ r