tj"ddlZddlZddlZddlmZddlmZddlmZddl Z ddl m Z ddl mZddlmZdd lmZdd lmZdd lmZmZmZmZdd lmZmZdd lmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-ddl.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7e$rddl8Z8e&r ddl9m:cm;cmZ>e'j?e@ZAdZBe"deBe-dGddeZCe#eCjDeC_DeCjDjE.eCjDjEFdddeCjD_EdSdS)N)Callable)partial)Any)is_offline_mode)validate_typed_dict)custom_object_save)TorchvisionBackend) BatchFeature)ChannelDimensionSizeDictis_vision_availablevalidate_kwargs)Unpack VideosKwargs) IMAGE_PROCESSOR_NAMEPROCESSOR_NAMEVIDEO_PROCESSOR_NAME TensorTypeadd_start_docstrings copy_funcis_torch_availableis_torchcodec_availableis_torchvision_v2_availableloggingsafe_load_json_file) cached_filehf_api)requires) VideoInput VideoMetadatagroup_videos_by_shapeinfer_channel_dimension_formatis_valid_video load_videomake_batched_metadatamake_batched_videosreorder_videos)PILImageResamplinga Args: do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the video's (height, width) dimensions to the specified `size`. Can be overridden by the `do_resize` parameter in the `preprocess` method. size (`dict`, *optional*, defaults to `self.size`): Size of the output video after resizing. Can be overridden by the `size` parameter in the `preprocess` method. size_divisor (`int`, *optional*, defaults to `self.size_divisor`): The size by which to make sure both the height and width can be divided. default_to_square (`bool`, *optional*, defaults to `self.default_to_square`): Whether to default to a square video when resizing, if size is an int. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the video. Only has an effect if `do_resize` is set to `True`. Can be overridden by the `resample` parameter in the `preprocess` method. do_center_crop (`bool`, *optional*, defaults to `self.do_center_crop`): Whether to center crop the video to the specified `crop_size`. Can be overridden by `do_center_crop` in the `preprocess` method. crop_size (`dict[str, int]` *optional*, defaults to `self.crop_size`): Size of the output video after applying `center_crop`. Can be overridden by `crop_size` in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the video by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `self.rescale_factor`): Scale factor to use if rescaling the video. Only has an effect if `do_rescale` is set to `True`. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the video. Can be overridden by the `do_normalize` parameter in the `preprocess` method. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`float` or `list[float]`, *optional*, defaults to `self.image_mean`): Mean to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_mean` parameter in the `preprocess` method. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `list[float]`, *optional*, defaults to `self.image_std`): Standard deviation to use if normalizing the video. This is a float or list of floats the length of the number of channels in the video. Can be overridden by the `image_std` parameter in the `preprocess` method. Can be overridden by the `image_std` parameter in the `preprocess` method. do_convert_rgb (`bool`, *optional*, defaults to `self.image_std`): Whether to convert the video to RGB. video_metadata (`VideoMetadata`, *optional*): Metadata of the video containing information about total duration, fps and total number of frames. do_sample_frames (`int`, *optional*, defaults to `self.do_sample_frames`): Whether to sample frames from the video before processing or to process the whole video. num_frames (`int`, *optional*, defaults to `self.num_frames`): Maximum number of frames to sample when `do_sample_frames=True`. fps (`int` or `float`, *optional*, defaults to `self.fps`): Target frames to sample per second when `do_sample_frames=True`. return_tensors (`str` or `TensorType`, *optional*): Returns stacked tensors if set to `pt, otherwise returns a list of tensors. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input video. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input video. If unset, the channel dimension format is inferred from the input video. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: video in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: video in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: video in (height, width) format. device (`torch.device`, *optional*): The device to process the videos on. If unset, the device is inferred from the input videos. return_metadata (`bool`, *optional*): Whether to return video metadata or not. z!Constructs a base VideoProcessor.)vision torchvision)backendsceZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZdZdZdZdZdZdZeZdgZdeeddffd Zdefd Zd d defd Z dAd ede dzde e!zdzfdZ" dAdedee#zde$dzde%dzde&d f dZ' dAdede(e)zdzde(dzde&d fdZ*e+e,dedeedefdZ- dBde&d de$de$de.ddde$d e.d!e$d"e!d#e$d$e!e&e!zdzd%e!e&e!zdzd&e(e/zdzdefd'Z0e1 dCd)e(e2j3zd*e(e2j3zdzd+e$d,e$d-e(e$zdzd.e(f d/Z4dDd0e(e2j3zd1e$fd2Z5e1d)e(e2j3zde6e#e(e7fe#e(e7fffd3Z8e1d4e#e(e7ffd5Z9de#e(e7fffd6 Z:de(fd7Z;d8e(e2j3zfd9Ze1dEd>Z?dBd?e(e&e(ze&e&e(zfd@Z@xZAS)FBaseVideoProcessorNTgp?Fpixel_values_videoskwargsreturnc :tjdi|dS)N)super__init__)selfr0 __class__s a/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/video_processing_utils.pyr5zBaseVideoProcessor.__init__s&""6"""""c |j|fi|SN) preprocess)r6videosr0s r8__call__zBaseVideoProcessor.__call__stv00000r9videoz torch.Tensorc:tj|}|jddks&|ddddddfdks|S|ddddddfdz }d|ddddddfz dz|ddddddf|dddddddfzz}|S)z Converts a video to RGB format. Args: video (`"torch.Tensor"`): The video to convert. Returns: `torch.Tensor`: The converted video. .Ngo@r)tvFgrayscale_to_rgbshapeany)r6r?alphas r8convert_to_rgbz!BaseVideoProcessor.convert_to_rgbs$U++ ;r?a  c1aaal(;c(A'F'F'H'H Lc1aaal#e+U3aaa?++s2U3aaa?5KeTWY[Z[Y[]^]^]^`a`a`aTaNb5bb r9metadata num_framesfpsc ||td||n|j}||n|j}|j}|4|2||jtdt ||jz |z}||krtd|d|d|,t jd|||z }n't jd|}|S)a% Default sampling function which uniformly samples the desired number of frames between 0 and total number of frames. If `fps` is passed along with metadata, `fps` frames per second are sampled uniformty. Arguments `num_frames` and `fps` are mutually exclusive. Args: metadata (`VideoMetadata`): Metadata of the video containing information about total duration, fps and total number of frames. num_frames (`int`, *optional*): Maximum number of frames to sample. Defaults to `self.num_frames`. fps (`int` or `float`, *optional*): Target frames to sample per second. Defaults to `self.fps`. Returns: np.ndarray: Indices to sample video frames. Nzc`num_frames`, `fps`, and `sample_indices_fn` are mutually exclusive arguments, please use only one!zAsked to sample `fps` frames per second but no video metadata was provided which is required when sampling with `fps`. Please pass in `VideoMetadata` object or use a fixed `num_frames` per input videoz(Video can't be sampled. The `num_frames=z` exceeds `total_num_frames=z`. r) ValueErrorrKrLtotal_num_framesinttorcharange)r6rJrKrLr0rOindicess r8 sample_framesz BaseVideoProcessor.sample_framess"0 ?z5u $.#9ZZt _cc$(#4  #/8<#7 h- zKBaseVideoProcessor._decode_and_sample_videos...s' O O Ou!3!3E!:!: O O Or9r)dim)rQstack)r\imagesr6s r8r^z@BaseVideoProcessor._decode_and_sample_videos..sNK O O O O O O OUVWWWr9zUSampling frames from a list of images is not supported! Set `do_sample_frames=False`.rW) r'r&r$zipframes_indicesappend isinstancelist fetch_imagesrN fetch_videos) r6r=rUrVrWsampled_videossampled_metadatar?rJrSs ` r8_decode_and_sample_videosz,BaseVideoProcessor._decode_and_sample_videosss%V,,.vnUUU &) $ $ h)9 hN! #&v~#>#> 2 2x++X>>>*1'%%eGn555 ''1111#F-NNq ** h&)T** h"&"3"3F";";$$o *.):):6Uf):)g)g&~%%r9input_data_formatdevicecg}|D]}t|tjr&tj|}|t |}|tjkr*| dddd}|| |}| ||S)z: Prepare the input videos for processing. NrrBr) rfnpndarrayrQ from_numpy contiguousr#r LASTpermutetore)r6r=rmrnprocessed_videosr?s r8_prepare_input_videosz(BaseVideoProcessor._prepare_input_videos*s + +E%,, =(//::<<!($B5$I$I! $4$999 aAq11<<>>!((  # #E * * * *r9c (t|t|jjdgzt |j||jjD]'}||t||d(|d}|d}|d}|d}|rt|j fi|nd}| ||||\}}| |||}|j d i|}|jd i||d |d } |jd d |i|} | r|| d<| S) Nreturn_tensors)captured_kwargsvalid_processor_keysrmrVrnrU)rUrVrW)r=rmrn data_formatreturn_metadatar=r3)rkeysrg valid_kwargs__annotations__r setdefaultgetattrpoprrTrlry_standardize_kwargs_validate_preprocess_kwargs _preprocess) r6r=r0 kwarg_namermrVrnrUrWrpreprocessed_videoss r8r<zBaseVideoProcessor.preprocessGs "KKMM!%d&7&G&L&L&N&N!O!OScRd!d    D-v666+; K KJ   j'$ D*I*I J J J J"JJ':;;!::&899H%%$455EU_GD$6AA&AAA[_!%!?!? )-/ "@" "  ++6M^gm+nn))33F33((226222  =!!! **%677.d.GGfGGG  C4B  0 1""r9do_convert_rgb do_resizesizeresamplez7PILImageResampling | tvF.InterpolationMode | int | Nonedo_center_crop crop_size do_rescalerescale_factor do_normalize image_mean image_stdr{c t|\}}i}|D];\}}|r||}|r||||}|||< E!%!4!4^!D!D [!%^$QY!Z!Z,: "5 ) )'(>@TUU0E^/T/T,,#% %3%9%9%;%; = = !E> M!%!1!1.)!L!L!77 NL*V_N/= $U + +)*BDXYY"79I!JXfggggr9mainpretrained_model_name_or_path cache_dirforce_downloadlocal_files_onlytokenrevisionc v||d<||d<||d<||d<|||d<|j|fi|\}}|j|fi|S)aK Instantiate a type of [`~video_processing_utils.VideoProcessorBase`] from an video processor. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained video hosted inside a model repo on huggingface.co. - a path to a *directory* containing a video processor file saved using the [`~video_processing_utils.VideoProcessorBase.save_pretrained`] method, e.g., `./my_model_directory/`. - a path to a saved video processor JSON *file*, e.g., `./my_model_directory/video_preprocessor_config.json`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model video processor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the video processor files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. return_unused_kwargs (`bool`, *optional*, defaults to `False`): If `False`, then this function returns just the final video processor object. If `True`, then this functions returns a `Tuple(video_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not video processor attributes: i.e., the part of `kwargs` which has not been used to update `video_processor` and is otherwise ignored. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. kwargs (`dict[str, Any]`, *optional*): The values in kwargs of any keys which are video processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* video processor attributes is controlled by the `return_unused_kwargs` keyword parameter. Returns: A video processor of type [`~video_processing_utils.ImagVideoProcessorBase`]. Examples: ```python # We can't instantiate directly the base class *VideoProcessorBase* so let's show the examples on a # derived class: *LlavaOnevisionVideoProcessor* video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf" ) # Download video_processing_config from huggingface.co and cache. video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "./test/saved_model/" ) # E.g. video processor (or model) was saved using *save_pretrained('./test/saved_model/')* video_processor = LlavaOnevisionVideoProcessor.from_pretrained("./test/saved_model/video_preprocessor_config.json") video_processor = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False ) assert video_processor.do_normalize is False video_processor, unused_kwargs = LlavaOnevisionVideoProcessor.from_pretrained( "llava-hf/llava-onevision-qwen2-0.5b-ov-hf", do_normalize=False, foo=False, return_unused_kwargs=True ) assert video_processor.do_normalize is False assert unused_kwargs == {"foo": False} ```rrrrNr)get_video_processor_dict from_dict) clsrrrrrrr0video_processor_dicts r8from_pretrainedz"BaseVideoProcessor.from_pretraineds{n({#1 %5!"%z  #F7O'Cs'CDa'l'lek'l'l$fs}1<>. ) ) h !f>!f!f!fgg g NT2222  J#ZZ(8$??NjjN,@,@,M,Mb,QRRG*fhh*7LLTLVLLTG#99.II    ' t^D A A A A')gll>CW&X&X# 5666 M0KMMNNN    ' ' -jj)) (   ,,,r9c  |dd |dd |dd|dd|dd|dd|d d |d d}|d d}d |d||d<trstddt t j}t jr}d}d}nt} tt d } f d|tfD} | r| dnd}n1#t$rt$rtddd|dwxYwd} |t|} d| vr| d} || t|} | tddd|d|rtd|n td|d|| |fS)a From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a video processor of type [`~video_processing_utils.VideoProcessorBase`] using `from_dict`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. Returns: `tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the video processor object. rNrFproxiesrrr subfolder_from_pipeline _from_autovideo processor) file_typefrom_auto_classusing_pipelinez+Offline mode: forcing local_files_only=TrueT filenamerrrrr user_agentrr%_raise_exceptions_for_missing_entriescN g|]!}t|  d x "S)Fr)r) r\rrrrrr resolved_filerrrrs r8r^z?BaseVideoProcessor.get_video_processor_dict..ssm222 )49%-&/+9$+-="''1%-&/BG * * *    "   r9rz Can't load video processor for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z filevideo_processorzloading configuration file z from cache at )rrrrstrrrisdirrrrrrOSError Exceptionr)rrr0 from_pipelineris_localresolved_video_processor_fileresolved_processor_filevideo_processor_fileresolved_video_processor_filesrprocessor_dictrrrrrrrrrs ` @@@@@@@@@r8rz+BaseVideoProcessor.get_video_processor_dict6s$JJ{D11 $4e<<**Y-- 7D))!::&8%@@::j$//JJ{B//  #3T::  **\599#4YY  $+8J' (    $%5 $ KKE F F F# (+,I(J(J%7==!>?? 7>>7 8 88 ,I )&* #HH#7 2 +61+'#1#%5)%':? + + +'2222222222222%9;O$P222.*:Xa2155]a.-      K7TKK9VKK0DKKK  $ " .01HIIN N22'56G'H$ ( 49M9U#67T#U#U  'G3PGG5RGG,@GGG    KKU6SUU V V V V KKr.BrrSprr   $V++s %A F//.Grc ~|}|dd}|fd|Dd i|}g}t t |D]Z}t||rH|jj vr:t||||d| |[|r&t djd|dtd||r||fS|S) a Instantiates a type of [`~video_processing_utils.VideoProcessorBase`] from a Python dictionary of parameters. Args: video_processor_dict (`dict[str, Any]`): Dictionary that will be used to instantiate the video processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [`~video_processing_utils.VideoProcessorBase.to_dict`] method. kwargs (`dict[str, Any]`): Additional parameters from which to initialize the video processor object. Returns: [`~video_processing_utils.VideoProcessorBase`]: The video processor object instantiated from those parameters. return_unused_kwargsFc8i|]\}}|jjv||Sr3)rr)r\kvrs r8 z0BaseVideoProcessor.from_dict..s/$n$n$ndaSM]MmHmHmQHmHmHmr9NzImage processor z : kwargs z were applied for backward compatibility. To avoid this warning, add them to valid_kwargs: create a custom TypedDict extending ImagesKwargs with these keys and set it as the `valid_kwargs` class attribute.zVideo processor r3)copyrupdaterreversedrgrhasattrrrsetattrrer warning_once__name__r)rrr0rr extra_keyskeys` r8rzBaseVideoProcessor.from_dictsx" 488::%zz*@%HH##$n$n$n$nfllnn$n$n$nooo#55 455 D//00 ' 'C,, 'C This API is experimental and may have some slight breaking changes in the next releases. Args: auto_class (`str` or `type`, *optional*, defaults to `"AutoVideoProcessor "`): The auto class to register this new video processor with. rNz is not a valid auto class.) rfrrtransformers.models.automodelsautorrNr)r auto_class auto_modules r8register_for_auto_classz*BaseVideoProcessor.register_for_auto_classsn *c** -#,J666666666{J// I GGGHH H$r9video_url_or_urlscd}tstjdd}t|tr#t t fd|DSt ||S)z Convert a single or a list of urls into the corresponding `np.array` objects. If a single url is passed, the return value will be a single object. If a list is passed a list of objects is returned. torchcodecz`torchcodec` is not installed and cannot be used to decode the video by default. Falling back to `torchvision`. Note that `torchvision` decoding is deprecated and will be removed in future versions. r+c>g|]}|S)rb)ri)r\xrWr6s r8r^z3BaseVideoProcessor.fetch_videos..Ds.sss\]d//EV/WWsssr9)backendrW)rwarningswarnrfrgrcr%)r6r!rWr&s` ` r8rizBaseVideoProcessor.fetch_videos4s&(( $ MI   $G ' . . gsssssarssstuu u/Tefff fr9)NNr;)NFFNr)F)r)Br __module__ __qualname__rrrrr size_divisordefault_to_squarerrrrrrrrVrLrKrUrrrmodel_input_namesrr5r r>r rIr!rPfloatrTdictboolrrgrlrr ryrBASE_VIDEO_PROCESSOR_DOCSTRINGr<r rr classmethodrPathLikerrtuplerrrrrrrrr ri __classcell__)r7s@r8r.r.s KHJI DLIINJNLN CJNOL./# !5#$######1L1111 8"&"& 333$J35[4  3333r)--1 &&&&&&&,&&+ && $d? && n  &&&&&&&&V<@!    !11D8 d  n      :&*#*#&*#  *#*#*#*#t37+h+h^$+h+h +h  +h L +h+h+h+h+h+hDK'$.+h4;&-+hj(4/+h !+h+h+h+hZ/3$!&#'`=`='*R['8`=$t+`= `=  `= TzD `=`=`=`=[`=D.-.-cBK.?.-d.-.-.-.-`{,,/"+,={, tCH~tCH~- .{,{,{,[{,z&#T#s(^&#&#&#[&#P c3h       G G G G G 03+< 0 0 0 0DDD+sR['8+++[+$%%%[%2ggcDIoT#Y.Oggggggggr9r.rrzvideo processor file)object object_class object_files)Grrr'collections.abcr functoolsrtypingrnumpyrqhuggingface_hubrhuggingface_hub.dataclassesrdynamic_module_utilsr image_processing_backendsr image_processing_utilsr image_utilsr r rrprocessing_utilsrrutilsrrrrrrrrrrr utils.hubrrutils.import_utilsr video_utilsr r!r"r#r$r%r&r'r(rQ$torchvision.transforms.v2.functional transformsv2 functionalrDr) get_loggerrrr1r.r__doc__formatr3r9r8rOs $$$$$$++++++;;;;;;444444999999000000 32222222                          +*******((((((                      LLL  76666666666660//////  H % %A" H'" ,---p gp gp gp gp g+p gp g.-  p gf"++=+I!J!J!)5-?-K-S-Z-Z /CRh.[.."***65r9