tjWdZddlmZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZdd lmZejeZerddlZed e Gd d eZd gZdS)z Processor class for SAM2. deepcopyN) ImageInput)ProcessorMixin) BatchEncoding) TensorTypeauto_docstringis_torch_availablelogging)requires)torch)backendsc.eZdZd$dedzdeffd Ze d%dedzdedzdeeeeee j zdzd eeeee j zdzd eeeee j zdzd eeee j zdzd e e zdzd e fdZ d&deddd dfdZd'dZd(dZd)dZdZdZ d(de j ejzezdede de dedzd ef dZd*dZ d+d"Zed#ZxZS), Sam2ProcessorN target_sizepoint_pad_valuec tj|fi|||_||n|jjd|_dS)a target_size (`int`, *optional*): The target size (in pixels) for normalizing input points and bounding boxes. If not provided, defaults to the image processor's size configuration. All input coordinates (points and boxes) are normalized to this size before being passed to the model. This ensures consistent coordinate representation regardless of the original image dimensions. point_pad_value (`int`, *optional*, defaults to -10): The value used for padding input points when batching sequences of different lengths. This value is used to mark padded positions and is preserved during coordinate normalization. Nheight)super__init__rimage_processorsizer)selfrrrkwargs __class__s f/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/sam2/processing_sam2.pyrzSam2Processor.__init__&sO 33F333.*5*A;;tG[G`aiGjimagessegmentation_maps input_points input_labels input_boxesoriginal_sizesreturn_tensorsreturnc ||j|f||d|} ne|Tt|tjr&|}t d|i|} ntd| d}|Bt|dkr/t|t|krtd||| | |dd d d } | |d dd} | |d ddd } | | | dd } | | | dd }| | | dd | | | |krtd| =t| d kr*tfd| Drtd| i| | | d gz}tj |tj}|||d| d|i| M| | |}tj |tj}| d|i| Otj | tj}|||d| d|i| S)as segmentation_maps (`ImageInput`, *optional*): The segmentation maps to process. input_points (`list[list[list[list[float]]]]`, `torch.Tensor`, *optional*): The points to add to the frame. input_labels (`list[list[list[int]]]`, `torch.Tensor`, *optional*): The labels for the points. input_boxes (`list[list[list[float]]]`, `torch.Tensor`, *optional*): The bounding boxes to add to the frame. original_sizes (`list[list[float]]`, `torch.Tensor`, *optional*): The original sizes of the images. Returns: A [`BatchEncoding`] with the following fields: - `pixel_values` (`torch.Tensor`): The processed image(s). - `original_sizes` (`list[list[float]]`): The original sizes of the images. - `labels` (`torch.Tensor`): The processed segmentation maps (if provided). - `input_points` (`torch.Tensor`): The processed points. - `input_labels` (`torch.Tensor`): The processed labels. - `input_boxes` (`torch.Tensor`): The processed bounding boxes. N)r!r&r%) tensor_typez0Either images or original_sizes must be providedz{original_sizes must be of length 1 or len(images). If you are passing a single image, you must pass a single original_size.pointsz;[image level, object level, point level, point coordinates])expected_depth input_nameexpected_formatexpected_coord_sizerlabelsz([image level, object level, point level])r.r/r0boxesz)[image level, box level, box coordinates]zbInput points and labels have inconsistent dimensions. Please ensure they have the same dimensions.c3JK|]}t|dkVdSr*N)len).0 img_boxesboxes_max_dimss r z)Sam2Processor.__call__..s3[[is9~~q(99[[[[[[rzInput boxes have inconsistent dimensions that would require padding, but boxes cannot be padded due to model limitations. Please ensure all images have the same number of boxes.)dtypeT)preserve_paddingr"r#is_bounding_boxr$)r isinstancerTensorcputolistr ValueErrorr6_validate_single_input_get_nested_dimensionsany_pad_nested_listtensorfloat32_normalize_tensor_coordinatesupdateint64)rr r!r"r#r$r%r&rencoding_image_processorprocessed_pointsprocessed_labelsprocessed_boxespoints_max_dimslabels_max_dims padded_points final_points padded_labels final_labels final_boxesr9s @r__call__zSam2Processor.__call__5sB  ';t';("3-(( (( $ $  '.%,77 ?!/!3!3!5!5!!>'46F5Wes't't't $ $OPP P22BC  #n"5"5":":s>?R?RVYZ`VaVa?a?aN   #|'?;CZ#:: # ]$% ;    $:: # J ;    #99 " K$% :O +"&"="=>N"O"OPRQRPR"S+"&"="=>N"O"OPRQRPR"S*!%!old_hold_wnew_hnew_ws r_normalize_coordinatesz$Sam2Processor._normalize_coordinatess % u"Ku&!!''))  .^^B1--F55=9v55=9v  +^^B**F rrc|dSt|tjrWdz kst|jdkr&|Sfd|DSt|tjrEdz kst|jdkr|Sfd|DSt|trkr|Sfd|DSt|ttfr|Stdt|)aS Recursively convert various input formats (tensors, numpy arrays, lists) to nested lists. Args: data: Input data in any format expected_depth: Expected nesting depth current_depth: Current depth in recursion Returns: Nested list representation of the data Nr-cDg|]}|dzSr*_convert_to_nested_listr7item current_depthr.rs r z9Sam2Processor._convert_to_nested_list..2ooobf44T>=[\K\]]ooorcDg|]}|dzSrfrgris rrlz9Sam2Processor._convert_to_nested_list..rmrcDg|]}|dzSrfrgris rrlz9Sam2Processor._convert_to_nested_list..rmrzUnsupported data type: )r?rr@r6shapenumpyrBnpndarraylistintr\ TypeErrortype)rdatar.rks` ``rrhz%Sam2Processor._convert_to_nested_listsk <4 dEL ) ) D 222c$*oo6J6Jzz||**,,,oooooojnoooo bj ) ) D 222c$*oo6J6J{{}}$oooooojnoooo d # # D.. pooooojnoooo sEl + + DKBd4jjBBCC Crc8|g}t|ts|St|dkr#|t|n&t |dt||d<t|dkr|D]}t|tru||}t |D]P\}}|dzt|kr||1t ||dz|||dz<Q|S)a` Get the maximum dimensions at each level of nesting. Args: nested_list (`list`): Nested list structure. max_dims (`list`, *optional*): Current maximum dimensions (for recursion). Returns: `list`: A list of maximum dimensions for each nesting level. Nrr*)r?rtr6appendmaxrE enumerate)r nested_listmax_dimsrjsub_dimsidims rrEz$Sam2Processor._get_nested_dimensionss+  H+t,, O x==A   OOC ,, - - - -hqk3{+;+;<.Gs!#b#b#b1HX$6$6#b#b#brc.g|]}tSrrrs rrlz2Sam2Processor._pad_nested_list..Ls!#S#S#S1HX$6$6#S#S#Sr)rr6r?rtextend_create_empty_nested_structurerangerG) rr} target_dims current_level pad_value current_sizer template_dimsrrs @rrGzSam2Processor._pad_nested_lists "  ,I C ,, , , +t,, (&-K;'' !-0  C ,,q0 0 0    {kL.HI J J J Ja 3{#3#3a#777$/ 0A0C0C$DM#BB=R[\\HH!*{[9J-KKH""#b#b#b#bkT`F`@a@a#b#b#bcccc!,MA,=,?,? @ >>}iXX""#S#S#S#Sk@R@R#S#S#STTT 3{++a/ / /3{++,, v vk!nd33v%)%:%:;q>;XehiXikt%u%uKNrctdkr gdzSfdtdDS)a Create an empty nested structure with given dimensions filled with pad_value. Args: dims (`list`): The dimensions of the nested structure. pad_value (`int`): The value to fill the structure with. r*rcLg|] }dd!Sr5)r)r7rdimsrrs rrlz@Sam2Processor._create_empty_nested_structure..cs2eeeQRD77QRR)LLeeer)r6r)rrrs```rrz,Sam2Processor._create_empty_nested_structureVsW t99>>;a( (eeeeeeV[\`ab\cVdVdeee erc t|tr3t|dkrdSd||dzSt|tjt jfrt|jSdS)z Get the nesting level of a list structure. Args: input_list (`list`): The list to get the nesting level of. rr*) r?rtr6_get_nesting_levelrrrsrr@rp)r input_lists rrz Sam2Processor._get_nesting_levelesz j$ ' ' ):!##qt..z!}=== =  RZ$> ? ? )z'(( (qrrxr.r/r0r1c |dSt|tjtjfry|j|kr!t d|d|d|d|jd |5|jd|kr$t d|d|d|jdd |||St|trM| |}||krt d|d |d |d|d |||SdS) a Validate a single input by ensuring proper nesting and raising an error if the input is not valid. Args: data (`torch.Tensor`, `np.ndarray`, or `list`): Input data to process. expected_depth (`int`): Expected nesting depth. input_name (`str`): Name of the input for error messages. expected_format (`str`): The expected format of the input. expected_coord_size (`int`, *optional*): Expected coordinate size (2 for points, 4 for boxes, None for labels). . NzInput z must be a tensor/array with z, dimensions. The expected nesting format is z. Got z dimensions.r[z as the last dimension, got .z must be a nested list with z( levels. The expected nesting format is z levels.) r?rr@rrrsndimrCrprhrtr)rrxr.r/r0r1rks rrDz$Sam2Processor._validate_single_inputvs0 <4 dU\2:6 7 7 FyN** qZqqnqqCRqqZ^Zcqqq%0:b>%888$MMMJ]MM{|FGI|JMMM//nEE E dD ! ! F 33D99M.. lZll^ll~MllUblll//nEE E  F Frc|r"||jk}|dd}tt|D]}||jdkr|t|kr||n|d}||j||||} |rA||} tj| ||| ||||<| ||<dS)a Helper method to normalize coordinates in a tensor across multiple images. Args: tensor (`torch.Tensor`): Input tensor with coordinates. original_sizes (`list`): Original image sizes. is_bounding_box (`bool`, *optional*, defaults to `False`): Whether coordinates are bounding boxes. preserve_padding (`bool`, *optional*, defaults to `False`): Whether to preserve padding values (for points). r[T)rkeepdimrr=N) rallrr6rprcrrwhere expand_as) rrHr%r>r<mask coord_maskimg_idxr^normalized_coordsimg_masks rrJz+Sam2Processor._normalize_tensor_coordinatess  8T11Db$77JS0011 8 8Ga((;BSEXEX;X;Xw 7 7^lmn^o $($?$?$fWo}Ve%@%%!$8)'2H&+k **6'?;;=NPVW^P_''F7OO'8F7O 8 8rTc 4|jj|||||||fi|S)a- Remove padding and upscale masks to the original image size. Args: masks (`Union[List[torch.Tensor], List[np.ndarray]]`): Batched masks from the mask_decoder in (batch_size, num_channels, height, width) format. original_sizes (`Union[torch.Tensor, List[Tuple[int,int]]]`): The original sizes of each image before it was resized to the model's expected input shape, in (height, width) format. mask_threshold (`float`, *optional*, defaults to 0.0): Threshold for binarization and post-processing operations. binarize (`bool`, *optional*, defaults to `True`): Whether to binarize the masks. max_hole_area (`float`, *optional*, defaults to 0.0): The maximum area of a hole to fill. max_sprinkle_area (`float`, *optional*, defaults to 0.0): The maximum area of a sprinkle to fill. apply_non_overlapping_constraints (`bool`, *optional*, defaults to `False`): Whether to apply non-overlapping constraints to the masks. Returns: (`torch.Tensor`): Batched masks in batch_size, num_channels, height, width) format, where (height, width) is given by original_size. )rpost_process_masks) rmasksr%mask_thresholdbinarize max_hole_areamax_sprinkle_area!apply_non_overlapping_constraintsrs rrz Sam2Processor.post_process_maskssCF7t#6       -      rc@|jj}t|dgzS)Nr%)rmodel_input_namesrt)rimage_processor_input_namess rrzSam2Processor.model_input_namess&&*&:&L#/3C2DDEEEr)Nr)NNNNNNN)F)r)N)rN)FF)rTrrF)__name__ __module__ __qualname__rurr rrtr\rr@strr rrXrcrhrErGrrrrrsrDrJrpropertyr __classcell__)rs@rrr#s k kS4Z kY\ k k k k k k%)/3LPDHEIBF26w(w(T!w(&,w(4T%[ 123elBTI w( 4S ?+el:TA w( $tE{+,u|;dB w(T%[)EL84?w(j(4/w( w(w(w(^w(tX](6 <$D$D$D$DL####J<<<<| f f f.+/ 0F0FlRZ'$.0F0F 0F  0F !4Z 0F 0F0F0F0Fd!8!8!8!8N*/, , , , \FFXFFFFFrr)__doc__copyrrqrr image_utilsrprocessing_utilsrtokenization_utils_baserutilsr r r r utils.import_utilsr get_loggerrloggerrr__all__rrrrs6%%%%%%......444444LLLLLLLLLLLL******  H % %LLL :WFWFWFWFWFNWFWFWFt  r