tjmp*dZddlmZddlZddlmZddlmZddl m Z m Z m Z ddl mZmZmZmZdd lmZejeZerddlZd Zd Zd Zd ZdZdZdZedeGddeZdgZ dS)z Processor class for SAM3. deepcopyN) ImageInput)ProcessorMixin) BatchEncodingPreTokenizedInput TextInput) TensorTypeauto_docstringis_torch_availablelogging)requiresc|d\}}}}|d|zz |d|zz |d|zz|d|zzg}tj|dSN?dimunbindtorchstackxx_cy_cwhbs f/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/sam3/processing_sam3.pybox_cxcywh_to_xyxyr"#s\XXb\\NCa a-3q=C#'MS37]LA ;qb ! ! !!c|d\}}}}|d|zz |d|zz ||g}tj|dSrrrs r!box_cxcywh_to_xywhr%)sLXXb\\NCa a-3q=A4A ;qb ! ! !!r#cz|d\}}}}||||z||zg}tj|dSNrrrryrrr s r!box_xywh_to_xyxyr*/D"JAq!Q qAEQU$A ;qb ! ! !!r#c|d\}}}}|d|zz|d|zz||g}tj|dSrrr(s r!box_xywh_to_cxcywhr-5sL"JAq!Q cAg+S1WQ0A ;qb ! ! !!r#cz|d\}}}}||||z ||z g}tj|dSr'r)rr)XYr s r!box_xyxy_to_xywhr1;r+r#c|d\}}}}||zdz ||zdz ||z ||z g}tj|dS)Nrrr)rx0y0x1y1r s r!box_xyxy_to_cxcywhr8AsTXXb\\NBB r'QbA R27>v>>>.*5*A;;tG[G`aiGjr#imagestextsegmentation_maps input_boxesinput_boxes_labelsoriginal_sizesreturn_tensorsreturnc d} ||j|f||d|} ng|Tt|tjr&|}t d|i|} n|td|||}|3| ||dd} | | | n| } |u| d}| |d d d d } | |ddd} | | | | } | | | dd} | | | dd}| | | |krtd| y|| | d gz}tj|tj}|||ddt%|}| d|i| M|| |}tj|tj}| d|i| S)a} images (`ImageInput`, *optional*): The image(s) to process. text (`str`, `list[str]`, `list[list[str]]`, *optional*): The text to process. segmentation_maps (`ImageInput`, *optional*): The segmentation maps to process. input_boxes (`list[list[list[float]]]`, `torch.Tensor`, *optional*): The bounding boxes to process. input_boxes_labels (`list[list[int]]`, `torch.Tensor`, *optional*): The labels for the bounding boxes. original_sizes (`list[list[float]]`, `torch.Tensor`, *optional*): The original sizes of the images. Returns: A [`BatchEncoding`] with the following fields: - `pixel_values` (`torch.Tensor`): The processed image(s). - `original_sizes` (`list[list[float]]`): The original sizes of the images. - `labels` (`torch.Tensor`): The processed segmentation maps (if provided). - `input_boxes_labels` (`torch.Tensor`): The processed labels for the bounding boxes. - `input_boxes` (`torch.Tensor`): The processed bounding boxes. N)rNrRrQ) tensor_typezKEither images or original_sizes must be provided if input_boxes is not None max_length )rRpaddingrVrr:z)[image level, box level, box coordinates])expected_depth input_nameexpected_formatexpected_coord_sizer3labelsz[image level, box level])rZr[r\zaInput boxes and labels have inconsistent dimensions. Please ensure they have the same dimensions.)dtypeT)is_bounding_boxpreserve_paddingrOrP)rF isinstancerTensorcputolistr ValueError_resolve_text_promptsrIupdate_validate_single_input_generate_default_box_labels_get_nested_dimensions_pad_nested_listtensorfloat32_normalize_tensor_coordinatesr8int64)rHrLrMrNrOrPrQrRrJencoding text_inputsprocessed_boxesprocessed_boxes_labelsboxes_max_dimsboxes_labels_max_dims padded_boxes final_boxespadded_boxes_labelsfinal_boxes_labelss r!__call__zSam3Processor.__call__esD  +t+"3- HH  '.%,77 ?!/!3!3!5!5!!>$&6%GUcdddHH  $jkk k))$ <<  ..nVboq.rrK# ,,,,&  "%&67N"99 " K$% :O&*%@%@" # : &A&& "*/E/M)-)J)J?)[)[&*!%!z>Sam3Processor._generate_default_box_labels..s0kkkK +!s;7G7G1Gkkkr#)rHrss r!rjz*Sam3Processor._generate_default_box_labelsskk[jkkkkr#rc|dSt|tjrWdz kst|jdkr&|Sfd|DSt|tjrEdz kst|jdkr|Sfd|DSt|trkr|Sfd|DSt|ttfr|Stdt|)a Recursively convert various input formats (tensors, numpy arrays, lists) to nested lists. Preserves None values within lists. Args: data: Input data in any format (may be None or contain None values) expected_depth: Expected nesting depth current_depth: Current depth in recursion Returns: Nested list representation of the data (or None) Nr3cDg|]}|dzSr_convert_to_nested_listritem current_depthrZrHs r!rz9Sam3Processor._convert_to_nested_list..2ooobf44T>=[\K\]]ooor#cDg|]}|dzSrrrs r!rz9Sam3Processor._convert_to_nested_list.. rr#cLg|] }||dznd!Srrrs r!rz9Sam3Processor._convert_to_nested_list..sN^b]mD00~}WXGXYYYswr#zUnsupported data type: )rbrrcrshapenumpyrenpndarraylistintrrftype)rHdatarZrs` ``r!rz%Sam3Processor._convert_to_nested_lists~ <4 dEL ) ) E 222c$*oo6J6Jzz||**,,,oooooojnoooo bj ) ) E 222c$*oo6J6J{{}}$oooooojnoooo d # # E..  $sEl + + EKCtDzzCCDD Dr#c`||rdndSt|ttfs|St|}|rPt|t|kr0t dt|dt|dt |D]\}}||r ||d||<|S)zQ Resolve text prompts by setting defaults based on prompt types. NvisualzEThe number of text prompts must match the number of input boxes. Got z text prompts and z input boxes.)rbrtuplerrf enumerate)rHrMrOi text_values r!rgz#Sam3Processor._resolve_text_promptss <*488 4$u .. KDzz  3t99K(8(888T4yyTT47 4D4DTTT  't__ # #MAz!k!k!n6P"Q r#c>|g}t|ts|St|dkr#|t|n&t |dt||d<t|dkr|D]}|t|tru||}t |D]P\}}|dzt|kr||1t ||dz|||dz<Q|S)a Get the maximum dimensions at each level of nesting, skipping None values. Args: nested_list (`list`): Nested list structure (may contain None values). max_dims (`list`, *optional*): Current maximum dimensions (for recursion). Returns: `list`: A list of maximum dimensions for each nesting level. Nrr)rbrrappendmaxrkr)rH nested_listmax_dimsrsub_dimsrrs r!rkz$Sam3Processor._get_nested_dimensions6s3  H+t,, O x==A   OOC ,, - - - -hqk3{+;+;<.s!#b#b#b1HX$6$6#b#b#br#c.g|]}tSrrrs r!rz2Sam3Processor._pad_nested_list..s!#S#S#S1HX$6$6#S#S#Sr#)rArrbrextend_create_empty_nested_structurerangerl) rHr target_dims current_level pad_value current_sizer@ template_dimsrrs @r!rlzSam3Processor._pad_nested_list^s["  ,I C ,, , , +t,, (&-K;'' !-0  C ,,q0 0 0    {kL.HI J J J Ja 3{#3#3a#777$/ 0A0C0C$DM#BB=R[\\HH!*{[9J-KKH""#b#b#b#bkT`F`@a@a#b#b#bcccc!,MA,=,?,? @ >>}iXX""#S#S#S#Sk@R@R#S#S#STTT 3{++a/ / /3{++,, v vq>)$/ 0A0C0C$DM%)%H%HXa%b%bKNN A55v%)%:%:;q>;XehiXikt%u%uKNr#ctdkr gdzSfdtdDS)a Create an empty nested structure with given dimensions filled with pad_value. Args: dims (`list`): The dimensions of the nested structure. pad_value (`int`): The value to fill the structure with. rrcLg|] }dd!S)rN)r)rrdimsrrHs r!rz@Sam3Processor._create_empty_nested_structure..s2eeeQRD77QRR)LLeeer#)rr)rHrrs```r!rz,Sam3Processor._create_empty_nested_structuresW t99>>;a( (eeeeeeV[\`ab\cVdVdeee er#ct|tr8t|dkrdS|D]}|d||zcSdSt|tjt jfrt|jSdS)z Get the nesting level of a list structure, skipping None values. Args: input_list (`list`): The list to get the nesting level of. rr) rbrr_get_nesting_levelrrrrcr)rH input_listrs r!rz Sam3Processor._get_nesting_levels j$ ' ' ):!##q" = =#t66t<<<<<<$1  RZ$> ? ? )z'(( (qr#rrZr[r\r]c |dSt|tjtjfry|j|kr!t d|d|d|d|jd |5|jd|kr$t d|d|d|jdd |||St|trM| |}||krt d|d |d |d|d |||SdS) a Validate a single input by ensuring proper nesting and raising an error if the input is not valid. Args: data (`torch.Tensor`, `np.ndarray`, or `list`): Input data to process. expected_depth (`int`): Expected nesting depth. input_name (`str`): Name of the input for error messages. expected_format (`str`): The expected format of the input. expected_coord_size (`int`, *optional*): Expected coordinate size (4 for boxes, None for labels). . NzInput z must be a tensor/array with z, dimensions. The expected nesting format is z. Got z dimensions.rz as the last dimension, got .z must be a nested list with z( levels. The expected nesting format is z levels.) rbrrcrrndimrfrrrr)rHrrZr[r\r]rs r!riz$Sam3Processor._validate_single_inputs0 <4 dU\2:6 7 7 FyN** qZqqnqqCRqqZ^Zcqqq%0:b>%888$MMMJ]MM{|FGI|JMMM//nEE E dD ! ! F 33D99M.. lZll^ll~MllUblll//nEE E  F Fr#c|r"||jk}|dd}tt|D]}||jdkr|t|kr||n|d}|||||} |rA||} t j| ||| ||||<| ||<dS)a Helper method to normalize coordinates in a tensor across multiple images. Args: tensor (`torch.Tensor`): Input tensor with coordinates. original_sizes (`list`): Original image sizes. is_bounding_box (`bool`, *optional*, defaults to `False`): Whether coordinates are bounding boxes. preserve_padding (`bool`, *optional*, defaults to `False`): Whether to preserve padding values (for boxes). rT)rkeepdimr)r`N) rAallrrrrrwhere expand_as) rHrmrQr`ramask coord_maskimg_idxrnormalized_coordsimg_masks r!roz+Sam3Processor._normalize_tensor_coordinatess  8T11Db$77JS0011 8 8Ga((;BSEXEX;X;Xw 7 7^lmn^o $($?$?7O]O%@%%!$8)'2H&+k **6'?;;=NPVW^P_''F7OO'8F7O 8 8r#rc>|j||||S)a6 Converts the output of [`Sam3Model`] into semantic segmentation maps. Args: outputs ([`Sam3ImageSegmentationOutput`]): Raw outputs of the model containing semantic_seg. target_sizes (`list[tuple]` of length `batch_size`, *optional*): List of tuples corresponding to the requested final size (height, width) of each prediction. If unset, predictions will not be resized. threshold (`float`, *optional*, defaults to 0.5): Threshold for binarizing the semantic segmentation masks. return_segmentation_scores (`bool`, *optional*, defaults to `False`): Whether to return segmentation scores alongside the segmentation map. Returns: semantic_segmentation: `list[torch.Tensor]` of length `batch_size`, where each item is a semantic segmentation map of shape (height, width) corresponding to the target_sizes entry (if `target_sizes` is specified). Each entry is a binary mask (0 or 1). ) target_sizes thresholdreturn_segmentation_scores)rF"post_process_semantic_segmentation)rHoutputsrrrs r!rz0Sam3Processor.post_process_semantic_segmentations1,#FF %'A G   r#333333?c:|j|||S)a Converts the raw output of [`Sam3Model`] into final bounding boxes in (top_left_x, top_left_y, bottom_right_x, bottom_right_y) format. This is a convenience wrapper around the image processor method. Args: outputs ([`Sam3ImageSegmentationOutput`]): Raw outputs of the model containing pred_boxes, pred_logits, and optionally presence_logits. threshold (`float`, *optional*, defaults to 0.3): Score threshold to keep object detection predictions. target_sizes (`list[tuple[int, int]]`, *optional*): List of tuples (`tuple[int, int]`) containing the target size `(height, width)` of each image in the batch. If unset, predictions will not be resized. Returns: `list[dict]`: A list of dictionaries, each dictionary containing the following keys: - **scores** (`torch.Tensor`): The confidence scores for each predicted box on the image. - **boxes** (`torch.Tensor`): Image bounding boxes in (top_left_x, top_left_y, bottom_right_x, bottom_right_y) format. Example: ```python >>> from transformers import AutoModel, AutoProcessor >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> model = AutoModel.from_pretrained("facebook/sam3-base") >>> processor = AutoProcessor.from_pretrained("facebook/sam3-base") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(images=image, text="cat", return_tensors="pt") >>> outputs = model(**inputs) >>> # Post-process to get bounding boxes >>> results = processor.post_process_object_detection(outputs, threshold=0.3, target_sizes=[image.size[::-1]]) >>> boxes = results[0]["boxes"] >>> scores = results[0]["scores"] ``` )rFpost_process_object_detection)rHrrrs r!rz+Sam3Processor.post_process_object_detection7s!V#AA'9Vbcccr#c<|j||||S)ay Converts the raw output of [`Sam3Model`] into instance segmentation predictions with bounding boxes and masks. This is a convenience wrapper around the image processor method. Args: outputs ([`Sam3ImageSegmentationOutput`]): Raw outputs of the model containing pred_boxes, pred_logits, pred_masks, and optionally presence_logits. threshold (`float`, *optional*, defaults to 0.3): Score threshold to keep instance predictions. mask_threshold (`float`, *optional*, defaults to 0.5): Threshold for binarizing the predicted masks. target_sizes (`list[tuple[int, int]]`, *optional*): List of tuples (`tuple[int, int]`) containing the target size `(height, width)` of each image in the batch. If unset, predictions will not be resized. Returns: `list[dict]`: A list of dictionaries, each dictionary containing the following keys: - **scores** (`torch.Tensor`): The confidence scores for each predicted instance on the image. - **boxes** (`torch.Tensor`): Image bounding boxes in (top_left_x, top_left_y, bottom_right_x, bottom_right_y) format. - **masks** (`torch.Tensor`): Binary segmentation masks for each instance, shape (num_instances, height, width). Example: ```python >>> from transformers import AutoModel, AutoProcessor >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> model = AutoModel.from_pretrained("facebook/sam3-base") >>> processor = AutoProcessor.from_pretrained("facebook/sam3-base") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(images=image, text="cat", return_tensors="pt") >>> outputs = model(**inputs) >>> # Post-process to get instance segmentation >>> results = processor.post_process_instance_segmentation( ... outputs, threshold=0.3, target_sizes=[image.size[::-1]] ... ) >>> masks = results[0]["masks"] >>> boxes = results[0]["boxes"] >>> scores = results[0]["scores"] ``` )rF"post_process_instance_segmentation)rHrrmask_thresholdrs r!rz0Sam3Processor.post_process_instance_segmentationds)r#FF Y    r#)Nr?)NNNNNNN)F)r|r}rSr})r)N)rN)FF)NrF)rN)rrN)!__name__ __module__ __qualname__rrEr rr r rrrrcstrr rr{rrjrrgrkrlrrrrrirorrr __classcell__)rKs@r!r>r>Usbe k k7:Tz k[^ k k k k k k%)ae/3EIJNBF26jjT!j++d9o=EV@WWZ^^j&, j $tE{+,u|;dB j !d3i1EL@4G jT%[)EL84?jj(4/j jjj^jX6lll(E(E(E(ET6&&&&P@@@@D f f f8+/ 0F0FlRZ'$.0F0F 0F  0F !4Z 0F 0F0F0F0Fd!8!8!8!8HUZ    :+d+d+d+d` ; ; ; ; ; ; ; ; r#r>)!__doc__copyrrr image_utilsrprocessing_utilsrtokenization_utils_baserr r utilsr r r rutils.import_utilsr get_loggerrloggerrr"r%r*r-r1r8r;r>__all__rr#r!rs%%%%%%......RRRRRRRRRRLLLLLLLLLLLL******  H % %LLL""" """ """ """ """ """ ! ! ! :H  H  H  H  H  NH  H  H  V  r#