tj. ddlZddlmZddlmZddlmZddlmZm Z m Z m Z erddl m Z  dd ejd ejd ed ed ef dZGddeZGdde Z ddZdS)N)center_to_corners_format)is_scipy_available)HungarianMatcher ImageLoss _set_aux_lossgeneralized_box_ioulinear_sum_assignment?inputstargets num_boxesalphagammac |}tj||d}||zd|z d|z zz}|d|z |zz}|dkr||zd|z d|z zz} | |z}||z S)aR Loss used in RetinaNet for dense detection: https://huggingface.co/papers/1708.02002. Args: inputs (`torch.FloatTensor` of arbitrary shape): The predictions for each example. targets (`torch.FloatTensor` with the same shape as `inputs`) A tensor storing the binary classification label for each element in the `inputs` (0 for the negative class and 1 for the positive class). num_boxes (`int`): The total number of boxes in the batch. alpha (`float`, *optional*, defaults to 0.25): Optional weighting factor in the range (0,1) to balance positive vs. negative examples. gamma (`int`, *optional*, defaults to 2): Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Returns: Loss tensor none) reductionrr)sigmoidnn functional binary_cross_entropy_with_logitssum) rrrrrprobce_lossp_tlossalpha_ts c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/loss/loss_grounding_dino.pysigmoid_focal_lossr!s4 >>  Dm<eZdZejdZdS)GroundingDinoHungarianMatcherc|djdd\}}|ddd}|ddd}|d}tjdt ||D}||d d z }tjd |D}d } d} d| z || zzd|z dz z} | d|z | zz|dz z} | | z |z} tj ||d}tt|t| }|j |z|j | zz|j|zz}|||d }d|D}dt#||d D}d|DS)a Args: outputs (`dict`): A dictionary that contains at least these entries: * "logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits * "pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates. * "label_maps": Tuple of tensors of dim [num_classes, hidden_dim]. targets (`list[dict]`): A list of targets (len(targets) = batch_size), where each target is a dict containing: * "class_labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels * "boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates. Returns: `list[Tuple]`: A list of size `batch_size`, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) logitsNrrr pred_boxes label_mapsc0g|]\}}||dS class_labels).0 label_maptargets r z9GroundingDinoHungarianMatcher.forward..bs(tttFWiQW &*@ Atttr"T)dimkeepdimcg|] }|d Sboxesr,r-vs r r0z9GroundingDinoHungarianMatcher.forward..gs = = =7 = = =r"r @g:0yE>)pc8g|]}t|dSr5lenr7s r r0z9GroundingDinoHungarianMatcher.forward..{s"222QQwZ222r"c>g|]\}}t||Sr,r )r-ics r r0z9GroundingDinoHungarianMatcher.forward..|s)ccc41a(1..cccr"cg|]E\}}tj|tjtj|tjfFS))dtype)torch as_tensorint64)r-r?js r r0z9GroundingDinoHungarianMatcher.forward..}sHssscgcdfg%+666QVQ\8]8]8]^sssr")shapeflattenrrCcatziprlogtcdistr r bbox_cost class_cost giou_costviewcpu enumeratesplit)selfoutputsr batch_size num_queriesout_probout_bboxr( target_bboxrrneg_cost_classpos_cost_classrOrNrP cost_matrixsizesindicess r forwardz%GroundingDinoHungarianMatcher.forwardDs/,#*("3"9"1""= K8$,,Q22::<<<(00A66\* Ytt[^_ikr[s[stttuu *..R."F"FF i = =W = = =>> e)%8a(lT>Q=V=V=X=XVW$~5G K+;;; ))A()K)KMefqMrMrsss ny04?Z3OORVR`clRll !&&z;CCGGII 22'222cc9[EVEVW\^`EaEa;b;bcccsskrssssr"N)__name__ __module__ __qualname__rCno_gradrar,r"r r$r$Cs:U]__8t8t_8t8t8tr"r$cTeZdZdZdZejdZdZdZ dS)GroundingDinoImageLossa This class computes the losses for `GroundingDinoForObjectDetection`. The process happens in two steps: 1) we compute hungarian assignment between ground truth boxes and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and box). Args: matcher (`GroundingDinoHungarianMatcher`): Module able to compute a matching between targets and proposals. focal_alpha (`float`): Alpha parameter in focal loss. losses (`list[str]`): List of all the losses to be applied. See `get_loss` for a list of all available losses. cntj|||_||_||_dSN)rModule__init__matcher focal_alphalosses)rUrlrmrns r rkzGroundingDinoImageLoss.__init__s2 4    & r"cz|d}|j}tjd|D|}|djdkd}tj | | } d| i} | S)z Compute the cardinality error, i.e. the absolute error in the number of predicted non-empty boxes. This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients. r&c8g|]}t|dSr*r<r7s r r0z;GroundingDinoImageLoss.loss_cardinality..s%)R)R)RQ#a.?*@*@)R)R)Rr")devicer1g?rcardinality_error) rqrCrDrmaxvaluesrrrl1_lossfloat) rUrVrr`rr&rqtarget_lengths card_predcard_errrns r loss_cardinalityz'GroundingDinoImageLoss.loss_cardinalitys")R)R')R)R)R[abbb^^%%))"--4s:??BB =(():):N Create one_hot based on the matching indices r&cg|]I\}\}\}}|dkr*|d|td|zn |d|JS)rr+r(r<)r-r?r/_JrVs r r0zFGroundingDinoImageLoss._get_target_classes_one_hot..st   'A'ANOQRUU~&q)C 0Ea0H,I,IIIX^_mXnopXq   r"r(r)r2)rqrB) rCrIrSrJ_get_source_permutation_idx zeros_likerqlongto) rUrVrr`r&r+r(idxtarget_classes_onehots ` r _get_target_classes_one_hotz2GroundingDinoImageLoss._get_target_classes_one_hots"y    +4S'5J5J+K+K     Yw|4!<<< ..w77 % 0 UZU_ ` ` `%/ %=%@%@%L%Lc"$$r"cVd|vrtdd|vrtd||||}|d}|d}tj||}tj||}|}t ||||jd}d|i} | S)z Classification loss (Binary focal loss) targets dicts must contain the key "class_labels" containing a tensor of dim [nb_target_boxes] r&z#No logits were found in the outputs text_maskz&No text_mask were found in the outputsr)rrrrrloss_ce)KeyErrorrrC masked_selectrvr!rm) rUrVrr`rr source_logitsrrrns r loss_labelsz"GroundingDinoImageLoss.loss_labelss 7 " "@AA A g % %CDD D $ @ @'SZ [ [) K( +M9EE % 34I9 U U 5 ; ; = =$ )"    W% r"N) rbrcrd__doc__rkrCrerzrrr,r"r rgrgsn   U]__  _ %%%(r"rgc t|j|j|j} gd} t | |j| } | |i}||d<||d<||d<||d<d}|jr$t||}|D] }||d<||d< ||d<| |||j rF| | ||d }| ||}d | D} |d |j |j d |j r3d  D} ||jrdi}t|jdz D]5| fd D6 |t!fdD}||fS)N)rOrNrP)labelsr6 cardinality)rlrmrnr&r'r(rauxiliary_outputs)r&r'r(rc i|] \}}|dz| S_encr,r-kr8s r z7GroundingDinoForObjectDetectionLoss.. s"QQQtq!QZQQQr"r9)r loss_bbox loss_giouc i|] \}}|dz| Srr,rs r rz7GroundingDinoForObjectDetectionLoss..s"IIITQ1v:qIIIr"rc(i|]\}}|dz|S)r}r,)r-rr8r?s r rz7GroundingDinoForObjectDetectionLoss..s)#S#S#Stq!AAK#S#S#Sr"c3BK|]}|v||zVdSrir,)r-r loss_dict weight_dicts r z6GroundingDinoForObjectDetectionLoss..s:TT1 CSCSy|k!n,CSCSCSCSTTr")r$rOrNrPrgrmrauxiliary_lossr two_stageitemsupdatebbox_loss_coefficientgiou_loss_coefficientrangedecoder_layersr)r&rrqr'configr(r outputs_class outputs_coordencoder_logitsencoder_pred_boxesrlrn criterion outputs_lossr aux_outputencoder_outputs_lossencoder_loss_dictenc_weight_dictaux_weight_dictrr?rrs @@@r #GroundingDinoForObjectDetectionLossrsx,$0@FL\G0 / /F&&I LLL#L!+L!+L )L >)-GG+ 0 0J'1J| $&/J{ # #,= () ,//I  ,$,$"    &I&:FCCQQ7H7N7N7P7PQQQ*+++11K ,II[5F5F5H5HIII?+++ ,v,q011 U UA  " "#S#S#S#S{?P?P?R?R#S#S#S T T T T?+++ TTTTTiTTT T TD - --r")r r)NNNN)rCtorch.nnrimage_transformsrutilsrloss_for_object_detectionrrr r scipy.optimizer Tensorintrvr!r$rgrr,r"r rs| 777777&&&&&&ffffffffffff5444444 $"$" L$" \$"$"  $"  $"$"$"$"N:t:t:t:t:t$4:t:t:tzVVVVVYVVVBF.F.F.F.F.F.r"