Ch0SSKJr SSKJr SSKrSSKJrJr SSKJr SSK J r "SS5r "S S \R5r g) ) annotations)IterableN)Tensornn)SentenceTransformerpytorch_cos_simcD\rSrSrSr\SSj5r\SSSjj5rSrg) $BatchHardTripletLossDistanceFunction z`This class defines distance functions, that can be used with Batch[All/Hard/SemiHard]TripletLosscS[X5- $)zWCompute the 2D matrix of cosine distances (1-cosine_similarity) between all embeddings.r) embeddingss i/var/www/html/shao/venv/lib/python3.13/site-packages/sentence_transformers/losses/BatchHardTripletLoss.pycosine_distance4BatchHardTripletLossDistanceFunction.cosine_distances?::::cl[R"XR55n[R"U5nUR S5SU-- UR S5-nSXDS:'U(dBUR S5R 5nXES--nSU- [R"U5-nU$)at Compute the 2D matrix of eucledian distances between all the embeddings. Args: embeddings: tensor of shape (batch_size, embed_dim) squared: Boolean. If true, output is the pairwise squared euclidean distance matrix. If false, output is the pairwise euclidean distance matrix. Returns: pairwise_distances: tensor of shape (batch_size, batch_size) rg@rgؗҜ<?)torchmatmultdiag unsqueezeeqfloatsqrt)rsquared dot_product square_norm distancesmasks reucledian_distance7BatchHardTripletLossDistanceFunction.eucledian_distancesll:||~> jj-  ))!,s[/@@;CXCXYZC[[ $% a- <<?((*D!5L0Ituzz)'<[TU]5 XlX0lX lg)a> BatchHardTripletLoss takes a batch with (sentence, label) pairs and computes the loss for all possible, valid triplets, i.e., anchor and positive must have the same label, anchor and negative a different label. It then looks for the hardest positive and the hardest negatives. The labels must be integers, with same label indicating sentences from the same class. Your train dataset must contain at least 2 examples per label class. Args: model: SentenceTransformer model distance_metric: Function that returns a distance between two embeddings. The class SiameseDistanceMetric contains pre-defined metrics that can be used margin: Negative samples should be at least margin further apart from the anchor than the positive. Definitions: :Easy triplets: Triplets which have a loss of 0 because ``distance(anchor, positive) + margin < distance(anchor, negative)``. :Hard triplets: Triplets where the negative is closer to the anchor than the positive, i.e., ``distance(anchor, negative) < distance(anchor, positive)``. :Semi-hard triplets: Triplets where the negative is not closer to the anchor than the positive, but which still have a positive loss, i.e., ``distance(anchor, positive) < distance(anchor, negative) + margin``. References: * Source: https://github.com/NegatioN/OnlineMiningTripletLoss/blob/master/online_triplet_loss/losses.py * Paper: In Defense of the Triplet Loss for Person Re-Identification, https://arxiv.org/abs/1703.07737 * Blog post: https://omoindrot.github.io/triplet-loss Requirements: 1. Each sentence must be labeled with a class. 2. Your dataset must contain at least 2 examples per labels class. 3. Your dataset should contain hard positives and negatives. Inputs: +------------------+--------+ | Texts | Labels | +==================+========+ | single sentences | class | +------------------+--------+ Recommendations: - Use ``BatchSamplers.GROUP_BY_LABEL`` (:class:`docs `) to ensure that each batch contains 2+ examples per label class. Relations: * :class:`BatchAllTripletLoss` uses all possible, valid triplets, rather than only the hardest positive and negative samples. * :class:`BatchSemiHardTripletLoss` uses only semi-hard triplets, valid triplets, rather than only the hardest positive and negative samples. * :class:`BatchHardSoftMarginTripletLoss` does not require setting a margin, while this loss does. Example: :: from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses from datasets import Dataset model = SentenceTransformer("microsoft/mpnet-base") # E.g. 0: sports, 1: economy, 2: politics train_dataset = Dataset.from_dict({ "sentence": [ "He played a great game.", "The stock is up 20%", "They won 2-1.", "The last goal was amazing.", "They all voted against the bill.", ], "label": [0, 1, 0, 0, 2], }) loss = losses.BatchHardTripletLoss(model) trainer = SentenceTransformerTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() N)super__init__sentence_embeddertriplet_margindistance_metric)selfmodelr7margin __class__s rr4BatchHardTripletLoss.__init__;s#d !&$.rcRURUS5SnURX#5$)Nrsentence_embedding)r5batch_hard_triplet_loss)r8sentence_featureslabelsreps rforwardBatchHardTripletLoss.forwards/$$%6q%9:;OP++F88rcURU5n[RU5R5nXC-nUR SSS9upg[R U5R5nUR SSS9upX9SU- --n U R SSS9upXk- UR-n SXS:'U R5n U $)aBuild the triplet loss over a batch of embeddings. For each anchor, we get the hardest positive and hardest negative to form a triplet. Args: labels: labels of the batch, of size (batch_size,) embeddings: tensor of shape (batch_size, embed_dim) margin: margin for triplet loss squared: Boolean. If true, output is the pairwise squared euclidean distance matrix. If false, output is the pairwise euclidean distance matrix. Returns: Label_Sentence_Triplet: scalar tensor containing the triplet loss rT)keepdimrr) r7r/ get_anchor_positive_triplet_maskrmax get_anchor_negative_triplet_maskminr6mean)r8rAr pairwise_distmask_anchor_positiveanchor_positive_disthardest_positive_dist_mask_anchor_negativemax_anchor_negative_distanchor_negative_disthardest_negative_disttl triplet_losss rr?,BatchHardTripletLoss.batch_hard_triplet_losss,,Z8  4TTU[\bbd 4C$8#;#;At#;#L  4TTU[\bbd'4&7&74&7&H# ,3QeKe/ff$8#;#;At#;#L # :T=P=P P6 wwy rc[R"URS5URS9R 5nU)nUR S5nUR S5nUR S5nX4-U-nUR S5UR S5:HnUR S5nUR S5n U )U-n X-$)aReturn a 3D mask where mask[a, p, n] is True iff the triplet (a, p, n) is valid. A triplet (i, j, k) is valid if: - i, j, k are distinct - labels[i] == labels[j] and labels[i] != labels[k] Args: labels: tf.int32 `Tensor` with shape [batch_size] rdevicerreyesizerZboolr) rA indices_equalindices_not_equal i_not_equal_j i_not_equal_k j_not_equal_kdistinct_indices label_equal i_equal_j i_equal_k valid_labelss rget_triplet_mask%BatchHardTripletLoss.get_triplet_masks &++a.GLLN *N)33A6 )33A6 )33A6 )9]J&&q)V-=-=a-@@ ))!, ))!, !zI- ..rc[R"URS5URS9R 5nU)nUR S5UR S5:HnX2-$)zReturn a 2D mask where mask[a, p] is True iff a and p are distinct and have same label. Args: labels: tf.int32 `Tensor` with shape [batch_size] Returns: mask: tf.bool `Tensor` with shape [batch_size, batch_size] rrYrr\)rAr`ra labels_equals rrG5BatchHardTripletLoss.get_anchor_positive_triplet_masks\ &++a.GLLN *N''*f.>.>q.AA //rcJURS5URS5:H)$)zReturn a 2D mask where mask[a, n] is True iff a and n have distinct labels. Args: labels: tf.int32 `Tensor` with shape [batch_size] Returns: mask: tf.bool `Tensor` with shape [batch_size, batch_size] rr)r)rAs rrI5BatchHardTripletLoss.get_anchor_negative_triplet_masks(!!!$(8(8(;;<r}s4"$ I6++\P299Pr