tj5ddlmZddlmZddlmZddlZddlZddlm Z ddl m Z ddlm Z ddl mZd d lmZmZmZmZejeZd dZd!dZd"dZd!dZd"dZd!dZd"dZd"dZd"dZd"dZ d#dZ!GddeZ"dS)$) annotations)Callable)EnumN)ndarray)pairwise_distances)Tensor)logging)_convert_to_batch_tensor_convert_to_tensornormalize_embeddings to_scipy_cooarbreturntuple[Tensor, Tensor]c|jr|js||fS|jr|js||fS||fS)aW Converts the dense tensor to sparse COO if exactly one of the two inputs is sparse, so that mixed inputs can reuse the all-sparse computations. The lone dense operand in such a mix typically still holds mostly zeros (e.g. sparse embeddings encoded with ``convert_to_sparse_tensor=False``), making the conversion cheap, whereas densifying the sparse operand would allocate a second full-size dense tensor. Args: a (Tensor): The first tensor. b (Tensor): The second tensor. Returns: tuple[Tensor, Tensor]: The two tensors, using the same layout. ) is_sparse to_sparserrs c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/sentence_transformers/util/similarity.py_match_layoutsrsY { 1; !++--{ 1; {{}}a a4Kc"t||S) Computes the cosine similarity between two tensors. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = cos_sim(a[i], b[j]) )cos_simrs rpytorch_cos_simr(s 1a==rlist | np.ndarray | Tensorct|}t|}t|}t|}tj||ddS)rrr )r r torchmm transposeto_denserra_normb_norms rrr6sf !##A ##A !! $ $F !! $ $F 8FF,,Q22 3 3 < < > >>rcdt|}t|}|js|jrIt|}t|}||zdSt t|t|S)a Computes the pairwise cosine similarity cos_sim(a[i], b[i]). Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = cos_sim(a[i], b[i]) dim)r rr sumr#pairwise_dot_scorer$s rpairwise_cos_simr-Is 1A1A {_ak_%a((%a(($$$,,55777!"6q"9"9;OPQ;R;RSS\\^^^rct|}t|}tj||ddS)a Computes the dot-product dot_prod(a[i], b[j]) for all i and j. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = dot_prod(a[i], b[j]) rr )r r r!r"r#rs r dot_scorer/`sJ !##A ##A 8Aq{{1a(( ) ) 2 2 4 44rct|}t|}||zdS)a Computes the pairwise dot-product dot_prod(a[i], b[i]). Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = dot_prod(a[i], b[i]) r(r))r r+r#rs rr,r,qsB 1A1A E;;2;   ' ' ) ))rc t|}t|}|js|jrtdt ||\}}t |}t |}t ||d}tj|  |j  Stj ||d S)a Computes the manhattan similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = -manhattan_distance(a[i], b[j]) z8Using scipy for sparse Manhattan similarity computation. manhattan)metricg?p)r rlogger warning_oncerrrr from_numpyfloattodevicer#cdist)rra_coob_coodists r manhattan_simr@s !##A ##A{ 4ak 4VWWWa##1QQ!%{CCC&&,,..11!(;;DDFFF AqC(((113333rct|}t|}t||\}}tjtj||z d S)a< Computes the manhattan similarity (i.e., negative distance) between pairs of tensors. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Vector with res[i] = -manhattan_distance(a[i], b[i]) r(r))r rr r+absr#rs rpairwise_manhattan_simrCsc 1A1A !Q  DAq IeiA&&B / / / 8 8 : : ::rct|}t|}|js|jr&t||\}}tj||zdd}tj||zdd}tj|| }|d|zz |z}tj |d}tj | Stj ||d S) a Computes the euclidean similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible. Args: a (Union[list, np.ndarray, Tensor]): The first tensor. b (Union[list, np.ndarray, Tensor]): The second tensor. Returns: Tensor: Matrix with res[i][j] = -euclidean_distance(a[i], b[j]) r r)rg)ming@r4) r rrr sparser+r# unsqueezematmultclampsqrtr<)rr a_norm_sq b_norm_sq dot_product squared_dists r euclidean_simrQs= !##A ##A{)ak)a##1L$$QU$22;;==GGJJ L$$QU$22;;==GGJJ l1accee,,5577 !1{?2Y> {>> similarity_fn = SimilarityFunction.to_similarity_fn("cosine") >>> similarity_scores = similarity_fn(embeddings1, embeddings2) >>> similarity_scores tensor([[0.3952, 0.0554], [0.0992, 0.1570]]) The provided function 4 is not supported. Use one of the supported values: .) rkCOSINEr DOT_PRODUCTr/ MANHATTANr@ EUCLIDEANrQ ValueErrorpossible_valuesros rto_similarity_fnz#SimilarityFunction.to_similarity_fns.11DEE "4"; ; ;N "4"@ @ @  "4"> > > "4"> > >  V%8 V VoAoQoQoSoS V V V   rc2t|}|tjkrtS|tjkrtS|tjkrt S|tjkrtStd|dt d)a Converts a similarity function into a pairwise similarity function. The pairwise similarity function returns the diagonal vector from the similarity matrix, i.e. it only computes the similarity(a[i], b[i]) for each i in the range of the input tensors, rather than computing the similarity between all pairs of a and b. Args: similarity_function (Union[str, SimilarityFunction]): The name or enum value of the similarity function. Returns: Callable[[Union[Tensor, ndarray], Union[Tensor, ndarray]], Tensor]: The pairwise similarity function. Raises: ValueError: If the provided similarity function is not supported. Example: >>> pairwise_fn = SimilarityFunction.to_similarity_pairwise_fn("cosine") >>> similarity_scores = pairwise_fn(embeddings1, embeddings2) >>> similarity_scores tensor([0.3952, 0.1570]) rsrtru) rkrvr-rwr,rxrCryrSrzr{r|s rto_similarity_pairwise_fnz,SimilarityFunction.to_similarity_pairwise_fnEs411DEE "4"; ; ;# # "4"@ @ @% % "4"> > >) ) "4"> > >) ) V%8 V VoAoQoQoSoS V V V   r list[str]c$dtDS)ad Returns a list of possible values for the SimilarityFunction enum. Returns: list: A list of possible values for the SimilarityFunction enum. Example: >>> possible_values = SimilarityFunction.possible_values() >>> possible_values ['cosine', 'dot', 'euclidean', 'manhattan'] cg|] }|j S)rY).0ms r z6SimilarityFunction.possible_values..{s444A444r)rkrrrr{z"SimilarityFunction.possible_valuesns54!34444rN)rorprrq)rr) __name__ __module__ __qualname____doc__rvrwDOTryrx staticmethodr}rr{rrrrkrksFK CII# # # \# J& & & \& P 5 5 5\ 5 5 5rrk)rrrrrr)rrrrrr)rrrrrr)rTrrUrrr)# __future__rcollections.abcrenumrnumpynpr rsklearn.metricsrrtransformers.utilsr tensorr r r r get_loggerrr6rrrr-r/r,r@rCrQrSrirkrrrrs""""""$$$$$$ ......&&&&&&dddddddddddd  H % %*    ????&____.5555"****"44448;;;;$))))@CCCC$*!*!*!*!Zl5l5l5l5l5l5l5l5l5l5r