jPddlmZddlZddlZddlmZmZddlmZGddZ y)) annotationsN)AnyOptional)DocumentceZdZUdZdZded< dZded< d dfd ZeddZ edd Z dd Z d dd Z d dd Z d dd Z d ddZxZS)RecursiveJsonSplitteraSplits JSON data into smaller, structured chunks while preserving hierarchy. This class provides methods to split JSON data into smaller dictionaries or JSON-formatted strings based on configurable maximum and minimum chunk sizes. It supports nested JSON structures, optionally converts lists into dictionaries for better chunking, and allows the creation of document objects for further use. intmax_chunk_sizeimin_chunk_sizeclt|||_|||_yt|dz d|_y)aInitialize the chunk size configuration for text processing. This constructor sets up the maximum and minimum chunk sizes, ensuring that the ``min_chunk_size`` defaults to a value slightly smaller than the ``max_chunk_size`` if not explicitly provided. Args: max_chunk_size: The maximum size for a chunk. Defaults to 2000. min_chunk_size: The minimum size for a chunk. If None, defaults to the maximum chunk size minus 200, with a lower bound of 50. N2)super__init__r maxr )selfr r __class__s g/var/www/html/Fitness-lenito-AI-main/venv/lib/python3.12/site-packages/langchain_text_splitters/json.pyrzRecursiveJsonSplitter.__init__sF ,)  ^c)2. c>ttj|S)z1Calculate the size of the serialized JSON object.)lenjsondumps)datas r _json_sizez RecursiveJsonSplitter._json_size/s4::d#$$rcL|ddD]}|j|i}|||d<y)z;Set a value in a nested dictionary based on the given path.N) setdefault)dpathvaluekeys r_set_nested_dictz&RecursiveJsonSplitter._set_nested_dict4s19C S"%A$r( rc2t|tr3|jDcic]\}}||j|c}}St|tr7t |Dcic] \}}t ||j|"c}}S|Scc}}wcc}}w)N) isinstancedictitems_list_to_dict_preprocessinglist enumeratestr)rrkviitems rr)z1RecursiveJsonSplitter._list_to_dict_preprocessing;s dD !GKzz|T|tq!At77::|T T dD ! ).GAtA88>>.   Us B "%Bc|xsg}||nig}t|tr|jD]\}}g||}|j|d}|j||i}|j|z } || kr|j |d||^||j k\r|ji|j||||S|j |d|||S)zESplit json into maximum size dictionaries while preserving structure.r) r&r'r(rr r$r append _json_split) rr current_pathchunksr#r"new_path chunk_sizesize remainings rr3z!RecursiveJsonSplitter._json_splitHs$)r !-B4 dD !"jjl U/\/3/!__VBZ8 U|4 //*< )#))&*hF!T%8%88 b)$$UHf=+&   ! !&*lD A rc|r!|j|j|}n|j|}|ds|j|S)z'Splits JSON into a list of JSON chunks.r)r3r)pop)r json_data convert_listsr5s r split_jsonz RecursiveJsonSplitter.split_jsongsI %%d&F&Fy&QRF%%i0Fbz JJL rc||j||}|Dcgc]}tj||c}Scc}w)z2Splits JSON into a list of JSON formatted strings.)r<r=) ensure_ascii)r>rr)rr<r=r@r5chunks r split_textz RecursiveJsonSplitter.split_textws=9MRKQQ& 5|<&QQQs9c|xsigt|z}g}t|D]T\}}|j|||D]8} tj||} t | | } |j | :V|S)z4Create documents from a list of json objects (Dict).)r<r=r@) page_contentmetadata)rr+rBcopydeepcopyrr2) rtextsr=r@ metadatas _metadatas documentsr/textrArEnew_docs rcreate_documentsz&RecursiveJsonSplitter.create_documentss32$U"3   'GAtm,) ==A7"I  ) (r)r N)r r r z Optional[int]returnNone)rdict[str, Any]rOr )r rQr! list[str]r"rrOrP)rrrOr)NN)rrr4zOptional[list[str]]r5zOptional[list[dict[str, Any]]]rOlist[dict[str, Any]])F)r<rQr=boolrOrS)FT)r<rQr=rTr@rTrOrR)FTN) rHrSr=rTr@rTrIzOptional[list[dict[Any, Any]]]rOzlist[Document])__name__ __module__ __qualname____doc__r __annotations__r r staticmethodrr$r)r3r>rBrN __classcell__)rs@rrr sINC<NCKO ! :G  ,%%  -115 */   D$!  &$! R! R R R  R$!48 #  2   rr) __future__rrFrtypingrrlangchain_core.documentsrrrrr`s " -JJr