dhw#SSKJr SSKrSSKrSSKJr SSKJrJrJ r J r J r SSK J r SSKJr \(a SSKrSSKJr SSjr"S S \5rSS jr"S S \5r\rg)) annotationsN)Path) TYPE_CHECKINGDictListOptionalUnion)Document) BaseLoader) EntityLikec6USnUSnUSnUSUSUS3$)zBCombine message information in a readable format ready to be used.datefromtextz on z:  )rowrsenderrs e/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/document_loaders/telegram.pyconcatenate_rowsrs6 v;D [F v;DXT$r$t ,,c,\rSrSrSrSSjrSSjrSrg) TelegramChatFileLoaderzLoad from `Telegram chat` dump.cXlg)zInitialize with a path.N file_path)selfpaths r__init__TelegramChatFileLoader.__init__src[UR5n[USS9n[R"U5nSSS5 SR SWS55nS[ U50n[XES9/$!,(df  N@=f) Load documents.utf8encodingNc3# UH5nUSS:XdM[US[5(dM([U5v M7 g7f)typemessagerN) isinstancestrr).0r*s r .TelegramChatFileLoader.load..'sA (v)+ &0:76?C0P & W % %(s >>>messagessource page_contentmetadata)rropenjsonloadjoinr,r )rpfdrr4s rr7TelegramChatFileLoader.load sv   !f % ! A&ww Z=   c!f%d>??& %s A11 A?rN)rzUnion[str, Path]returnList[Document])__name__ __module__ __qualname____firstlineno____doc__r r7__static_attributes__rrrrrs)@rrcSSKJn U"S/SQSS9n[U[5(aU/nUVs/sH n[ US9PM nn[ U5HupVUS-UR S 'M /nUHnURUR5n[ U5H]upY[ XR S US .S 9nUR S S UR S 3UR S'URU5 M_ M U$s snf)zIConvert a string or list of strings to a list of Documents with metadata.r)RecursiveCharacterTextSplitteri )r .!?, r') chunk_size separators chunk_overlap)r3page)rSchunkr2-rTr1) langchain_text_splittersrGr+r,r enumerater4 split_textr3append) rrG text_splitterrS page_docsidoc doc_chunkschunksrTs r text_to_docsr`1s G2>M $v9=>t,I>I& 1u V'J))#*:*:;!&)HA"ll66JUV-WC), V(<'=Qs||G?T>U%VCLL "   c " * %?sC;cv\rSrSrSrS S SjjrS SjrS SjrSSjrSSjr S r g)TelegramChatApiLoaderSz)Load `Telegram` chat json directory dump.Nc@XlX lX0lX@lXPlg)a Initialize with API parameters. Args: chat_entity: The chat entity to fetch data from. api_id: The API ID. api_hash: The API hash. username: The username. file_path: The file path to save the data to. Defaults to "telegram_data.json". N) chat_entityapi_idapi_hashusernamer)rrerfrgrhrs rr TelegramChatApiLoader.__init__Vs$'   "rc # SSKJn /nU"URURUR5IShvN nUR UR 5ShvN nURSLnU(aURROSnURURURURR5URUUS.5 MNN SSS5IShvN O!,IShvN (df  O=f[UR SSS9n["R$"X'SS S 9 SSS5 g!,(df  g=f7f) z8Fetch data from Telegram API and save it as a JSON file.r)TelegramClientN) sender_idrr message.idis_reply reply_to_idwzutf-8r%F) ensure_asciiindent) telethon.syncrkrhrfrg iter_messagesrereply_toreply_to_msg_idrYrlrr isoformatidr5rr6dump)rrkdataclientr*rnror:s rfetch_data_from_telegram.TelegramChatApiLoader.fetch_data_from_telegramns0!$--dmmLLPV!'!5!5d6F6F!G g"++47BJg..>>PT  %,%6%6 ' ' 6 6 8&-jj$,'2   M !GMLLLLL$..# 8A IIdE! <9 8 8s5ECEC4C!CC!A>C4EC!!C4" E-C0.E4D :C= ; D E#E: E E Ec ^SU4SjjmXS)nXSRS/S9nUSR[5US'USVs0sHnXD/T"XC5-_M nnU$s snf)aCreate a dictionary of message threads from the given data. Args: data (pd.DataFrame): A DataFrame containing the conversation data with columns: - message.sender_id - text - date - message.id - is_reply - reply_to_id Returns: dict: A dictionary where the key is the parent message ID and the value is a list of message IDs in ascending order. cp>XSU:HSR5n/nUHnX4/T"XA5-- nM U$)a  Recursively find all replies to a given parent message ID. Args: parent_id (int): The parent message ID. reply_data (pd.DataFrame): A DataFrame containing reply messages. Returns: list: A list of message IDs that are replies to the parent message ID. rorm)tolist) parent_id reply_datadirect_replies all_repliesreply_id find_repliess rr@TelegramChatApiLoader._get_message_threads..find_repliessV(=(AY(NOfh  K*zL,NNN + rrnro)subsetrm)rintr pd.DataFramer>z List[int])dropnaastyper)rr{parent_messagesreply_messagesrmessage_threadsrs @r_get_message_threads*TelegramChatApiLoader._get_message_threadss$ 0Z 001:./66}o6N)7}(E(L(LS(Q}% -\: :  {\)%LL L:    s A"c,SnUR5HjupEX"SRU5RSS9SR5nUVs/sHn[ U5PM nnUSR U5S-- nMl UR 5$s snf)a Combine the message texts for each parent message ID based on the list of message threads. Args: message_threads (dict): A dictionary where the key is the parent message ID and the value is a list of message IDs in ascending order. data (pd.DataFrame): A DataFrame containing the conversation data: - message.sender_id - text - date - message.id - is_reply - reply_to_id Returns: str: A combined string of message texts sorted by date. r'rmr)byrrMz. )itemsisin sort_valuesrr,r8strip)rrr{ combined_textr message_ids message_textselems r_combine_message_texts,TelegramChatApiLoader._combine_message_textss* '6&;&;&= "I,',,[9:'0  4AA=4SY=MA SXXm4u< ""$$ BsBc<URb9SSKnUR5 [R"UR 55 [UR5n[USS9n[R"U5nSSS5 SSK nUR"W5nUR"U5nURU5nUR!X5n [#U 5$![ a [ S5ef=f!,(df  N}=f![ a [ S5ef=f)r#Nrzy`nest_asyncio` package not found. please install with `pip install nest_asyncio` r$r%zf`pandas` package not found. please install with `pip install pandas` )re nest_asyncioapplyasynciorunr} ImportErrorrrr5r6r7pandasjson_normalize DataFramerrr`) rrr9r:r;pdnormalized_messagesdfrcombined_textss rr7TelegramChatApiLoader.loads     ' #""$ D99;<   !f % ! A&  !//2 \\- .33B744_IN++3 ! & %   s#8C&C4DC14 DD)rgrfrerrh)NNNNztelegram_data.json) rezOptional[EntityLike]rfz Optional[int]rg Optional[str]rhrrr,)r>None)r{rr>dict)rzDict[int, List[int]]r{rr>r,r=) r@rArBrCrDr r}rrr7rErrrrbrbSsy3-1 $"&"&- #)## #  #  #0=.9v$%3$%;G$% $%L",rrb)rrr>r,)rzUnion[str, List[str]]r>r?) __future__rrr6pathlibrtypingrrrrr langchain_core.documentsr )langchain_community.document_loaders.baser rrtelethon.hintsr rrr`rbTelegramChatLoaderrrrrsV" ==-@)-@Z@0Du,Ju,r,r