dh&SrSSKrSSKrSSKJrJr SSKJrJrJ r J r J r J r J r SSKJr SSKJr \(aSSKJr SSKJr SS KJr SS KJrJrJr \R4"\5r"S S \5r"S S\5r"SS\5rg)zQLoader that uses Playwright to load a page, then uses unstructured to parse html.N)ABCabstractmethod) TYPE_CHECKING AsyncIteratorDictIteratorListOptionalUnion)Document) BaseLoader)Browser)Page)Response)rrrc `\rSrSrSr\SSSSSSS \4S j5r\SS SS SS S \4Sj5rSr g)PlaywrightEvaluatorzAbstract base class for all evaluators. Each evaluator should take a page, a browser instance, and a response object, process the page as necessary, and return the resulting text. pagerbrowserrresponserreturncg)zSynchronously process the page and return the resulting text. Args: page: The page to process. browser: The browser instance. response: The response from page.goto(). Returns: text: The text content of the page. Nselfrrrs k/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/document_loaders/url_playwright.pyevaluatePlaywrightEvaluator.evaluates AsyncPage AsyncBrowser AsyncResponsec # g7f)zAsynchronously process the page and return the resulting text. Args: page: The page to process. browser: The browser instance. response: The response from page.goto(). Returns: text: The text content of the page. Nrrs revaluate_async"PlaywrightEvaluator.evaluate_async+s  srN) __name__ __module__ __qualname____firstlineno____doc__rstrrr$__static_attributes__rrrrrsl   V  i  :  RU        *8  DS      rrcj\rSrSrSrSS\\\4SjjrSSSS S S S \4S jr SSSSS SS \4Sjr Sr g)UnstructuredHtmlEvaluator<z@Evaluate the page HTML content using the `unstructured` library.Nremove_selectorscLSSKnXlg![a [S5ef=f)z%Initialize UnstructuredHtmlEvaluator.rNzQunstructured package not found, please install it with `pip install unstructured`) unstructured ImportErrorr0)rr0r2s r__init__"UnstructuredHtmlEvaluator.__init__?s5  !1  -  s #rrrrrrrcpSSKJn UR=(d /HSnURU5R 5nUH+nUR 5(dMUR S5 M- MU UR5nU"US9nSRUV s/sHn [U 5PM sn 5$s sn f)z3Synchronously process the HTML content of the page.rpartition_htmlelement => element.remove()text unstructured.partition.htmlr8r0locatorall is_visiblercontentjoinr+ rrrrr8selectorelementselement page_sourceels rr"UnstructuredHtmlEvaluator.evaluateKs>--33H||H-113H#%%''$$%BC$4 lln !{3{{h7hCGh7887sB3r r!r"c# SSKJn UR=(d /HknURU5R 5IShvN nUH;nUR 5IShvN (dM"UR S5IShvN M= Mm UR5IShvN nU"US9nSRUV s/sHn [U 5PM sn 5$NNvNXN:s sn f7f)z4Asynchronously process the HTML content of the page.rr7Nr9r:r<r=rDs rr$(UnstructuredHtmlEvaluator.evaluate_asyncYs ?--33H!\\(37799H# ++---!**+HIII$4 !LLN* !{3{{h7hCGh788:-I*7s`AC"CC" C!C",C"CC" C!C":CC"C"C"C"C")r0)N) r&r'r(r)r*r r r+r4rr$r,rrrr.r.<saJ 1$s))< 1 9V 9i 9: 9RU 999*89DS9 9rr.c\rSrSrSrSS\\S\S\S\\\S\\ S \\ \\4S \\ \\ R\44S jjrS \\4S jrS \\4SjrS \\4SjrSrg)PlaywrightURLLoaderjaLoad `HTML` pages with `Playwright` and parse with `Unstructured`. This is useful for loading pages that require javascript to render. Attributes: urls (List[str]): List of URLs to load. continue_on_failure (bool): If True, continue loading other URLs on failure. headless (bool): If True, the browser will run in headless mode. proxy (Optional[Dict[str, str]]): If set, the browser will access URLs through the specified proxy. browser_session (Optional[Union[str, os.PathLike[str]]]): Path to a file with browser session data that can be used to restore the browser session. Example: .. code-block:: python from langchain_community.document_loaders import PlaywrightURLLoader urls = ["https://api.ipify.org/?format=json",] proxy={ "server": "https://xx.xx.xx:15818", # https://: "username": "username", "password": "password" } loader = PlaywrightURLLoader(urls, proxy=proxy) data = loader.load() Nurlscontinue_on_failureheadlessr0 evaluatorproxybrowser_sessioncSSKnXlX lX0lX`lXplU(aU(a [S5eU=(d [U5Ul g![a [S5ef=f)z%Load a list of URLs using Playwright.rNzMplaywright package not found, please install it with `pip install playwright`z:`remove_selectors` and `evaluator` cannot be both not None) playwrightr3rPrQrRrTrU ValueErrorr.rS) rrPrQrRr0rSrTrUrWs rr4PlaywrightURLLoader.__init__st   #6   .  L  #Q&?@P&Q% +  s AA-rc ## SSKJn U"5nURRURUR S9nSnUR (aj[RRUR 5(aURUR S9nO"[RSUR 35 UcUR5nURHnUR5nURU5nUc[!SU35eUR#S5 UR$R'XcU5nUR)5 S U0n [+XS 9v M UR)5 SSS5 g![,a9n UR.(a![R1S US U 35 Sn A MU eSn A ff=f!,(df  g=f7f) zLoad the specified URLs using Playwright and create Document instances. Returns: A list of Document instances with loaded content. r)sync_playwrightrRrTN storage_stateSession file not found: "page.goto() returned None for url loadsource page_contentmetadataError fetching or processing , exception: )playwright.sync_apir[chromiumlaunchrRrTrUospathexists new_contextloggerwarningrPnew_pagegotorXwait_for_load_staterSrcloser ExceptionrQerror) rr[prcontexturlrrr;rees r lazy_loadPlaywrightURLLoader.lazy_loads 8  !jj''djj'QGG##77>>$"6"677%11@T@T1UGNN%=d>R>R=S#TU!--/yy "++-D#yy~H'(+McU)STT,,V4>>224(KDJJL (#H"HH!( MMOC 4! // ;C5 aSQ   5 sNGCGA>E:G1 G: F=,F80G6F88F==G G Gc`# UR5Vs/sShvN oPM N sn$s snf7f)Load the specified URLs with Playwright and create Documents asynchronously. Use this function when in a jupyter notebook environment. Returns: A list of Document instances with loaded content. N) alazy_load)rdocs raloadPlaywrightURLLoader.aloads%&*__%677c777s$.)%# %)%).c # SSKJn U"5IShvN nURRURUR S9IShvN nSnUR (ar[RRUR 5(a"URUR S9IShvN nO"[RSUR 35 UcUR5IShvN nURHnUR5IShvN nURU5IShvN nUc[!SU35eUR#S5IShvN UR$R'XcU5IShvN nUR)5IShvN S U0n [+XS 97v M UR)5IShvN SSS5IShvN gGNGNGN-NNNNNjNT![,a:n UR.(a"[R1S US U 35 Sn A GM*U eSn A ff=fNqNc!,IShvN (df  g=f7f) r~r)async_playwrightNr\r]r_r`rarbrcrfrg)playwright.async_apirrirjrRrTrUrkrlrmrnrorprPrqrrrXrsrSr$rtr rurQrv) rrrwrrxryrrr;rerzs rrPlaywrightURLLoader.alazy_loads :#%%JJ--t}}DJJ-WWGG##77>>$"6"677$+$7$7&*&:&:%8%GNN%=d>R>R=S#TU ' 3 3 55yy !(!1!1!33D%)YYs^3H'(+McU)STT226:::!%!>!>th!WWD**,&& (#H"HH!(--/ ! !G&%%W 643;W&! // ;C5 aSQ    "G&%%%sI GI 2H3 G A!H3,G-=H3*G+H3?G(GG(+G ,)G(G"#G(9G$:G(G&G(&H3=H/>H3 I  H1I H3H3H3G( G("G($G(&G(( H,2,H'H3%H''H,,H31I 3I 9H< : I I )rUrQrSrRrTrP)TTNNNN)r&r'r(r)r*r r+boolr rrr rkPathLiker4rr r{rrrr,rrrrNrNjs>%)0437*.BFR3iR"R R #49- R /0 RS#X'R"%R[[-=(=">?RB)8H-)V8T(^8,"-"9,"rrN) r*loggingrkabcrrtypingrrrrr r r langchain_core.documentsr )langchain_community.document_loaders.baser rrr!rr rr"rh getLoggerr&rorr.rNrrrrsmW #VVV-@<6>;;   8 $# ## L+9 3+9\^"*^"r