a xdF@sZddlZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z m Z m Z ddlmZddlmZddlmZddlmZmZmZmZddlmZmZdd lmZddlZddlZdd l m Z m!Z!ddl"Z"eZ#d d Z$d dZ%d6ddZ&ddZ'ddZ(e)edddZ*e e)e+fe e,dddZ-e)e edddZ.e)edddZ/e e)e+fed d!d"Z0e)e edd#d$Z1d7e e)e e)e+dfe e e)e e)e ee ed%d&d'Z2d8e)e ee ed(d)d*Z3d9e)e ee ed+d,d-Z4eeej5d.d/d0Z6d:e e)e e)e+dfe)e)d1d2d3Z7ej8fd4d5Z9e:e;ZdS);N)AnyDictIteratorOptionalSetUnion)cookiejar_from_dict)DEFAULT_REQUESTS_TIMEOUT)FacebookScraper) CredentialsPostRawPostProfile)html_element_to_stringparse_cookie_file) exceptions)datetime timedeltac st|tr|dkrBzddl}|jdd}WqtdYq0qz t|}Wqty}ztd|WYd}~qd}~00nt|t rt |}|durdd|Dfd dd D}|rtd |t j j |t std dS) NZ from_browserrz .facebook.com)Z domain_namez8browser_cookie3 must be installed to use browser cookiesz"Cookies are in an invalid format: cSsg|] }|jqS)name.0crrQ/var/www/html/Ranjet/env/lib/python3.9/site-packages/facebook_scraper/__init__.py -zset_cookies..csg|]}|vr|qSrrrZ cookie_namesrrr.r)Zc_userxszMissing cookies with name(s): zCookies are not valid) isinstancestrbrowser_cookie3loadModuleNotFoundErrorr ValueErrorrZInvalidCookiesdictr_scrapersessioncookiesupdateZ is_logged_in)r(r!eZmissing_cookiesrrr set_cookiess.   ( r+cCstitj_dSN)rr&r'r(rrrr unset_cookies6sr-TcCst||dSr,)r& set_proxy)proxyverifyrrrr.;sr.cCst|dSr,)r&set_user_agent) user_agentrrrr1?sr1cCst|dSr,)r& set_noscript)noscriptrrrr3Csr3)accountreturncKs8|dttjd<|dd}t|tj|fi|S)aGet a Facebook user's profile information Args: account(str): The account of the profile. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). timeoutr(N)popr r&requests_kwargsr+ get_profiler5kwargsr(rrrr:Gs  r:)post_idr6cKs8|dttjd<|dd}t|tj|fi|S)aGet reactors for a given post ID Args: post_id(str): The post ID, as returned from get_posts cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). r7r(N)r8r r&r9r+ get_reactors)r=r<r(rrrr>Ws  r>cKs8|dttjd<|dd}t|tj|fi|S)zGet a Facebook user's friends Args: account(str): The account of the profile. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). r7r(N)r8r r&r9r+ get_friendsr;rrrr?gs  r?cKs8|dttjd<|dd}t|tj|fi|S)zGet a page's information Args: account(str): The account of the profile. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). r7r(N)r8r r&r9r+ get_page_infor;rrrr@ws r@)groupr6cKs8|dttjd<|dd}t|tj|fi|S)zGet a group's profile information Args: group(str or int): The group name or ID cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). r7r(N)r8r r&r9r+get_group_info)rAr<r(rrrrBs rBcKs8|dttjd<|dd}t|tj|fi|S)zGet a page's shop listings Args: account(str): The account of the profile. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). r7r(N)r8r r&r9r+get_shopr;rrrrCs rC)r5rA post_urlshashtag credentialsr6c Kstdd||||fD}|dkr*td|dttjd<|dd}|dur`|dur`tdt||d i}t|t rt j d d d d d|D}|d|d|vrt j dd d |dd|vr|d|d<d|vr|dd|d<|dd|d<|durtj ||dur4tj |fi|S|durPtj|fi|S|durltj|fi|S|durtj|fi|StddS)a-Get posts from a Facebook page or group. Args: account (str): The account of the page. group (int): The group id. post_urls ([str]): List of manually specified post URLs. credentials (Optional[Tuple[str, str]]): Tuple of email and password to login before scraping. timeout (int): Timeout for requests. page_limit (int): How many pages of posts to go through. Use None to try to get all of them. extra_info (bool): Set to True to try to get reactions. youtube_dl (bool): Use Youtube-DL for video extraction. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). Yields: dict: The post representation in a dictionary. css|]}|duVqdSr,r)rargrrr rzget_posts..r z7You need to specify either account, group, or post_urlsr7r(Nrzget_posts..r5sleep?The sleep parameter has been removed, it won't have any effect.pages page_limit reactions extra_infoF youtube_dlNo account nor group)sumr$r8r r&r9r+ setdefaultrsetwarningswarnlogin get_postsZget_group_postsZget_posts_by_hashtagZget_posts_by_url) r5rArDrErFr<Z valid_argsr(rJrrrrasD           ra)r5rFr6cKs|durtd|dttjd<|dd}|durF|durFtdt||di}t|trt j ddd d d |D}|d |d |vrt j ddd |d d|vr|d|d<|dd|d<|dd|d<|durtj |tj |fi|S)aGet photo posts from a Facebook page. Args: account (str): The account of the page. credentials (Optional[Tuple[str, str]]): Tuple of email and password to login before scraping. timeout (int): Timeout for requests. page_limit (int): How many pages of posts to go through. Use None to try to get all of them. extra_info (bool): Set to True to try to get reactions. youtube_dl (bool): Use Youtube-DL for video extraction. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). Yields: dict: The post representation in a dictionary. NzYou need to specify accountr7r(rIrJrKrLrMcSsi|] }|dqSrOrrPrrrrRrzget_photos..r5rSrTrUrVrXFrWrY) r$r8r r&r9r+r\rr]r^r_r` get_photos)r5rFr<r(rJrrrrbs0      rb)wordrFr6cKs |s td|dttjd<|dd}|durB|durBtdt||di}t|tr|t j ddd d d |D}|d |d |vrt j ddd |d d|vr|d|d<d|vr|dd|d<|dd|d<|durtj ||durtj |fi|StddS)aGet posts from a Facebook page or group. Args: word (str): The word for searching posts. group (int): The group id. credentials (Optional[Tuple[str, str]]): Tuple of email and password to login before scraping. timeout (int): Timeout for requests. page_limit (int): How many pages of posts to go through. Use None to try to get all of them. extra_info (bool): Set to True to try to get reactions. youtube_dl (bool): Use Youtube-DL for video extraction. cookies (Union[dict, CookieJar, str]): Cookie jar to use. Can also be a filename to load the cookies from a file (Netscape format). Yields: dict: The post representation in a dictionary. zYou need to specify wordr7r(NrIrJrKrLrMcSsi|] }|dqSrOrrPrrrrRLrz'get_posts_by_search..rcrSrTrUrVrWrXFrYrZ) r$r8r r&r9r+r\rr]r^r_r`get_posts_by_search)rcrFr<r(rJrrrrd(s6       rd)postsourcelocationcCs|d}|d}td|t||ddH}|dtj||dtd|d |t|d d Wdn1s~0YdS) Nr=z.htmlzWriting post %swt)modez T)pretty) loggerdebugopenjoinpathwritejsondumpr r)rerfrgr=filenamefrrrwrite_post_to_diskis    rw)r5rAruencodingc s|dd}|dur(|jddd|d<|durJt|p8|d|d}|durZt}|d krjtj}nt|d d |d }d}|d d}|dd} t t | d} d} |dr$zRtd} | } Wdn1s0Ytd| | r | } Wnty"Yn0fdd}|dztf||| |d|D]}|dur|d}zt|||WntytdYn0n|dr|dj|d<|r |ddkr|dn*st|tj|dd}|n|ddkr$|dd}|d r|tj|d!|d tjd"}|d#r|tj|d#|d tjd"r|d}|r|ddkrˆrfd$d%|D}t j!||td&d'n |"||s|d(r|d(| krtd)|d(d*| d+| d,q$d}t#$|qVWn@t%y:Yn.tyf}zt&'WYd}~n d}~00|ddkr|d-|rt(d.tj)d/|*dS)0arWrite posts from an account or group to a CSV or JSON file Args: account (str): Facebook account name e.g. "nike" or "nintendo" group (Union[str, int, None]): Facebook group id e.g. 676845025728409 filename (str): Filename, defaults to _posts.csv encoding (str): Encoding for the output file, defaults to locale.getpreferredencoding() credentials (Optional[Tuple[str, str]]): Tuple of email and password to login before scraping. Defaults to scrape anonymously timeout (Optional[int]): Timeout for requests. page_limit (Optional[int]): How many pages of posts to go through. Use None to try to get all of them. extra_info (Optional[bool]): Set to True to try to get reactions. dump_location (Optional[pathlib.Path]): Location where to write the HTML source of the posts. dump_locationNT)exist_okFZ remove_sourcez_posts.format-w)newlinerxrSr days_limitiB)days resume_filerz Existing URL:cs@r.handle_pagination_urlkeys)r5rA start_urlZrequest_url_callbackrfzError writing post to diskrsz[ ignore) extrasaction,textZmatching)flagsZ not_matchingcsi|]\}}|vr||qSrr)rrQv)rrrrRrz&write_posts_to_csv..rj)rlrktimezReached days_limit - z is more than z days old (older than )z ]zCouldn't get any posts.)file)+r8mkdirr getlocalegetpreferredencodingsysstdoutrprnowrreadlinestriprnroFileNotFoundErrorrarw Exception exceptionhtmlrrlistrcsv DictWriter writeheaderresearch IGNORECASEitemsrsrtwriterowrrSKeyboardInterrupt traceback print_excprintstderrclose)r5rArurxr<ryZ output_fileZ first_postrSrZ max_post_timerrv existing_urlrrerfZ dict_writermatchr*r)rrrwrite_posts_to_csvus      *             rcCs*t}||t|t|dSr,)logging StreamHandlersetLevelrn addHandler)levelhandlerrrrenable_loggings  r)T)NNNNN)N)N)NNNN)?rrsrrpathlibrr^typingrrrrrrZrequests.cookiesr constantsr Zfacebook_scraperr Zfb_typesr r rrutilsrrr~rrrrrrr&r+r-r.r1r3r r:intr%r>r?r@rBrCrarbrdPathrwrDEBUGr getLogger__name__rnr NullHandlerrrrrs               Q = A   |