o 6WjM@s*dZddlmZdZdgZddlmZddlZddlm Z m Z m Z m Z m Z mZmZmZmZmZmZddlmZmZmZmZmZmZdd lmZmZdd lmZm Z m!Z!m"Z"dd l#m$Z$ersdd l%m&Z&dd lm'Z'ddl(m)Z)m*Z*m+Z+dZ,e e e-e-fe-e-gdfZ.GdddeeZ/Gddde!Z0dS)zCUse the HTMLParser library to parse HTML files that aren't too bad.) annotationsMITHTMLParserTreeBuilder) HTMLParserN) AnyCallablecastDictIterableListOptional TYPE_CHECKINGTupleTypeUnion) AttributeDictCDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution UnicodeDammit)DetectsXMLParsedAsHTMLHTMLHTMLTreeBuilderSTRICTParserRejectedMarkup) BeautifulSoup)NavigableString) _Encoding _Encodings _RawMarkupz html.parserc@seZdZUdZded<dZded< edd;ddZd ed <ded<ded<dd?dd Z d>d@d"d#Z dAd%d&Z e d'Ze d(ZedBd+d,ZdCd-d.ZdCd/d0ZdAd1d2ZdDd4d5ZdAd6d7ZdAd8d9Zd:S)EBeautifulSoupHTMLParserreplacestrREPLACEignoreIGNOREon_duplicate_attributesouprargsrr+&Union[str, _DuplicateAttributeHandler]kwargscOs@||_||_|jj|_tj|g|Ri|g|_|dSN)r,r+builderattribute_dict_classr__init__already_closed_empty_element_initialize_xml_detector)selfr,r+r-r/r7f/home/esfera/Documents/content_generation/venv/lib/python3.10/site-packages/bs4/builder/_htmlparser.pyr3Us   z BeautifulSoupHTMLParser.__init__z List[str]r4messagereturnNonecCst|r0r)r6r9r7r7r8errorpszBeautifulSoupHTMLParser.errortagattrsList[Tuple[str, Optional[str]]]cCs"|j||dd|j|dddS)zHandle an incoming empty-element tag. html.parser only calls this method when the markup looks like . F)handle_empty_elementcheck_already_closedN)handle_starttag handle_endtag)r6r=r>r7r7r8handle_startendtags z*BeautifulSoupHTMLParser.handle_startendtagTr@boolc Cs|}|D]3\}}|durd}||vr5|j}||jkrq|d|jfvr)|||<qtt|}||||q|||<q|jjjrF| \}} nd}} |jj |dd||| d} | durl| j rl|rl|j |dd|j ||jdurx||dSdS)zHandle an opening tag, e.g. '' :param handle_empty_element: True if this tag is known to be an empty-element tag (i.e. there is not expected to be any closing tag). N) sourceline sourceposFrA)r2r+r)r'r_DuplicateAttributeHandlerr,r1store_line_numbersgetposrCis_empty_elementrDr4append_root_tag_name_root_tag_encountered) r6r=r>r@ attr_dictkeyvalueon_duperHrItagObjr7r7r8rCs2         z'BeautifulSoupHTMLParser.handle_starttagrBcCs.|r||jvr|j|dS|j|dS)zHandle a closing tag, e.g. '' :param tag: A tag name. :param check_already_closed: True if this tag is expected to be the closing portion of an empty-element tag, e.g. ''. N)r4remover,rD)r6r=rBr7r7r8rDs z%BeautifulSoupHTMLParser.handle_endtagdatacCs|j|dS)z4Handle some textual data that shows up between tags.N)r, handle_datar6rWr7r7r8rXsz#BeautifulSoupHTMLParser.handle_dataz ^([0-9]+)(.*)z^([0-9a-f]+)(.*)nameTuple[str, bool, str]c Csd}d}d}d}|j}|ds|dr |dd}d}|j}d}zt||}Wn!tyJ||}|durHt|d |}|d}Ynw|durTd}|}nt|\}}|||fS) aConvert a numeric character reference into an actual character. :param name: The number of the character reference, as obtained by html.parser :return: A 3-tuple (dereferenced, replacement_added, extra_data). `dereferenced` is the dereferenced character reference, or the empty string if there was no reference. `replacement_added` is True if the reference could only be dereferenced by replacing content with U+FFFD REPLACEMENT CHARACTER. `extra_data` is a portion of data following the character reference, which was deemed to be normal data and not part of the reference at all. rGF xXNr) &_DECIMAL_REFERENCE_WITH_FOLLOWING_DATA startswith"_HEX_REFERENCE_WITH_FOLLOWING_DATAint ValueErrorsearchgroupsrnumeric_character_reference) clsrZ dereferencedreplacement_added extra_databasereg real_namematchr7r7r8(_dereference_numeric_character_references0     z@BeautifulSoupHTMLParser._dereference_numeric_character_referencecCsH||\}}}|rd|j_|dur|||dur"||dSdS)zHandle a numeric character reference by converting it to the corresponding Unicode character and treating it as textual data. :param name: Character number, possibly in hexadecimal. TN)rqr,contains_replacement_charactersrX)r6rZrjrkrlr7r7r8handle_charref"s z&BeautifulSoupHTMLParser.handle_charrefcCs0tj|}|dur |}nd|}||dS)zHandle a named entity reference by converting it to the corresponding Unicode character(s) and treating it as textual data. :param name: Name of the entity reference. Nz&%s)rHTML_ENTITY_TO_CHARACTERgetrX)r6rZ characterrWr7r7r8handle_entityref1s z(BeautifulSoupHTMLParser.handle_entityrefcCs&|j|j||jtdS)zOHandle an HTML comment. :param data: The text of the comment. N)r,endDatarXrrYr7r7r8handle_commentDs  z&BeautifulSoupHTMLParser.handle_commentdeclcCs6|j|tdd}|j||jtdS)zYHandle a DOCTYPE declaration. :param data: The text of the declaration. zDOCTYPE N)r,rxlenrXr)r6rzr7r7r8 handle_declMs  z#BeautifulSoupHTMLParser.handle_declcCsN|drt}|tdd}nt}|j|j||j|dS)z{Handle a declaration of unknown type -- probably a CDATA block. :param data: The text of the declaration. zCDATA[N)upperrbrr{rr,rxrX)r6rWrir7r7r8 unknown_declWs  z$BeautifulSoupHTMLParser.unknown_declcCs0|j|j||||jtdS)z\Handle a processing instruction. :param data: The text of the instruction. N)r,rxrX_document_might_be_xmlrrYr7r7r8 handle_pifs   z!BeautifulSoupHTMLParser.handle_piN)r,rr-rr+r.r/r)r9r&r:r;)r=r&r>r?r:r;)T)r=r&r>r?r@rFr:r;)r=r&rBrFr:r;)rWr&r:r;)rZr&r:r[)rZr&r:r;)rzr&r:r;)__name__ __module__ __qualname__r'__annotations__r)r3r<rErCrDrXrecompilerarc classmethodrqrsrwryr|r~rr7r7r7r8r$>s2     >     6   r$cseZdZUdZdZded<dZded<eZded<ee e gZ d ed <d ed <dZ ded <  d&d'fdd Z   d(d)dd Zefd*d$d%ZZS)+rzA Beautiful soup `bs4.builder.TreeBuilder` that uses the :py:class:`html.parser.HTMLParser` parser, found in the Python standard library. FrFis_xmlT picklabler&NAMEz Iterable[str]featuresz$Tuple[Iterable[Any], Dict[str, Any]] parser_argsTRACKS_LINE_NUMBERSNOptional[Iterable[Any]] parser_kwargsOptional[Dict[str, Any]]r/rc spt}dD]}||vr||}|||<qtt|jdi||p#g}|p'i}||d|d<||f|_dS)aConstructor. :param parser_args: Positional arguments to pass into the BeautifulSoupHTMLParser constructor, once it's invoked. :param parser_kwargs: Keyword arguments to pass into the BeautifulSoupHTMLParser constructor, once it's invoked. :param kwargs: Keyword arguments for the superclass constructor. r*Fconvert_charrefsNr7)dictpopsuperrr3updater)r6rrr/extra_parser_kwargsargrS __class__r7r8r3s  zHTMLParserTreeBuilder.__init__markupr#user_specified_encodingOptional[_Encoding]document_declared_encodingexclude_encodingsOptional[_Encodings]r:DIterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]]ccst|tr|dddfVdSg}|r||g}|r!||t|||d|d}|jdur3td|j|j|j|jfVdS)a2Run any preliminary steps necessary to make incoming markup acceptable to the parser. :param markup: Some markup -- probably a bytestring. :param user_specified_encoding: The user asked to try this encoding. :param document_declared_encoding: The markup itself claims to be in this encoding. :param exclude_encodings: The user asked _not_ to try any of these encodings. :yield: A series of 4-tuples: (markup, encoding, declared encoding, has undergone character replacement) Each 4-tuple represents a strategy for parsing the document. This TreeBuilder uses Unicode, Dammit to convert the markup into Unicode, so the ``markup`` element of the tuple will always be a string. NFT)known_definite_encodingsuser_encodingsis_htmlrzPCould not convert input to Unicode, and html.parser will not accept bytestrings.) isinstancer&rNrunicode_markuproriginal_encodingdeclared_html_encodingrr)r6rrrrrrdammitr7r7r8prepare_markups4     z$HTMLParserTreeBuilder.prepare_markup _parser_classtype[BeautifulSoupHTMLParser]r;c Cs|j\}}t|ts J|jdusJ||jg|Ri|}z |||Wnty:}zt|d}~wwg|_dS)z :param markup: The markup to feed into the parser. :param _parser_class: An HTMLParser subclass to use. This is only intended for use in unit tests. N) rrr&r,feedcloseAssertionErrorrr4)r6rrr-r/parserer7r7r8rs    zHTMLParserTreeBuilder.feed)NN)rrrrr/r)NNN) rr#rrrrrrr:r)rr#rrr:r;)rrr__doc__rrr HTMLPARSERrrrrrr3rr$r __classcell__r7r7rr8rqs     ! H)1r __future__r __license____all__ html.parserrrtypingrrrr r r r r rrr bs4.elementrrrrrr bs4.dammitrr bs4.builderrrrrbs4.exceptionsrbs4rr bs4._typingr!r"r#rr&rJr$rr7r7r7r8s,  4    5