dh_ SSKrSSKrSSKrSSKrSSKJrJrJrJrJ r J r J r J r SSK Jr SSKJr SSKJr SSKJr \R*"\5r"SS\5r"S S \5r"S S \5r"S S\5r"SS\5rS\S\4Sjrg)N)AnyCallableDictIteratorLiteralOptionalTupleUnion)Document)BaseBlobParser)Blob) is_openai_v1c\rSrSrSrSSSSSSSSSS. S\\S\\S\\S \\/\4S4S \\S \\S \\ S S4S\\ S\S\ 4Sjjr \ S\\\44Sj5rS\S\\4SjrSrg)AzureOpenAIWhisperParsera Transcribe and parse audio files using Azure OpenAI Whisper. This parser integrates with the Azure OpenAI Whisper model to transcribe audio files. It differs from the standard OpenAI Whisper parser, requiring an Azure endpoint and credentials. The parser is limited to files under 25 MB. **Note**: This parser uses the Azure OpenAI API, providing integration with the Azure ecosystem, and making it suitable for workflows involving other Azure services. For files larger than 25 MB, consider using Azure AI Speech batch transcription: https://learn.microsoft.com/azure/ai-services/speech-service/batch-transcription-create?pivots=rest-api#use-a-whisper-model Setup: 1. Follow the instructions here to deploy Azure Whisper: https://learn.microsoft.com/azure/ai-services/openai/whisper-quickstart?tabs=command-line%2Cpython-new&pivots=programming-language-python 2. Install ``langchain`` and set the following environment variables: .. code-block:: bash pip install -U langchain langchain-community export AZURE_OPENAI_API_KEY="your-api-key" export AZURE_OPENAI_ENDPOINT="https://your-endpoint.openai.azure.com/" export OPENAI_API_VERSION="your-api-version" Example Usage: .. code-block:: python from langchain.community import AzureOpenAIWhisperParser whisper_parser = AzureOpenAIWhisperParser( deployment_name="your-whisper-deployment", api_version="2024-06-01", api_key="your-api-key", # other params... ) audio_blob = Blob(path="your-audio-file-path") response = whisper_parser.lazy_parse(audio_blob) for document in response: print(document.page_content) Integration with Other Loaders: The AzureOpenAIWhisperParser can be used with video/audio loaders and `GenericLoader` to automate retrieval and parsing. YoutubeAudioLoader Example: .. code-block:: python from langchain_community.document_loaders.blob_loaders import ( YoutubeAudioLoader ) from langchain_community.document_loaders.generic import GenericLoader # Must be a list youtube_url = ["https://your-youtube-url"] save_dir = "directory-to-download-videos" loader = GenericLoader( YoutubeAudioLoader(youtube_url, save_dir), AzureOpenAIWhisperParser(deployment_name="your-deployment-name") ) docs = loader.load() N) api_keyazure_endpoint api_versionazure_ad_token_providerlanguagepromptresponse_format temperature max_retriesrrrrrrrjsontextsrt verbose_jsonvttrdeployment_namerc <U=(d [RRS5UlU=(d [RRS5UlU=(d [RRS5UlX@lXPlX`lXpl Xl Xl Xl SSK n [5(aKU R!URURUR URUR S9UlgUR(aURU lUR(aURU lUR (aUR U lSU lXlg![a [S5ef=f) a Initialize the AzureOpenAIWhisperParser. Args: api_key (Optional[str]): Azure OpenAI API key. If not provided, defaults to the `AZURE_OPENAI_API_KEY` environment variable. azure_endpoint (Optional[str]): Azure OpenAI service endpoint. Defaults to `AZURE_OPENAI_ENDPOINT` environment variable if not set. api_version (Optional[str]): API version to use, defaults to the `OPENAI_API_VERSION` environment variable. azure_ad_token_provider (Union[Callable[[], str], None]): Azure Active Directory token for authentication (if applicable). language (Optional[str]): Language in which the request should be processed. prompt (Optional[str]): Custom instructions or prompt for the Whisper model. response_format (Union[str, None]): The desired output format. Options: "json", "text", "srt", "verbose_json", "vtt". temperature (Optional[float]): Controls the randomness of the model's output. deployment_name (str): The deployment name of the Whisper model. max_retries (int): Maximum number of retries for failed API requests. Raises: ImportError: If the required package `openai` is not installed. AZURE_OPENAI_API_KEYAZURE_OPENAI_ENDPOINTOPENAI_API_VERSIONrNEopenai package not found, please install it with `pip install openai`)rrrrrazure)osenvirongetrrrrrrrrr"ropenai ImportErrorr AzureOpenAI_clientapi_baseapi_type) selfrrrrrrrrr"rr,s j/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/document_loaders/parsers/audio.py__init__!AzureOpenAIWhisperParser.__init__Vs>`H"**..1G"H ,W ?V0W&N"**..9M*N'>$  .&.&   >>!-- #22 ,, ,,(,(D(D .DL||!%"""&"5"5%)%5%5"%FO!L+ W  s 3FFreturncURURURURS.nUR 5VVs0sH up#UcM X#_M snn$s snnfN)rrrrrrrritemsr2paramskvs r3_create_params'AzureOpenAIWhisperParser._create_paramsS kk#33++   "(A1AAA AAblobc## [[UR5S5n[5(aHURR R R"SURUS.URD6nOIURRR"SURURUS.URD6n[[U[5(d UROUSUR 0S9v g![a ef=f7f)a Lazily parse the provided audio blob for transcription. Args: blob (Blob): The audio file in Blob format to be transcribed. Yields: Document: Parsed transcription from the audio file. Raises: Exception: If an error occurs during transcription. rbmodelfile)rG deployment_idrHsource page_contentmetadataN)openstrpathrr/audiotranscriptionscreater"r?Audio transcribe Exceptionr isinstancerrJ)r2rCfile_obj transcripts r3 lazy_parse#AzureOpenAIWhisperParser.lazy_parses"DII- ~~!\\//>>EE..!)) "\\//::.."&"6"6!))  j#..$ ,      s* D AC=9D :AC=;D = D  D ) r/rrrrr"rrrrr)__name__ __module__ __qualname____firstlineno____doc__rrPr rrfloatintr4propertyrrr?r rr r[__static_attributes__rNr3rrsCP"&(,%)BF"& $ '+T"#T"! T" c] T" "'xC'8$'>!? T"3-T" T" @ A4 G T"e_T"T"T"lBS#XBB* t* (:* rfrc\rSrSrSrSSSSSSSSS.S\\S\S \\S \\S4S \\S4S \\ S S4S\\S4S\4Sjjjr \ S\ \\ 44Sj5rS\S\\4SjrSrg)OpenAIWhisperParserazTranscribe and parse audio files. Audio transcription is with OpenAI Whisper model. Args: api_key: OpenAI API key chunk_duration_threshold: Minimum duration of a chunk in seconds NOTE: According to the OpenAI API, the chunk duration should be at least 0.1 seconds. If the chunk duration is less or equal than the threshold, it will be skipped. Ng?z whisper-1)chunk_duration_thresholdbase_urlrrrrrGrrjrkrrrrrrGcXlX lUbUO[RR S5UlX@lXPlX`lXpl Xl g)NOPENAI_API_BASE) rrjr)r*r+rkrrrrrG) r2rrjrkrrrrrGs r3r4OpenAIWhisperParser.__init__sK (@% ,H"**..AR2S !  .& rfr6cURURURURS.nUR 5VVs0sH up#UcM X#_M snn$s snnfr8r9r;s r3r?"OpenAIWhisperParser._create_params rArBrCc ## SSKn[U5n[5(a%UR UR UR S9nODUR (aUR UlUR (aUR UlSnUS-S-n[[S[U5U55GHaupxX8X-n U RUR::aM([R"U RSS 9R!55n UR"bUR"S US 3-U lO S US 3U l['S US-S35 Sn U S:au[5(a>WR(R*R,"SUR.U S.UR0D6n O&UR2R5UR.U 5n O['S5 GM'[?[AU [85(d U RBOU UR"US.S9v GMd g![a [S5ef=f![6a?n U S- n ['SU S[9U 535 [:R<"S5 Sn A OSn A ff=fU S:aGM1N7f)Lazily parse the blob.rNr')rrk<mp3format_part_z.mp3part_Transcribing part !rrFzAttempt z failed. Exception: z&Failed to transcribe after 3 attempts.)rJchunkrKrN)"r,r-_get_audio_from_blobrOpenAIrrkr0 enumeraterangelenduration_secondsrjioBytesIOexportreadrJnameprintrRrSrTrGr?rUrVrWrPtimesleepr rXr)r2rCr,rRclientchunk_durationchunk_duration_ms split_numberirrYattemptsrZes r3r[OpenAIWhisperParser.lazy_parsesF   %T* >>]]4<<$--]PF||!%}}"&--*R/$6 )q#e*>O)PQOLa34E%%)F)FFzz%,,e,"<"A"A"CDH{{& $ |nD.I I "' ~T :  &|a'7&8: ;HQ, "#~~%+\\%@%@%G%G&"&**8&?C?R?R& &,\\%<%?!*c22(__$(KK,G  A R/ W  ^!"MHHXJ.B3q6(KLJJqMM"Q,sHJ HEJ A3H4 AJ H11J 4 I=>5I83J 8I== J )rrkrjrrGrrr)N)r]r^r_r`rarrPrbr rr4rdrrr?r rr r[rerNrfr3rhrhs "&+."&%)#' *. ##(  3-  T " c4i  @ A4 G 5$;'2BS#XBBAtA(:Arfrhc n\rSrSrSrSS\S\\S\S\S\\\ 4 S jjr S \ S \ \ 4S jrS rg)OpenAIWhisperParserLocaliWaTranscribe and parse audio files with OpenAI Whisper model. Audio transcription with OpenAI Whisper model locally from transformers. Parameters: device - device to use NOTE: By default uses the gpu if available, if you want to use cpu, please set device = "cpu" lang_model - whisper model to use, for example "openai/whisper-medium" forced_decoder_ids - id states for decoder in multilanguage model, usage example: from transformers import WhisperProcessor processor = WhisperProcessor.from_pretrained("openai/whisper-medium") forced_decoder_ids = WhisperProcessor.get_decoder_prompt_ids(language="french", task="transcribe") forced_decoder_ids = WhisperProcessor.get_decoder_prompt_ids(language="french", task="translate") Ndevice lang_model batch_size chunk_lengthforced_decoder_idscSSKJn SSKnUS:XaSUlO(UR R 5(aSOSUlURS:XaSnU(aUOUUlO_UR RUR5RS - n U S :aSn OU S :aS n O U S :aSn OSn U(aUOU Ul[SUR5 X0l U"SURUURS9Ul Ub&XPRRRlgg![a [S5ef=f![a [S5ef=f![ a#n ["R%SU S35 Sn A gSn A ff=f)aInitialize the parser. Args: device: device to use. lang_model: whisper model to use, for example "openai/whisper-medium". Defaults to None. forced_decoder_ids: id states for decoder in a multilanguage model. Defaults to None. batch_size: batch size used for decoding Defaults to 8. chunk_length: chunk length used during inference. Defaults to 30s. r)pipelinezQtransformers package not found, please install it with `pip install transformers`NCtorch package not found, please install it with `pip install torch`cpuzcuda:0zopenai/whisper-baseiXzopenai/whisper-smalli.zopenai/whisper-mediumzopenai/whisper-largezUsing the following model: zautomatic-speech-recognition)rGchunk_length_srzOUnable to set forced_decoder_ids parameter for whisper modelText of exception: z9Therefore whisper model will use default mode for decoder) transformersrr-torchrcuda is_availablerget_device_properties total_memoryrrpiperGconfigrrWloggerinfo) r2rrrrrrr default_modelmem rec_modelexception_texts r3r4!OpenAIWhisperParserLocal.__init__ns*  -    U?DK&+jj&=&=&?&?(UDK ;;% 1M,6jMDO**224;;?LLPWXCTz1 t2 u3 2 ,6jIDO +T__=$ *//';;    ) )sr)rrrJrK)librosar-rrrrrrrQloadrcopyrr rJ)r2rCrrRrYyr predictions r3r[#OpenAIWhisperParserLocal.lazy_parses  %T*::ell%l8==?@ "499+Q/0 X% 0YYqvvxDOOYDVL # ,  # (  sC B3B+C 3C  C )rrrr)0NN)r]r^r_r`rarPrrcr rr4r rr r[rerNrfr3rrWsy0$(48 LLSML L  L %U4[1 L\ t (: rfrc d\rSrSrSrSSSSS.S\\S\\S \S \4S jjrS \S \ \ 4Sjr Sr g)YandexSTTParserizSTranscribe and parse audio files. Audio transcription is with OpenAI Whisper model.Ngeneralauto)r iam_tokenrGrrrrGrc^USLUSL:Xa [S5eXlX lX0lX@lg)aInitialize the parser. Args: api_key: API key for a service account with the `ai.speechkit-stt.user` role. iam_token: IAM token for a service account with the `ai.speechkit-stt.user` role. model: Recognition model name. Defaults to general. language: The language in ISO 639-1 format. Defaults to automatic language recognition. Either `api_key` or `iam_token` must be provided, but not both. Nz?Either 'api_key' or 'iam_token' must be provided, but not both.) ValueErrorrrrGr)r2rrrGrs r3r4YandexSTTParser.__init__s;* tOd!2 3Q  "  rfrCr6c# # SSKJnJnJn SSKJn [U5nUR(aU"URURS9S9 OU"URURS9S9 UR5nURUl URUl URUlUR!U5nUH%n [#U R$SUR&0S 9v M' g ![ a [ S5ef=f7f) rrr)configure_credentialscredsmodel_repository)AudioProcessingTypezYyandex-speechkit package not found, please install it with `pip install yandex-speechkit`)r)yandex_credentials)rrJrKN) speechkitrrr speechkit.sttrr-rrYandexCredentialsrrecognition_modelrGrFullaudio_processing_typerVr normalized_textrJ) r2rCrrrrrRrGresultress r3r[YandexSTTParser.lazy_parses  P P 9%T* << !#(#:#:4<<#:#P  "#(#:#:T^^#:#T !224jj &9&>&>#!!%(C 00"DKK0 3 1  sDC+CD+DD)rrrrG r]r^r_r`rarrPr4r rr r[rerNrfr3rrsb9 "&#' !#!C= !  !  !<#t#(:#rfrcX\rSrSrSrSSS.S\\S\\4SjjrS \S \ \ 4S jr Sr g) FasterWhisperParseri aTranscribe and parse audio files with faster-whisper. faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is up to 4 times faster than openai/whisper for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU. It can automatically detect the following 14 languages and transcribe the text into their respective languages: en, zh, fr, de, ja, ko, ru, es, th, it, pt, vi, ar, tr. The gitbub repository for faster-whisper is : https://github.com/SYSTRAN/faster-whisper Example: Load a YouTube video and transcribe the video speech into a document. .. code-block:: python from langchain.document_loaders.generic import GenericLoader from langchain_community.document_loaders.parsers.audio import FasterWhisperParser from langchain.document_loaders.blob_loaders.youtube_audio import YoutubeAudioLoader url="https://www.youtube.com/watch?v=your_video" save_dir="your_dir/" loader = GenericLoader( YoutubeAudioLoader([url],save_dir), FasterWhisperParser() ) docs = loader.load() rN)r model_sizerrcSSKnUS:XaSUlO(URR 5(aSOSUlURS:XaSUlOcURR UR5RS- nUS:aSUlO#US :aS UlOUS :aS UlOS UlUbUS;aX lggg![a [S5ef=f)zInitialize the parser. Args: device: It can be "cuda" or "cpu" based on the available device. model_size: There are four model sizes to choose from: "base", "small", "medium", and "large-v3", based on the available GPU memory. rNrrrbaserrui smallrmediumlarge-v3)rrrr)rr-rrrrrr)r2rrrrs r3r4FasterWhisperParser.__init__Cs   U?DK$)JJ$;$;$=$=&5DK ;;% $DO**224;;?LLPWXCTz"(t")t"*",  !DD",E "5 U  s C CrCr6c # # SSKJn [U5n[R "UR SS9R55nU"URURS9nURUSS9upgUHpn[URURS URUR4-UR S [#UR$S -5-S .UR&ES 9v Mr g![a [S5ef=f7f)rrr) WhisperModelzUfaster_whisper package not found, please install it with `pip install faster-whisper`rvrw)rr~) beam_sizez[%.2fs -> %.2fs]z%d%%d)rJ timestampsr probabilityrKN)faster_whisperrr-rrrrrrrrVr rrJstartendrroundlanguage_probabilityrM) r2rCrrRrYrGsegmentsrsegments r3r[FasterWhisperParser.lazy_parseps  3%T*::ell%l8==?@T__T[[A))(a)@G$\\"kk"4 w{{7S"S $ #)E$2K2Kc2Q,R#R  mm     /  sDC*C D*DDrrNrfr3rr sJ J!'$( +- +-SM +-Zt(:rfrrCr6cnSSKJn [UR[ 5(a1UR [R"UR55nU$URc.UR(aUR UR5nU$[S5e![a [S5ef=f)aGet audio data from blob. Args: blob: Blob object containing the audio data. Returns: AudioSegment: Audio data from the blob. Raises: ImportError: If the required package `pydub` is not installed. ValueError: If the audio data is not found in the blob r) AudioSegmentzCpydub package not found, please install it with `pip install pydub`zUnable to get audio from blob) pydubrr-rXdatabytes from_filerrrQr)rCrrRs r3rrs & $))U##&&rzz$))'<= L  tyy&&tyy1 L899   Q   s BB4)rloggingr)rtypingrrrrrrr r langchain_core.documentsr )langchain_community.document_loaders.baser 1langchain_community.document_loaders.blob_loadersr langchain_community.utils.openair getLoggerr]rrrhrrrrrNrfr3rs  QQQ-DB9   8 $P ~P fq.qh~ ~~ BEnEPn.nbtrf