dh`bSSKJr SSKJrJrJrJrJr \(a SSKJ r J r J r "SS5r g)) annotations) TYPE_CHECKINGAnyIterableListOptional) DataFrameRow SparkSessionc\rSrSrSrSSSjjr\SSSjj5rSSjrSSjr SSjr SSS jjr SS jr SS jr SS jrSSS jjrSSSjjrSSSjjrSrg)SparkSQL z;SparkSQL is a utility class for interacting with Spark SQL.NcSSKJn U(aUOURR 5UlUb%UR R RU5 Ub%UR R RU5 [UR55Ul U(a [U5O [5Ul UR(a/URUR- nU(a[SUS35eU(a [U5O [5UlUR(a/URUR- nU(a[SUS35eUR5n U (a [U 5O URUl[#U[$5(d ['S5eX`lg![a [S5ef=f) aQInitialize a SparkSQL object. Args: spark_session: A SparkSession object. If not provided, one will be created. catalog: The catalog to use. If not provided, the default catalog will be used. schema: The schema to use. If not provided, the default schema will be used. ignore_tables: A list of tables to ignore. If not provided, all tables will be used. include_tables: A list of tables to include. If not provided, all tables will be used. sample_rows_in_table_info: The number of rows to include in the table info. Defaults to 3. rr Fpyspark is not installed. Please install it with `pip install pyspark`Nzinclude_tables  not found in databasezignore_tables z,sample_rows_in_table_info must be an integer) pyspark.sqlr ImportErrorbuilder getOrCreate_sparkcatalogsetCurrentCatalogsetCurrentDatabaseset_get_all_table_names _all_tables_include_tables ValueError_ignore_tablesget_usable_table_names_usable_tables isinstanceint TypeError_sample_rows_in_table_info) self spark_sessionrschema ignore_tablesinclude_tablessample_rows_in_table_infor missing_tables usable_tabless _/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/utilities/spark_sql.py__init__SparkSQL.__init__ s2  0+M 0D0D0P0P0R    KK   1 1' :   KK   2 26 :t88:;6Ds>2#%   !11D4D4DDN %n%55KL5Bc-0su   !0043C3CCN $^$44JK335 4Ac-0tGWGW3S99JK K*C'E X  s F::Gc SSKJn URR U5R 5nU"U40UD6$![a [S5ef=f)zjCreating a remote Spark Session via Spark connect. For example: SparkSQL.from_uri("sc://localhost:15002") rrr)rr rrremoter)cls database_uri engine_argskwargsr sparks r/from_uriSparkSQL.from_uriKs\  0 $$++L9EEG5#F##  X  s :AcUR(a UR$[URUR- 5$)zGet names of tables available.)rsortedrr )r's r/r!SparkSQL.get_usable_table_names\s3   '' 'd&&)<)<<==cURRS5RS5R5n[ [ SU55$)Nz SHOW TABLES tableNamecUR$N)r@)rows r//SparkSQL._get_all_table_names..esCMMr>)rsqlselectcollectlistmap)r'rowss r/rSparkSQL._get_all_table_namescs;{{}-44[AIIKC14899r>cURRSU35R5SRnUR S5nUSUS-$)NzSHOW CREATE TABLE rUSING;)rrFrHcreatetab_stmtfind)r'table statementusing_clause_indexs r/_get_create_table_stmtSparkSQL._get_create_table_stmtgsX KKOO08 9 A A CA F U U '^^G4,,-33r>crUR5nUb2[U5RU5nU(a[SUS35eUn/nUHXnUR U5nUR (a"US- nUSUR U5S3- nUS- nURU5 MZ SRU5nU$)Nz table_names rz /* z*/z ) r!r differencerrUr&_get_sample_spark_rowsappendjoin)r' table_namesall_table_namesr-tables table_name table_info final_strs r/get_table_infoSparkSQL.get_table_infoos557  " -88IN </??U!VWW)O)J44Z@J..h& 4#>#>z#J"K2NN d" MM* % *KK' r>cSUSUR3nURRU5nSR[ [ SUR R555nURU5nSRUVs/sHnSRU5PM sn5nURSUSUSU3$s snf![a SnN*f=f) NzSELECT * FROM z LIMIT  cUR$rB)name)fs r/rD1SparkSQL._get_sample_spark_rows..s166r>rXz rows from z table: ) r&rrFr\rIrJr)fields_get_dataframe_results Exception)r'rRquerydf columns_str sample_rowsrCsample_rows_strs r/rZSparkSQL._get_sample_spark_rowss wt/N/N.OP [[__U #iiS)9299;K;K%L MN  !55b9K"ii;(O;C3;(OPO ../{5'm2 ! )P ! O !s$( CC"CC CCcl[[[UR5R 555$rB)tuplerJstrasDictvalues)r'rCs r/_convert_row_as_tupleSparkSQL._convert_row_as_tuples#Scjjl113455r>c\[[URUR555$rB)rIrJrzrH)r'rps r/rmSparkSQL._get_dataframe_resultss C22BJJLABBr>cURRU5nUS:XaURS5n[UR U55$)None)rrFlimitrwrm)r'commandfetchrps r/run SparkSQL.runs> [[__W % E>!B4..r233r>c`URU5$![anSU3sSnA$SnAff=f)a6Get information about specified tables. Follows best practices as specified in: Rajkumar et al, 2022 (https://arxiv.org/abs/2204.00498) If `sample_rows_in_table_info`, the specified number of sample rows will be appended to each table description. This can increase performance as demonstrated in the paper. Error: N)rcr)r'r]es r/get_table_info_no_throw SparkSQL.get_table_info_no_throws7 !&&{3 3 ! *QC=  ! -(--c`URX5$![anSU3sSnA$SnAff=f)a Execute a SQL command and return a string representing the results. If the statement returns rows, a string of the results is returned. If the statement returns no rows, an empty string is returned. If the statement throws an error, the error message is returned. rN)rrn)r'rrrs r/ run_no_throwSparkSQL.run_no_throws5 !88G+ + ! *QC=  !r)rr rr&rr")NNNNN) r(zOptional[SparkSession]r Optional[str]r)rr*Optional[List[str]]r+rr,r$rB)r5rwr6zOptional[dict]r7rreturnr )rz Iterable[str])rRrwrrw)r]rrrw)rCr rrv)rpr rrI)all)rrwrrwrrw)__name__ __module__ __qualname____firstlineno____doc__r0 classmethodr9r!rrUrcrZrzrmrrr__static_attributes__r>r/r r sE15!% $-1.2)*=D-=D=D =D + =D , =D$'=D~>B$$-;$NQ$ $$ >:4$ "6C4 ! ! !r>r N) __future__rtypingrrrrrrr r r r rr>r/rs#"??88q!q!r>