dh76 S SK r S SKrS SKJr S SKJr S SKJrJrJrJ r J
r
J r S SK J
r
S SKJrJr S SKJr S SKJrJr S S KJrJr " S
S \5 rS \ \\4 S
\
\\4 4S jr " S S\5 r g) N)datetime)Enum)AnyDictListOptionalTupleUnion)UUID) AgentAction AgentFinish)BaseCallbackHandler) BaseMessage ChatMessage)
Generation LLMResultc \ rS rSrSrSrSrSrg)LabelStudioMode z Label Studio mode enumerator.promptchat N)__name__
__module__ __qualname____firstlineno____doc__PROMPTCHAT__static_attributes__r j/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/callbacks/labelstudio_callback.pyr r s '
F Dr! r modereturnc [ R R S[ R R S0n[ U [
5 ( a [ U 5 n XR U 4$ )zGet default Label Studio configs for the given mode.
Parameters:
mode: Label Studio mode ("prompt" or "chat")
Returns: Tuple of Label Studio config and mode
a
a
)r r valuer
isinstancestr)r# _default_label_configss r" get_default_label_configsr* sb $$ ' * "" % -' R $ t$ !** -t 33r! c ^ \ rS rSr% SrSr\\S' SSS\S\R 4S\
\ S\
\ S\
\ S \S
\
\ S \ \\4 4 U 4S jjjr
S
\S\\\ SS4S jrS\\\4 S\\ S\SS4S jrS\S\4S jrSSSS.S\\\4 S\\\ S
\S\
\ S\
\\ S\
\\\4 S\S\4S jjrS \S\SS4S jrS \S\SS4S jrS!\S\SS4S" jr S\\\4 S#\\\4 S\SS4S$ jr S%\\\4 S\SS4S& jr S!\S\SS4S' jr S\\\4 S(\S\SS4S) jr S*\!S\S\4S+ jr" S6S,\S-\
\ S.\
\ S\SS4
S/ jjr#S!\S\SS4S0 jr$S1\S\SS4S2 jr%S3\&S\SS4S4 jr'S5r(U =r)$ )7LabelStudioCallbackHandlerN ae Label Studio callback handler.
Provides the ability to send predictions to Label Studio
for human evaluation, feedback and annotation.
Parameters:
api_key: Label Studio API key
url: Label Studio URL
project_id: Label Studio project ID
project_name: Label Studio project name
project_config: Label Studio project config (XML)
mode: Label Studio mode ("prompt" or "chat")
Examples:
>>> from langchain_community.llms import OpenAI
>>> from langchain_community.callbacks import LabelStudioCallbackHandler
>>> handler = LabelStudioCallbackHandler(
... api_key='',
... url='http://localhost:8080',
... project_name='LangChain-%Y-%m-%d',
... mode='prompt'
... )
>>> llm = OpenAI(callbacks=[handler])
>>> llm.invoke('Tell me a story about a dog.')
zLangChain-%Y-%m-%dDEFAULT_PROJECT_NAMENapi_keyurl
project_id project_nameproject_configr# c t > [ T
U ] 5 SS KnU( du [ R " S5 ( a [ [ R " S5 5 nO:[ SU R R
SU R R
S35 eXl
U( dy [ R " S5 ( a [ R " S5 nOG[ R " SUR S U R R
S
35 UR nX l 0 U l UR! U R U R S 9U l X@l U( a XPl S U l O[+ U5 u U l U l U=( d [ R " S 5 U l U R, b4 U R" R/ [1 U R, 5 5 U l O[4 R6 " 5 R9 U R$ 5 nU R" R; US
9n U ( a& U S U l U R2 R< U l OCU R" R? XR& S9U l U R2 R< U l U R2 R@ U l Su U l! U l" U l# U l$ U R@ RK 5 HC u pU S S:X d M Xl! U S S U l" U S S S U l# U S S S U l$ O U RB ( dD SU R$ S3n U R( [L RN :X a U S-
n OU S-
n [ U 5 eg ! [ a$ [ SU R R
S35 ef = f)Nr z
You're using z in your code, but you don't have the LabelStudio SDK Python package installed or upgraded to the latest version. Please run `pip install -U label-studio-sdk` before using this callback.LABEL_STUDIO_API_KEYz in your code, Label Studio API key is not provided. Please provide Label Studio API key: go to the Label Studio instance, navigate to Account & Settings -> Access Token and copy the key. Use the key as a parameter for the callback: zr(label_studio_api_key='', ...) or set the environment variable LABEL_STUDIO_API_KEY=LABEL_STUDIO_URLz5Label Studio URL is not provided, using default URL: z8If you want to provide your own URL, use the parameter: zj(label_studio_url='', ...) or set the environment variable LABEL_STUDIO_URL=)r0 r/ LABEL_STUDIO_PROJECT_ID)title)r8 label_config)NNNNtypeTextAreato_nameinputsr& zLabel Studio project "zI" does not have a TextArea tag. Please add a TextArea tag to the project.z
HINT: go to project Settings -> Labeling Interface -> Browse Templates and select "Generative AI -> Supervised Language Model Fine-tuning" template.z
HINT: go to project Settings -> Labeling Interface -> Browse Templates and check available templates under "Generative AI" section.)(super__init__label_studio_sdk ImportError __class__r osgetenvr(
ValueErrorr/ warningswarnLABEL_STUDIO_DEFAULT_URLr0 payloadClient ls_clientr2 r3 r# r* r1 get_projectint
ls_projectr todaystrftime get_projectsidcreate_projectparsed_label_config from_namer< r&
input_typeitemsr r )selfr/ r0 r1 r2 r3 r# ls
project_titleexisting_projectstag_nametag_info
error_messagerB s r" r? #LabelStudioCallbackHandler.__init__j sZ ) yy/00 bii(> ?@ #DNN$;$;#< =D ~~../XY
yy+,,ii 23
**,*E*E)FN~~../W X 11 )+ txx F ( "0 DI-Ft-L *D $L 2K(L ?? & "nn 8 8T__9M NDO $NN , 5 5d6G6G HM $ ; ;- ; P "3A"6"&//"4"4"&.."?"?'6I6I #@ # #'//"4"4#'??#F#F E
A dj$/ #'":":"@"@"B H :-!) ' 21 5 %h / 27 ;
"*8"4Q"7"?
#C ~~():):(; << =
yyO 2 22 G
/
]+ +) i 7 7 8 9/ 0
s N .N7run_id generationsr$ c / nU R U S nU R U S R S0 5 R S5 n[ XB5 H` u pgUR U R USU0U R
U R SSU Vs/ sH oR PM sn0S./US ./S
.5 Mb U R R U5 g s snf ) Npromptskwargsinvocation_params
model_namer` textareatext)rU r< r: r& )result
model_version)data predictions)
rI getzipappendr& rU r< rh rN import_tasks) rX r` ra tasksrc rj r
generationgs r" add_prompts_generations2LabelStudioCallbackHandler.add_prompts_generations s ,,v&y1 LL *
S$b
)
S
#&g"; F LL
F & 26 /3||,6.4z6Rz!vvz6R-S !"' .;
$
#<,
$$U+ 7Ss C
serializedrc rd c U R S:w a& [ SU R SU R S35 e[ US 5 nX#S.U R U' g).Save the prompts in memory when an LLM starts.Text
Label Studio project "" has an input type . To make it work with the mode="chat", the input type should be .
Read more here https://labelstud.io/tags/textr` )rc rd N)rV rE r2 r( rI )rX rv rc rd r` s r" on_llm_start'LabelStudioCallbackHandler.on_llm_start sh ??f $ *4+<+<*= >&&*oo%6 7@A
VH%&+2 E V r! messagec p [ U[ 5 ( a UR $ UR R $ )z Get the role of the message.)r' r rolerB r )rX r~ s r" _get_message_role,LabelStudioCallbackHandler._get_message_role s+ g{ + +<< $$-- -r! )
parent_run_idtagsmetadatamessagesr r r c V U R S:w a& [ SU R SU R S35 e/ nU HL n / n
U H0 n U
R U R U 5 U R
S.5 M2 UR U
5 MN UUUUUUS.U R [ U5 ' g)rx
Paragraphsrz r{ z>. To make it work with the mode="chat", the input type should be .
Read more here https://labelstud.io/tags/paragraphs)r content)rc r r r` r rd N)rV rE r2 ro r r rI r( ) rX rv r r` r r r rd rc message_listdialogr~ s r" on_chat_model_start.LabelStudioCallbackHandler.on_chat_model_start s ??l * *4+<+<*= >&&*oo%6 7FG
$LF'
$ 6 6w ?#*?? (
NN6 " % *
%
S[!r! tokenc g)z)Do nothing when a new token is generated.Nr )rX r rd s r" on_llm_new_token+LabelStudioCallbackHandler.on_llm_new_tokenG r! responsec [ US 5 nU R X1R 5 U R R U5 g)z>Create a new Label Studio task for each prompt and generation.r` N)r( rt ra rI pop)rX r rd r` s r"
on_llm_end%LabelStudioCallbackHandler.on_llm_endK s; VH%&
$$V-A-AB
r! errorc g)z%Do nothing when LLM outputs an error.Nr rX r rd s r" on_llm_error'LabelStudioCallbackHandler.on_llm_errorU r r! r= c g Nr )rX rv r= rd s r" on_chain_start)LabelStudioCallbackHandler.on_chain_startY s
r! outputsc g r r )rX r rd s r" on_chain_end'LabelStudioCallbackHandler.on_chain_end^ s r! c g)z+Do nothing when LLM chain outputs an error.Nr r s r" on_chain_error)LabelStudioCallbackHandler.on_chain_errora r r! input_strc g)z Do nothing when tool starts.Nr )rX rv r rd s r"
on_tool_start(LabelStudioCallbackHandler.on_tool_starte s
r! actionc g)z.Do nothing when agent takes a specific action.Nr )rX r rd s r" on_agent_action*LabelStudioCallbackHandler.on_agent_actionn r r! outputobservation_prefix
llm_prefixc g)zDo nothing when tool ends.Nr )rX r r r rd s r" on_tool_end&LabelStudioCallbackHandler.on_tool_endr s
r! c g)z&Do nothing when tool outputs an error.Nr r s r"
on_tool_error(LabelStudioCallbackHandler.on_tool_error| r r! rh c gz
Do nothingNr )rX rh rd s r" on_text"LabelStudioCallbackHandler.on_text r r! finishc gr r )rX r rd s r" on_agent_finish*LabelStudioCallbackHandler.on_agent_finish r r! )r/ rU rV rK rN r# rT rI r3 r1 r2 r< r0 r& )NN)*r r r r r r. r( __annotations__r r r rM r
r? r r rt r r r| r r r r r r r
BaseExceptionr r r r r r r r r r r
r r
__classcell__)rB s @r" r, r, N s< 2 !5#4 "& !$( 0(,,;,B,Bv,# v, c]v, SM v,
v, !
v, C ()v, v,p!,!,(,T*-=(>!,
!,FFcNF cF F
F$. . . )-$(-1'
cN'
tK()'
'
~
'
tCy!'
4S>*'
'
'
R
c
S
T
!9 ! ! !
-
3
4
sCx.
26sCx.
LO
DcN
c
d
M
S
T
cN
k
S
S
-1$(
%SM
SM
=
C
D
C
3
4
k
S
T
r! r, ) rC rF r enumr typingr r r r r r
uuidr langchain_core.agentsr r
langchain_core.callbacksr langchain_core.messagesr r langchain_core.outputsr r r r( r* r, r r! r" r sk : : : 8 < 8d 64
_$
%64
3 64rx
!4 x
r!