ޠj؉%SSKJr SSKrSSKrSSKJrJrJrJrJ r SSK J r SSK J r "SS\ 5r\"S S S 9rS*S jrS+S jrSS/4S,SjjrS-S.SjjrSS/4S,SjjrS*SjrS*SjrS*SjrS*SjrS*SjrS*SjrS/SjrSS/4S0SjjrS*SjrS1SjrS2Sjr S3Sjr!S4Sjr"S5S jr#S6S!jr$S7S"jr%S8S#jr&S9S:S$jjr'S%r(S&\)S''S;S(jr*S.lower_case_column,sU::B:aQ6)A:B 47FLLN3T B Cs A A Nc>T"UT5$Nr)r(r5necessary_columns rlower_case_column_creator=necessary_column_validator..lower_case_column_creator9s(-=>>rz - The `z ` column/key should be lowercasezLower case column name to ``z^` column/key is missing. Please make sure you name your columns/keys appropriately, then retryr9)rrrrr)r( pd.DataFramer2rreturnr=)r(r=r>r=)r.rr0r ) r(r9rrrrr3r:r5s ` @rnecessary_column_validatorr?'s  MLMIrzz)  C 1A  C C ? ?5L'(8'99YZM9:J9K1MM,-.LMI  ##!   Ds"Bprompt completionc^^/nSnSnSn[UR5S:avURVs/sH ofT;dM UPM nnSnUH8nUVs/sH ohU;dM UPM n n[U 5S:dM,USUSUS3- nM: SUU3nS U3nS U4S jjn[S UUUS 9$s snfs snf)zC This validator will remove additional columns from the dataframe. Nr r$rz9 WARNING: Some of the additional columns/keys contain `z<` in their name. These will be ignored, and the column/key `z`` will be used instead. This could also result from a duplicate column/key in the provided file.zh - The input file should contain exactly two columns/keys per row. Additional columns/keys present are: z Remove additional columns/keys: c>UT$r8rxfieldss rr1additional_column_validator..necessary_fn^s V9 radditional_columnrrrrrErr>r)r'r.r ) r(rFadditional_columnsrrrr3 warn_messageacdupss ` radditional_column_validatorrOKsMML 2::)+GAaG $B1=1!1WA1D=4y1}"\]_\`a]^`]aaA!BB %DEWDXYeXfg :;M:NO    ##!  H>sB%B%B*B*c^SnSnSnUTRS5R5(d&UTR5R5(aiUTS:HUTR5-nUR5RUR 5nSTSU3nS U4SjjnS[ U5STS 3n[S T3UUUS 9$) z9 This validator will ensure that no completion is empty. Nc US:H$)Nr$rrEs r+non_empty_field_validator..qsbrr$z - `z?` column/key should not contain empty strings. These are rows: c4>XTS:gRT/S9$)Nr$subset)dropna)rEfields rr/non_empty_field_validator..necessary_fnvs$uX^$++E7+; ;rRemove z rows with empty sempty_rIrJ)applyanyisnull reset_indexindextolistr'r )r(rYrrr empty_rows empty_indexess ` rnon_empty_field_validatorrfisMLM %y()--//2e93C3C3E3I3I3K3Ki2o"U)*:*:*<= (..z:AAC w&efsetu  <"#m"4!55FugQO eW ##!  rc.^URTS9nUR5RUR5nSnSnSn[ U5S:a:S[ U5SSR T5SU3nS[ U5S 3nS U4S jjn[ S UUUS 9$)zQ This validator will suggest to the user to remove duplicate rows if they exist. rVNr - There are z duplicated -z sets. These are rows: r[z duplicate rowsc">URTS9$)NrV)drop_duplicatesrDs rr.duplicated_rows_validator..optional_fns$$F$3 3rduplicated_rowsrrrrrJ) duplicatedrarbrcr'joinr )r(rFrmduplicated_indexesrrrs ` rduplicated_rows_validatorrrsmm6m2O)//@GGIMLK "(-?)@(AchhW]N^M__vxJwKL  %7!8 9I  4  #!  rc^^SnSnSn[U5nUS:waFS SjmT"U5m[T5S:a*S[T5STS3nS[T5S 3nSUU4S jjn[S UUUS 9$)zO This validator will suggest to the user to remove examples that are too long. Nopen-ended generationczURSSS9nUR5RUR5$)Nc^[UR5[UR5-S:$)Ni')r'r@rArRs rrSClong_examples_validator..get_long_indexes..s c!((mc!,,>O.ORW.Wr)axis)r^rarbrc)d long_exampless rget_long_indexes1long_examples_validator..get_long_indexess6GG$W^_G`M==?((7>>@ @rrrhz. examples that are very long. These are rows: zf For conditional generation, and for classification the examples shouldn't be longer than 2048 tokens.r[z long examplesc>T"U5nTU:wa/[RRS[U5SUS35 UR U5$)NzeThe indices of the long examples has changed as a result of a previously applied recommendation. The z? long examples to be dropped are now at the following indices:  )sysstdoutwriter'drop)rElong_indexes_to_dropr| long_indexess rr,long_examples_validator..optional_fns'7':$#77JJ$$ABEFZB[A\\[\p[qqstvv233rr{rn)rzr=r>rrJ)infer_task_typer'r )r(rrrft_typer|rs @@rlong_examples_validatorrsMLKb!G)) A(+ | q ,S->,??mnzm{|cdM$S%6$7~FL 4 4  #!  rc|^ ^ SnSnSnSnSm /SQnUH~nUS:Xa:URRRS5R5(aMCURRRUSS9R5(aM|Um O T R SS5n[ U5nUS :Xa [ S S 9$SS jm [URS S9n URU :HR5(aSU S3n[ S US9$U S:waU R SS5n SU S3n[U 5S:a USUS3- nURRS[U 5*RRU SS9R5(a USU S3- nOSnU S:XaSUS3nS U U 4Sjjn[ SUUUUS9$)!z This validator will suggest to add a common suffix to the prompt if one doesn't already exist in case of classification or conditional generation. Nz ### => ) ->z ### z === z --- z ===> z ---> rrFregex\nrt common_suffixrsuffixc US==U- ss'U$Nr@rrErs r add_suffix2common_prompt_suffix_validator..add_suffixs ( v rxfixzAll prompts are identical: `zt` Consider leaving the prompts blank if you want to do open-ended generation, otherwise ensure prompts are differentrrr$z - All prompts end with suffix `r< R. This suffix seems very long. Consider replacing with a shorter suffix, such as `z5 WARNING: Some of your prompts contain the suffix `zZ` more than once. We strongly suggest that you review your prompts and add a unique suffixa - Your data does not contain a common separator at the end of your prompts. Having a separator string appended to the end of the prompt makes it clearer to the fine-tuned model where the completion should begin. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detail and examples. If you intend to do open-ended generation, then you should leave the prompts emptyzAdd a suffix separator `z` to all promptsc>T"UT5$r8rrErsuggested_suffixs rr3common_prompt_suffix_validator..optional_fna!12 2rcommon_completion_suffixrrrrrrErrrr>rrJ) r@rcontainsr_replacerr get_common_xfixallr') r(rrrrsuffix_options suffix_optiondisplay_suggested_suffixrrcommon_suffix_new_line_handledrrs @@rcommon_prompt_suffix_validatorrs$IMLK(N( E !yy}}%%d+//11 99== ! !-u ! = A A C C ( ( 077eDb!G))00$BIIH=M ]"''))2=/Bwx 9EE)6)>)>tU)K&;.remove_common_prefixs#kooc&km4( rz" - All prompts start with prefix `r<z. Fine-tuning doesn't require the instruction specifying the task, or a few-shot example scenario. Most of the time you should only add the input data into the prompt, and the desired output into the completionRemove prefix `z` from all promptsc>T"UT5$r8r)rErrs rr3common_prompt_prefix_validator..optional_fn!s+A}==rcommon_prompt_prefixrn)rErrrr>rrJ)rr@r rr')r(MAX_PREFIX_LENrrrrrs @@rcommon_prompt_prefix_validatorrsNMLK#BIIH=M00 ]"''))00=m_AN C . . r rM,]O;MNL > >  ##!  rcB^^^Sn[URSS9m[T5S:=(a TSS:Hm[T5U:a [SS9$SSjmURT:HR 5(a [SS9$S TS 3nS TS 3nSUUU4S jjn[SUUUS9$)z` This validator will suggest to remove a common prefix from the completion if a long one exist. rrr rrcfUSR[U5SUS'U(a SUS3US'U$)NrArr)rEr ws_prefixs rr@common_completion_prefix_validator..remove_common_prefix7s=L/--c&km<,  !!L/!23AlOrz& - All completions start with prefix `z_`. Most of the time you should only add the output data into the completion, without any prefixrz` from all completionsc>T"UTT5$r8r)rErrrs rr7common_completion_prefix_validator..optional_fnEs#A}i@@rcommon_completion_prefixrn)rErrrrrr>rrJ)rrAr'r r)r(rrrrrrrs @@@r"common_completion_prefix_validatorr,sN#BMMAMM"Q&B=+;s+BI =N*00 &++--00=m_MlmM$]O3IJLAA  '#!  rc^ ^ SnSnSnSn[U5nUS:XdUS:Xa [SS9$[URSS9nURU:HR 5(aSUS US 3n[SUS 9$S m /S QnUH>nURR R USS9R5(aM<Um O T RSS5n SSjm US:waURSS5n SU S 3n[U5S:a USU S 3- nURR S[U5*R R USS9R5(a USUS3- nOSnUS:XaSU S3nS U U 4Sjjn[SUUUUS9$)!z This validator will suggest to add a common suffix to the completion if one doesn't already exist in case of classification or conditional generation. Nrtclassificationrrrrz All completions are identical: `zJ` Ensure completions are different, otherwise the model will just repeat `r<rz [END]) r.z ENDz***z+++z&&&z$$$z@@@z%%%Frrrc US==U- ss'U$NrArrs rr6common_completion_suffix_validator..add_suffixvs ,6!rr$z$ - All completions end with suffix `rrz9 WARNING: Some of your completions contain the suffix `zU` more than once. We suggest that you review your completions and add a unique endingaH - Your data does not contain a common ending at the end of your completions. Having a common ending string appended to the end of the completion makes it clearer to the fine-tuned model where the completion should end. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detail and examples.zAdd a suffix ending `z` to all completionsc>T"UT5$r8rrs rr7common_completion_suffix_validator..optional_fnrrrrrrJ) rr rrArrrr_rr') r(rrrrrrrrrrrrs @@r"common_completion_suffix_validatorrPsIMLKb!G))W8H-H00#BMMAM &++--6}oFQR_Q``ab 9EE  N( ==   % %m5 % A E E G G ( ( 077eD)6)>)>tU)K&?@^?__`a }  " qsKrLLMN NM ==  2M 22 3 7 7 @ @V[ @ \ ` ` b b YZgYhi~ Md ./G.HH\]  3 3  '#!  rcS SjnSnSnSnURRSSR5S:wd URRSSS:waSnSnUn[ SUUUS 9$) z This validator will suggest to add a space at the start of the completion if it doesn't already exist. This helps with tokenization. c6USRS5US'U$)NrAcBURS5(aSU-$SU-$)Nrr$) startswith)r\s rrSLcompletions_space_start_validator..add_space_start..s!cARAR2_`:`X[_`:`r)r^rRs radd_space_start:completions_space_start_validator..add_space_starts L///0`a,rNrxrrz - The completion should start with a whitespace character (` `). This tends to produce better results due to the tokenization we use. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detailsz=Add a whitespace character to the beginning of the completioncompletion_space_startrnrJ)rArnuniquevaluesr )r(rrrrs r!completions_space_start_validatorrs LKM }}!$$&!+r}}/C/CA/Fq/IS/PB V %  %#!  rc^S U4SjjnUTRS5R5nUTRS5R5nUS-U:a[SSTSTS3S TS 3US 9$g )zl This validator will suggest to lowercase the column values, if more than a third of letters are uppercase. cH>UTRR5UT'U$r8)rr0)rEr2s r lower_case(lower_case_validator..lower_cases"fIMM'')& rc&[SU55$)Nc3~# UH4oR5(dMUR5(dM0Sv M6 g7frxN)isalphaisupper.0r3s r 9lower_case_validator....&0]AqQRQZQZQ\A == =sumrRs rrS&lower_case_validator..S0]A0]-]rc&[SU55$)Nc3~# UH4oR5(dMUR5(dM0Sv M6 g7fr)rislowerrs rrrrrrrRs rrSrrrr rz - More than a third of your `z%` column/key is uppercase. Uppercase zs tends to perform worse than a mixture of case encountered in normal language. We recommend to lower case the data if that makes sense in your domain. See https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detailsz'Lowercase all your data in column/key `r<rnNrJ)r^rr )r(r2r count_upper count_lowers ` rlower_case_validatorrs V*""#]^bbdKV*""#]^bbdKQ$;F8ChiohpqhiB6(!L"   rcSnSnSnSnSn[RRU5(GaUR5R S5(d$UR5R S5(a`UR5R S5(aSOSupxSUS3nSUS 3n[ R "X[S 9RS 5nGOmUR5R S 5(adS nSn[ R"U5n U Rn [U 5S:aUS- n[ R"U[S9RS 5nGOUR5R S5(auSnSn[US5n U R5n [ R"U R!S5V s/sHn S U /PM sn U[S9RS 5nSSS5 GOLUR5R S5(af[ R""US[S9RS 5n[U5S:Xa-SnSn[ R""U[S9RS 5nOOUR5R S5(ag[ R""US[S9RS 5n[U5S:Xa)[ R""U[S9RS 5nO>rBB''00 Ce4sCJJ2NB2w!|\\%s;BB2F)J (Y t%<<w6RSXS^S^_bScdfSgRhiB"CCI<w6T!UUIE7"23  ## K ?cE&%6"Ce37>>rBBCI& v!&S!1"!5!;!;!= %eW,MN`Mab[\n[oouvI vsB&N#BN#!4N#4M M M0N#:BN#$N#)A M+ M+&N#6 N#M M($N#(N#+2N N#N  N##>O$#O$cH[U5nSnUS:XaSUS3n[SUS9$)z This validator will infer the likely fine-tuning format of the data, and display it to the user if it is classification. It will also suggest to use ada and explain train/validation split benefits. NrzK - Based on your data it seems like you're trying to fine-tune a model for z - For classification, we recommend you try one of the faster and cheaper models, such as `ada` - For classification, you can estimate the expected model performance by keeping a held out dataset, which is not used for trainingr%r&)rr )r(rrs rformat_inferrer_validatorrsA b!GM""fgnfopUV N- HHrchURbP[RRSURSURS35 [R "S5 UR b)[RRUR 5 URbURU5nU$)zk This function will apply a necessary remediation to a dataframe, or print an error message if one exists. z ERROR in z validator: z Aborting...rx) rrstderrrrexitrrr)r(rs rapply_necessary_remediationr(s( =)9)9(:,{G\G\F]]lmn    , 223+  % %b ) Irc[RRU5 U(a [RRS5 g[5R 5S:g$)NzY Tn)rrrinputr0) input_text auto_accepts raccept_suggestionr#6s?JJZ   7==?c !!rcSnSURS3nURb2[XB5(a"URceURU5nSnURb-[R R SURS35 X4$)z[ This function will apply an optional remediation to a dataframe, based on the user input. Fz- [Recommended] z [Y/n]: Tz- [Necessary] r)rr#rrrrr)r(rr"optional_appliedr!s rapply_optional_remediationr&>s #K$<$<#=XFJ+ Z 5 5**6 66((,B#   , >+*C*C)DBGH rc[U5nSnUS:Xa[U5nUS-nO"URSS9R5nUS-nS SjnU"US-5n[R R S US 35 g ) z7 Estimate the time it'll take to fine-tune the dataset g?rg ףp= ?T)rbg|?5^?cUS:a[US5S3$US:a[US- S5S3$US:a[US- S5S3$[US- S5S3$) N<r z secondsiz minutesiQz hoursz days)round)times r format_time.estimate_fine_tuning_time..format_time]sv "9D!n%X. . D[D2Iq)*(3 3 E\D4K+,F3 3D5L!,-U3 3rz:Once your model starts training, it'll approximately take z~ to train a `curie` model, and less for `ada` and `babbage`. Queue will approximately take half an hour per job ahead of you. N)r+floatr>r)rr' memory_usagerrrr)r( ft_format expected_timer%sizer, time_strings restimate_fine_tuning_timer5Ps #IM$$2w $t+ T*..0v 4mc12KJJ D[MRQ RrcU(aSS/OS/nSnUS:aSUS3OSnUVs/sH-n[RRU5SSUUS3PM/ nn[S U55(dU$US - nMgs snf) N_train_validr$rz () _preparedrc3^# UH$n[RRU5v M& g7fr8)rrr)rrs rr get_outfnames..ss"?.>277>>!$$.>s+-rx)rrsplitextr_)rr suffixesi index_suffixrcandidate_fnamess r get_outfnamesrBms',(#2$H A $%EA3ayr owxowekrww//6q9:)F8L>Y_`owx?.>???# # Q xs3A9cURR5nSnUS:Xa'URR5RSnX4$)Nr r)rAr value_countsrb)r( n_classes pos_classs rget_classification_hyperparamsrGxsF %%'IIA~MM..066q9  rc V[U5n[URSS9n[URSS9nSnSnUS:Xa[ X5(aSnSn UR SS 5n UR SS 5n [ U 5S :aS U S 3OSn S nU(d?U(d8[RRSUSU SU SU S3 5 [U5 g[ X5(Ga[X5n U(a[ U 5S:XaSU S ;a SU S;deSn[[ U5U- [[ U5S-55nURUSS9nURUR 5nUSS/R#U S SSSSS9 USS/R#U SSSSSS9 [%U5unnU S- n US:Xa U S US3- n O5U S!U3- n O,[ U 5S:XdeUSS/R#U S SSSSS9 U(aS"OSS#-S$R'U 5-nU(a S%U SS3OSn[ U 5S :XaSOS&U S3n[RRS'US(U S SUU S)UU S3 5 [U5 g[RRS*5 g)+aE This function will write out a dataframe to a file, if the user would like to proceed, and also offer a fine-tuning command with the newly created file. For classification it will optionally ask the user if they would like to split the data into train/valid files, and modify the suggested command to include the valid set. rrFzQ- [Recommended] Would you like to split into training and validation set? [Y/n]: rTr$rrrz Make sure to include `stop=["z;"]` so that the generated texts ends at the expected place.z@ Your data will be written to a new JSONL file. Proceed [Y/n]: zK You can use your file for fine-tuning: > openai api fine_tunes.create -t ""ue After you’ve fine-tuned a model, remember that your prompt has to end with the indicator string `zX` for the model to start generating completions, rather than continuing with the prompt.r trainvalidrxig?*)r random_stater@rArecordsN)rorient force_asciiindentz! --compute_classification_metricsz" --classification_positive_class "z --classification_n_classes r\z to `z` and `z -v "ucAfter you’ve fine-tuned a model, remember that your prompt has to end with the indicator string `z Wrote modified filezd` Feel free to take a look! Now use that file when fine-tuning: > openai api fine_tunes.create -t "z z#Aborting... did not write the file )rrr@rAr#rr'rrrr5rBmaxintsamplerrbto_jsonrGrp)r(rany_remediationsr"r1common_prompt_suffixrr r!additional_params%common_prompt_suffix_new_line_handled)common_completion_suffix_new_line_handledoptional_ending_stringfnamesMAX_VALID_EXAMPLESn_traindf_traindf_validrErF files_string valid_stringseparator_reminders rwrite_out_filerds  #I*2998D.r}}8L EdJ$$ Z 5 5E,@,H,Hu,U)0H0P0PQUW\0]- 8 9A = ))R(STO P  VJ E [\a[bbcducvw^_D^EE]^t]uuw x  ""% : 3 3u, v;!#6!9(<FSTIAU UU!% #b'$66CGcM8JKGyy7y>&;QR /4vayk+" 89Q> vx]w^^vw  #L>2Z[abc[dZeefgsftuFtGGKL^K_`v_wwy z  ""% ?@rcSn[URRR55S:Xag[URR 55[U5U- :agg)z6 Infer the likely fine-tuning task type from the data rrtrzconditional generation)rr@rr'rAunique)r(CLASSIFICATION_THRESHOLDs rrrsU ! 299==   1$& 2==   !"SW/G%GG #rcSnUS:XaUR[U5S-*SOURS[U5S-nUR5S:waU$X#RS:XaU$URSnMz)zI Finds the longest common suffix or prefix of all the values in a series r$rrxNr)rr'rr)seriesr common_xfix common_xfixess rrrsK 59X5EFJJ[)A-.0 16::VlX[\gXhklXlKm   "a '   003 3  (..q1K rz,Callable[[pd.DataFrame], Remediation | None]r Validatorc[SS[[[[[ SS[ [[[[/$)Nc[US5$rr?rRs rrS get_validators..s ,Q9rc[US5$rrprRs rrSrqs ,Q =rc[US5$rrrRs rrSrqs &q(3rc[US5$rrtrRs rrSrqs &q,7r) r+rOrfrrrrrrrrrrrrget_validatorsrvs99=#!!!37&&**) rcd/nUbURU5 UH,nU"U5nUcMURU5 [X5nM. [UVs/sH!nURcURcMUPM# sn5n[UVs/sHo"RcMUPM sn5n Sn U(aB[ R RS5 UHn[XU5up U =(d U n M O[ R RS5 U =(d U n U"XX5 gs snfs snf)NFz? Based on the analysis we will perform the following actions: z No remediations found. ) appendrr_rrrrrr&) r(rr validatorsr"write_out_file_funcoptional_remediations validator&any_optional_or_necessary_remediationsany_necessary_appliedany_optional_appliedr%!any_optional_or_necessary_applieds rapply_validatorsrs.02$$[1 m  " ! ( ( 5,R=B  .1 5 4 ''3{7P7P 4 .* (=g(=AZAZ(=g!- ]^0K#=b{#[ B#7#K;K 1 78(<(U@U%#DR+  hsD(3D( D-D-)r(r=r>r )r(r=r9rr>r )r(r=rF list[str]r>r )rA)r(r=rYrr>r )r(r=r2rr>Remediation | None)rrrFrr>z'tuple[pd.DataFrame | None, Remediation])r(r rr r>r )r!rr"boolr>r)r(r=rr r"rr>ztuple[pd.DataFrame, bool])r(r=r>None)rrr rr>r)r(r=r>ztuple[int, object]) r(r=rrrVrr"rr>r)r(r=r>r)r)rjrrrr>r)r>list[Validator])r(r=rrrrryrr"rrzzCallable[..., Any]r>r), __future__rrrtypingrrrrrtyping_extensionsr _extrasr rr r r+r?rOrfrrrrrrrrrrrrr#r&r5rBrGrdrrrmrrvrrrrrsx" ??'"$*$19QR I!HHPQ]F^<4FN|D\2"JAH$N!HAH2.&.|$<V V!V,Vr I "  #. =A  $: HAV $$F 9E('S'S 'S$'S 'S  'S , 'S 'Sr