ipHddlZddlZddlZddlZddlZddlZddlZddlmZddl m Z m Z ddl m Z mZddlZddlZddlZddlZddlmZmZddlmZddlmZ ddlZej8dk7rddlZer,dd lmZdd l m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-dddZ.e/ej`ejbzZ2e/ej`ejfzejbzdzZ4ddiddiddiddiddiddiddidZ5dZ6ddjoe5jqdZ9Gd d!Z:Gd"d#Z;d$ee<dzd%e?fd(Z@d3d)ee?fd*ZAd+ee?d-e?d%e?@[]^_`{|}~textz5There is a Llama in my lawn, how can I get rid of it?zyWrite a Python function that integrates any Python function f(x) numerically over an arbitrary interval [x_start, x_end].z4How many helicopters can a human eat in one sitting?z4Count to 10 but skip every number ending with an 'e'zWhy aren't birds real?z2Why is it important to eat socks after meditating?z$Which number is larger, 9.9 or 9.11?)llamacode helicopternumbersbirdssocksnumbers2a **TRANSFORMERS CHAT INTERFACE** Chat interface to try out a model. Besides chatting with the model, here are some basic commands: - **!help**: shows all available commands (set generation settings, save chat, etc.) - **!status**: shows the current status of the model and generation settings - **!clear**: clears the current conversation and starts a new one - **!exit**: closes the interface am **TRANSFORMERS CHAT INTERFACE HELP** Full command list: - **!help**: shows this help message - **!clear**: clears the current conversation and starts a new one - **!status**: shows the current status of the model and generation settings - **!example {NAME}**: loads example named `{NAME}` from the config and uses it as the user input. Available example names: `z`, `a#` - **!set {ARG_1}={VALUE_1} {ARG_2}={VALUE_2}** ...: changes the system prompt or generation settings (multiple settings are separated by a space). Accepts the same flags and format as the `generate_flags` CLI argument. If you're a new user, check this basic flag guide: https://huggingface.co/docs/transformers/llm_tutorial#common-options - **!save {SAVE_NAME} (optional)**: saves the current chat and settings to file by default to `./chat_history/{MODEL_ID}/chat_{DATETIME}.yaml` or `{SAVE_NAME}` if provided - **!exit**: closes the interface ceZdZdededefdZdeedeeeezdzffdZ defd Z d Z d efd Z d ed efdZ ddededefdZddefdZdefdZdefdZy) RichInterfacemodel_iduser_idbase_urlcLt|_||_||_||_yN)r_consoler$r%r&)selfr$r%r&s f/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/cli/chat.py__init__zRichInterface.__init__ks        streamreturnNc PK|jjd|jdt|jd5}d}d}t j}d}|d{23d{}|j dj j}t|j dd|}t|dd} | t| d |}|setjd d |}||z }g} |jD]G} | j| | jd r| jd 7| jdItdj| j!d} |j#| d7!76 dddn #1swYnxYwt jz } | dkDrKdkDrF|| z }|jj|jjd|d| dd|dd|jjfSw)Nz [bold blue]:)consolerefresh_per_secondr finish_reasonusagecompletion_tokensz <(/*)(\w*)>z\<\1\2\>z``` z z github-dark) code_themeT)refreshz[dim]z tokens in z.1fzs (z tok/s)[/dim])r)printr$rtimechoicesdeltacontentgetattrresub splitlinesappend startswithrjoinstripupdate)r*r.liverr7 start_timer5tokenoutputsr6lineslinemarkdownelapsed tok_per_secs r+ stream_outputzRichInterface.stream_outputqs l4==/<= $--A >. 4$D ! J(,M%+|) 4) 4e--*0088 ' a(8/= Y w5$(/7JL](^%&&gF OO--DLL&u- T* V,-$BGGEN$8$8$:}U Hd 3S ,) 4| . 4. 4. 4`))+ * Q;,q0+g5K MM   ! MM  %(9':+gc]RUVabeUffs t u ]""sPAH& F#F$F(F ,F-F 0DFF  F H&FB H&c|jjd|jd}|jj|S)z!Gets user input from the console. [bold red]: )r)inputr%r;)r*rUs r+rUzRichInterface.inputs8 ##k$,,t$DE  r-c8|jjy)zClears the console.N)r)clear)r*s r+rWzRichInterface.clears r-rc|jjd|jd||jjy)z%Prints a user message to the console.rTz>:[/ bold red] N)r)r;r%)r*rs r+print_user_messagez RichInterface.print_user_messages6 k$,,7GvNO r-colorcz|jjd|d||jjy)z,Prints text in a given color to the console.z[bold ]N)r)r;)r*rrZs r+ print_colorzRichInterface.print_colors1 fUG1TF34 r-messagedefaultc|rdnd}|jjd|d|d}|jj|jj }|s|S|dvS)zFDisplays a yes/no prompt to the user, returning True for confirmation.zY/nzy/Nz [bold yellow]z (z): >yyes)r)rUr;rGlower)r*r^r_ default_hintresponses r+confirmzRichInterface.confirmsf 'uU ==&&wir,s'ST >>#))+N<''r-minimalc|jjt|rtnt|jjy)z'Prints the help message to the console.N)r)r;rHELP_STRING_MINIMAL HELP_STRING)r*rgs r+ print_helpzRichInterface.print_helps. HG%8UV r-modelc tj|jjdddid}|j Gddt }dd d d d |j jtd zdzk\fd}ttdtd|t|j }|j|dd}d}t||j d5|jD]}|r|j!dst#j$|dd} | j'd} | dk(r| j'dd}n| dk(rt)| j'dd| d k(s| j'd!} | j'd"} || } | r2| d#k(rd$nd%}|j+|| | d&| j'd'|(|j+|| d)d*ddd|r)|j j-t/d+d,n(|j j-t/d+d-|j j-y#1swYwxYw).N/z /load_modelrlT)jsonr.ceZdZdZy)3RichInterface.print_model_load..StatsColumnc|js tdS|jjddk(rt j t |j}t j t |j}|jr,dt j t |jdnd}|j6dt |jdzdt |jdzd}nd}t|d |||d Stt |jd t |jS) Nr4unitbytesz z/s<:02drnzprogress.download)style) totalrfieldsgetr decimalint completedspeedtime_remaining)r*taskdonetotretas r+renderz:RichInterface.print_model_load..StatsColumn.renderszz8O;;??6*g5#++C,?@D"**3tzz?;CJN**b!1!1#djj/!B C2FZ\E**6 "3t':':b'@#A"B!CH[H[^`H`DabeCfg 4&#ugcU ;CVWWs4>>231S_4EFGGr-N)__name__ __module__ __qualname__rr-r+ StatsColumnrqs Hr-rzLoading processorzLoading configzDownloading fileszLoading into memory) processorconfigdownloadweightsFc>j||}rd|S|S)Nu → )r{) stage_key stage_textrlshow_model_prefix stage_labelss r+_labelz.RichInterface.print_model_load.._labels/%)))Y?J  |44 r-z[bold]{task.description}() bar_width)r2r)ryF)r2 transientsdata: statusreadycachederrorr^z Unknown errorloadingstageprogressrrtitemscurrentry) descriptionr~ryrsr)rr~ryz_*z was already loaded.*_z is warm.*_)requestspostr&rstripraise_for_statusrr)widthlenrrrradd_taskr iter_linesrEroloadsr{ RuntimeErrorrHr;r)r*rlrerrrtask_idrrNeventrrproglabelrsrrs ` @@r+print_model_loadzRichInterface.print_model_loadsd==DMM$8$8$=#>k!JRY[`Qajno!!# H. H&-&+,  !MM//3u:>B3FF   1 2  # M  MM  ##F;$7t#D (DMMT B ] ++- ]4??9#= 48,8,W$"YYx7FW$&uyyO'LMMY&!IIg.E 99Z0D"5ME*/:*=w7 #$y/Y]YaYabiYjqu(!UaW[\1 ] ]6  MM  2eW4J)K L M MM  2eWK)@ A B ? ] ]s max_new_tokens=100 do_sample=False eos_token_id=[1,2]`. If you're a new user, check this basic flag guide: https://huggingface.co/docs/transformers/llm_tutorial#common-optionsuserzKUsername to display in chat interface. Defaults to the current user's name. system_promptzSystem prompt. save_folderzFolder to save chat history. examples_pathz"Path to a yaml file with examples.generation_configzPath to a local generation config file or to a HuggingFace repo containing a `generation_config.json` file. Other generation settings passed as CLI arguments will be applied on top of this generation config.r/c ||_t|j} | jtdk(r1| jtdk(r|j |j||_||_||_t|} | jdd| jd it|| |_ |||jjd|_||n t|_|r/t#|5} t%j&| |_dddn t*|_t-s t/dt1j2|j5y#1swYBxYw) rrrT) do_samplemax_new_tokens)r&r$rNzHYou need to install rich to use the chat interface. (`pip install rich`)r)r&rrDEFAULT_HTTP_ENDPOINTr check_healthr$rrload_generation_configrHparse_generate_flagsrto_dictsettings get_usernameropenyaml safe_loadexamplesDEFAULT_EXAMPLESr ImportErrorasynciorun _inner_run) r*r$r&rrrrrrparsedrfs r+r,z Chat.__init__/s:F! $--( ??3J? ?FKKShioSpDp   dmm ,  *&((9: S 9 =,^<= %-8t{{ObObOde !,D,.  m$ 2 $q 1  2 2-DM!"hi i  DOO%& 2 2s <E$$E-ct|dzd} tj|}|jdk7rt d|d|jd y #tj $rt d|dwxYw) NrnhealthzThe server running on z returned status code z on health check (/health).zNo server currently running on z. To run a local server, please run `transformers serve` in aseparate shell. Find more information here: https://huggingface.co/docs/transformers/servingT)rhttpxr{ status_code ValueError ConnectError)url health_urloutputs r+rzChat.check_healthusS3Y1  YYz*F!!S( ,SE1GHZHZG[[vw) !! 1#7op  s AA#A6 user_input interfacerrchatcd}|dk(r't|j}|jnV|dk(r|jn?|j drt |j dkr|j }t |dk(r|dnKtjj|j|jdtjdd }t|||j |j!d |d d nr|j drd|ddj#} | j } | D] } d| vs|j!d| ddn|j$d"it'| n|j drt |j dk(r|j d} | |vrD|jg}|j)|| d|j+d|| ddnud| dt-|j/d} |j!| dnA|dk(r|j1|n)d}|j!d |d!d|j|||fS)#z Handles all user commands except for `!exit`. May update the chat history (e.g. reset it) or the generation config (e.g. set a new flag). T!clear!help!savechat_%Y-%m-%d_%H-%M-%S.jsonfilenamerrChat saved to !greenrrZ!setr1N=(Invalid flag format, missing `=` after `;`. Please use the format `arg_1=value_1 arg_2=value_2 ...`.red!examplerrroler?Example * not found in list of available examples: .!statusrF'/' is not a valid command. Showing help message.r)new_chat_historyrrWrkrErsplitospathrFrr$r<strftime save_chatrr]rGrHrrYrDlistkeysr) r*rrrrr valid_command split_inputrnew_generate_flagsflag example_name example_errors r+handle_non_exit_user_commandsz"Chat.handle_non_exit_user_commandss  !#D$6$67D OO  7 "  "  " "7 +J4D4D4F0G!0K$**,K{#q(AWW\\$"2"2DMMU4==YlKmJnnsCtu  xdT]] K  ! !z'C7 ! S  " "6 *",AB!5!5!7 !3!9!9!; * Jd?))FtfMAA$ * J I 45G HI  " ": .3z7G7G7I3Ja3O%++-a0Lx'!,,Xl-CF-KL V8Nv8VWX|n,VW[\d\i\i\kWlVmmno%%=%F 9 $  " "& " 1"M  ! !:,6e'fns ! t  "]F**r-c Kt|j|j|j}|j t |j }|jd|j|j|j}t|j4d{}d} ||}d}|j|n|j}|dk(rnh|dk(r&t |j }|j \|dk(r|jr|jdrt|jd kr|j}t|d k(r|d nKt j"j%|j&|jd t)j*d d }t-|||j.|j1d|dd?|jdrd|ddj3} | j} | D]} d| vs|j1d| ddn|j4d,it7| |jdrt|jd k(r|jd } | |j8vrX|j g}|j|j8| d|j;d|j8| ddnd| dt=|j8j?d} |j1| dng|d k(r|jA|!|jdr)|j1d"|d#d|j|j;d|d|jC|jd$} |jE|d|j| %}|jG|d{\}}|j;d&|d|d'k(r'|j1d(d)|jId*rd+}dddd{y77]#tJ$rY!wxYw7#1d{7swYyxYww)-N)r$r%r&T)rg)r&z!exitrrrrrrrrrrrrrrr1rrrrrrrrrrrrrrr)rrl)r.rl extra_body assistantlengthz2Generation stopped after reaching the token limit.yellowzContinue generating?u'Please continue. Do not repeat text.”r)&r#r$rr&rWrrrkrrrrYrUrErrrrrFrr<rr rr]rGrHrrrDr r rto_json_stringchat_completionrRrfKeyboardInterrupt)r*rrrclientpending_user_inputrr rrrrrrr. model_outputr5s r+rzChat._inner_runs!4==$))VZVcVcd  2 23 T*""4==1'?i i 6-1 f)5%7 -1*!44Z@%.__%6 "W,#x//0B0BC!) #w.!,,. #..w7C @P@P@RAQQ. RQ,RQ Q)&Q.(Q))Q.,R.R4Q7 5R<R)rrr__doc__rrrtyperArgumentr Optionr, staticmethodrr#dictr rrrrr-r+rr*s2+J78:OPV:W9X Y  TXYjhl AD'C5p!qqrD' $J,ef f D' " I  ENN[    D'& $J ELLk l n 'D'.!t\U\\?O-P!PQ/D'0sLELL6T$UUV1D'2!t\U\\?c-d!de3D'6% $J ELLg   7D'B CD'L H+H+!H+sDcN*+ H+ ! H+ 4j H+ tDz++ ,H+Ttr-rrr/c| tSd|vrTtjj|}tjj |}tj ||Stj |S)Nr)r rrdirnamebasenamefrom_pretrained)rr%rs r+rrGsg !!##''//"3477##$56//BB//0ABBr-rc |t|dk(riS|Dcic]/}d|jddzdz|jdd1}}|jDcic]*\}}||jdvr|jn|,}}}|jDcic]\}}||dk(rdn|}}}dtd t fd }|jDcic]\}}|||sd|dn|}}}d j |jDcgc] \}}|d |c}}}d |zdz}|jdd}|jdd}|jdd}|jdd}|jdd}|jdd} tj|}|Scc}wcc}}wcc}}wcc}}wcc}}w#tj$r tdwxYw)zUParses the generate flags from the user input into a dictionary of `generate` kwargs.r"rr)truefalseNonenullsr/cf|jd}|jdddjS)N-rr4r) removeprefixreplaceisdigit)r.s r+ is_numberz'parse_generate_flags..is_numberes, NN3 yyb!$,,..r-z, z: {}z"null"z"true"r*z"false"r+z"[[z]"r\rvzFailed to convert `generate_flags` into a valid JSON object. `generate_flags` = {generate_flags} Converted JSON string = {generate_flags_string}) rrrrcrrrFr2rorJSONDecodeErrorr)rrgenerate_flags_as_dictkvr4generate_flags_stringprocessed_generate_flagss r+rrSsC^!4!9 ^llUYcDJJsOA$66PPll KaJfJfJhBF!Q %661779A=KaJfJfJhi$!Qa1;A=ii/S/T/ RhQmQmQopAa11QCq1Dpp !II?U?[?[?]&^tq!!Bqc{&^_ "77#=299(FK199(FK199)WM199$D199$D299#sC #'::.C#D  $#Wmjq '_"     @   s)4G /G%G GG )GG<rc|rd|dgSgS)z Returns a new chat conversation.systemrr)rs r+rrs=JX- 8 9RPRRr-rrrc tjtjj|dt |d5}t j ||d|ddddtjj|S#1swY(xYw)z!Saves the chat history to a file.T)exist_okw)r chat_historyr1)indentN)rmakedirsrr%rrodumpabspath)rrrrs r+r r sjKK)D9 h M x>!LM 77??8 $$MMs BB ctjdk(rtjSt j tj jS)z)Returns the username of the current user.r )platformr?rgetloginpwdgetpwuidgetuidpw_namerr-r+rrs8I%{{}||BIIK(000r-__main__z meta-llama/Llama-3.2-3b-Instruct)r$r()ErrorrIrAstringr<collections.abcrtypingrr urllib.parserrrrrrhuggingface_hubrr transformersr transformers.utilsr readlinerr?rKrichr rich.consoler rich.liver rich.markdownr rich.progressrrrrr rich.textrrset ascii_letters whitespaceALLOWED_KEY_CHARSdigitsALLOWED_VALUE_CHARSrrirFr rjr#rrrr r#rrr rrrr-r+rds  )!*  L)0  8?? !$&``%0$?,,v/@/@@A 6==(6+<+<?@A &}}@ZZz CcDj C=M C3$cT)93$d3$lSC$JS$t*S %%4:%%#%1c1 z45s  sFF! F!