i<RdZddlZddlZddlmZddlmZddlmZddl m Z e r4ddl m Z m Z dd lmZmZmZdd lmZdd lmZmZmZdd lmZdd lmZdd lmZddlmZmZmZm Z m!Z!er ddl"m#Z#m$Z$m%Z%m&Z&GddedZ'hdZ(ejRe*Z+GddeZ,y)z Handler for the /v1/chat/completions endpoint. Supports streaming (SSE via DirectStreamer) and non-streaming (JSON) responses. N)AsyncGenerator) TYPE_CHECKING)logging)is_serve_available) JSONResponseStreamingResponse)ChatCompletionChatCompletionMessageChatCompletionMessageToolCall)Choice)ChatCompletionChunk ChoiceDeltaChoiceDeltaToolCall)CompletionCreateParamsStreaming)CompletionUsage)BaseGenerateManager BaseHandlerToolCallParser _StreamErrordetect_tool_format)GenerationConfigPreTrainedModelPreTrainedTokenizerFastProcessorMixinc"eZdZUeed<eed<y)+TransformersCompletionCreateParamsStreaminggeneration_configseedN)__name__ __module__ __qualname__str__annotations__inty/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/cli/serving/chat_completion.pyrr3s  Ir(rF)total>nuseraudiostorelogprobsmetadata functions modalities prediction tool_choice service_tier top_logprobs function_callstream_optionsresponse_formatpresence_penaltyreasoning_effortweb_search_optionsparallel_tool_callsmax_completion_tokenscXeZdZdZeZeZdede de e zfdZ dde ddd d d e d ed dde dedzde fdZ dde ddd d d e d ed dde dedzde fdZddedddeffd Z dde de d e de dedzdeedzdefdZ d de de dzde dzde dzde dzdedzdedzde fdZxZS)!ChatCompletionHandlerz|Handler for the `/v1/chat/completions` endpoint. Supports both streaming (SSE) and non-streaming (JSON) responses. body request_idreturnc .K|j||j|\}}}|jj||}|jj ||}t jd|d||jj||}|j|d|} |j| d|jd|rdnd dd } |s| j|j} |j||j|} |r|j!|| |jdr t#|nd} |jd } | r|j%||||| | || S|j'||||| | || d{S7w) aValidate the request, load the model, and dispatch to streaming or non-streaming. Args: body (`dict`): The raw JSON request body (OpenAI chat completion format). request_id (`str`): Unique request identifier (from header or auto-generated). Returns: `StreamingResponse | JSONResponse`: SSE stream or JSON depending on ``body["stream"]``. ) processorz[Request received] Model: z, CB: use_cbmessagesTtoolsNpt)add_generation_promptrIreturn_tensors return_dicttokenizestream) gen_manager tool_format)_validate_request_resolve_model model_managerget_model_modalitygeneration_stateuse_continuous_batchingloggerwarning get_manager"get_processor_inputs_from_messagesapply_chat_templategettodevice_build_generation_configrinit_cbr _streaming_non_streaming)selfrArBmodel_idmodelrEmodalityrGrPprocessor_inputsinputs gen_configrQ streamings r)handle_requestz$ChatCompletionHandler.handle_request_s t$%)%8%8%>"%%%88)8T&&>>uhO3H:VF8LM++777P BB4 CSU]^.. "&((7##)4t / YYu||,F2249P9PY_2`    z 2 4888G3D(/$ HHX& ??''#  ,,''-    sF FFFNrfrrEz(ProcessorMixin | PreTrainedTokenizerFastrerirjrrPrQc  |j|||\ |d} t| tr t| n| jd |r t |nd dt tdff  fd } t| dS) z(Stream tokens as SSE via DirectStreamer.rB input_idsNrCc Kd} jd d}|s=jd{}|g} |jj jduxrj jk\}|rd }n|rd}nd}t! j jz} j| | y7#tj $rYnwxYwg}|D]}|d}nt |tr5|jd|jddj|yd|i} F j|x}3|tjurzd}d td d d |d |ddgi}|jjfd i||rdj||sn#t"tj$f$rj'wxYww)NF assistant)rolerfTzdata: {"error": "z"} content tool_callsrfunction _tool_callname argumentsryrz)indextypeidrwrflengthstop prompt_tokenscompletion_tokens total_tokens) finish_reasonrfusage)_build_chunk_sser]append get_nowaitasyncio QueueEmpty isinstancermsgjoinfeedrCONSUMEDrmax_new_tokensrr GeneratorExitCancelledErrorcancel)has_tool_callsdonetextbatch sse_parts chunk_kwargsresulthit_maxrrrj input_lenreparserqueuerBrdstreamers r)sse_genz1ChatCompletionHandler._streaming..sse_gensT"NC ++J[PX+YY!&,D!FE"!LL)9)9);<#J%334?vHDYDY]g]v]vDv!$0M$,M$*M'"+&.&;&;!*X-B-B!B ++"/" ,i- #--,.I %l<#'D!%dL9%,,/A$((7-ST"$'')"44")24'8 !-V[[=N3N62[%)@)@@ (-1N ,$7./-7.8\+D:@.W]^iWj1k %&/" ,L"(()>)>)>z)jQY)j]i)jk5l8! ggi00Ov"7#9#9: !  s^G;.G CG !CA2G G;G C*'G )C**AG >G;?B G +G88G;ztext/event-stream media_type) generate_streamingrlistlenshaperrr$r )rdrBrfrErerirjrPrQrorrrrrs`` ` ` @@@@r)rbz ChatCompletionHandler._streamings&88 6S]jt8ux;' &0D&AC NyWYGZ 0; ,E ~c4i8E E N!7JKKr(c K|j|||||d{\} } } |jduxrt| |jk\} t| } t| | | | z}d}|?t j | |}|'|Dcgc]}t |dd|d|dd }}|d }n| rd }nd }t|j|| |||| dS7cc}ww)z)Run generation and return a JSONResponse.rnNrrxrwryrzr{)r~r}rwrvrr)rrrvzapplication/jsonr) generate_non_streamingrrrrparser r_build_completion)rdrBrfrErerirjrPrQrur generated_idsrrrrvparsedtcrs r)rcz$ChatCompletionHandler._non_streamingsE3>2T2T 9fjZ3U3 - )M++47kC YL  YL  YL'YL)YLD[YL YLL$(7 7 !7 > 7  7  7 '7 )7 D[7  7 v!T!L^!hl!0)-(,%4%4%4 %4  %4 % %4J%%4 %4R" $("&(,'('(t'(Tz '( Dj '( Tz '(4K'(%'( '(r(r@)-rrrcollections.abcrtypingrutilsrutils.import_utilsrfastapi.responsesrr openai.types.chatr r r !openai.types.chat.chat_completionr 'openai.types.chat.chat_completion_chunkrrrr*openai.types.chat.completion_create_paramsropenai.types.completion_usagerrrrrr transformersrrrrrr get_loggerr!rXr@r'r(r)rs  * 4Aff8mmMZ=gg2QY^!0   H %D(KD(r(