i-ddlmZddlmZddlZddlmZddlmZddlm Z eje Z ejZejZdZej$ej$ej&ej&ej(ej(ej*ej*ej,ej,ej.ej.ejejej0ej0ej2ejej4ej0i ZdZd Zeedfd Zeed fd Zeedd fd Z eedfdZ!dej(fdZ"y))fuzz)utilsN)process)partialc |tjtjtjtjtj tj tjtjfvrS|tjtjfv}ttj|rtjk(rSfd}|S)z thefuzz runs both the default preprocessing of the function and the preprocessing function passed into process.* while rapidfuzz only runs the one passed into process.*. This function wraps the processor to mimic this behavior ) force_asciic |SN)s pre_processor processors `/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/thefuzz/process.pywrapperz_get_processor..wrapper!sYq\**) rWRatioQRatiotoken_set_ratiotoken_sort_ratiopartial_token_set_ratiopartial_token_sort_ratioUWRatioUQRatiorr full_process)rscorerr rrs` @r_get_processorrs dkk4;;**D,A,A22D4Q4QllDLL22t|| <.wrapper@sb"~r)r)_scorer_loweringget)rrs` r _get_scorerr%;s     00rcl|r||n|}t|dk(rtjd|d|S)NrzcApplied processor reduces input query to empty string, all comparisons will have score 0. [Query: 'z'])len_loggerwarning)queryrprocessed_querys r_preprocess_queryr,FsB*3i&O ?q %%*G30 1 rc#Kt|d}|tv}t||}tj||t ||t ||}|D])\}} } |rtt| } |r|| | fn|| f+yw)a  Select the best match in a list or dictionary of choices. Find best matches in a list or dictionary of choices, return a generator of tuples containing the match and its score. If a dictionary is used, also returns the key for each match. Arguments: query: An object representing the thing we want to find. choices: An iterable or dictionary-like object containing choices to be matched against the query. Dictionary arguments of {key: value} pairs will attempt to match the query against each value. processor: Optional function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. This can be used to match against, say, the first element of a list: lambda x: x[0] Defaults to thefuzz.utils.full_process(). scorer: Optional function for scoring matches between the query and an individual processed choice. This should be a function of the form f(query, choice) -> int. By default, fuzz.WRatio() is used and expects both query and choice to be strings. score_cutoff: Optional argument for score threshold. No matches with a score less than this number will be returned. Defaults to 0. Returns: Generator of tuples containing the match and its score. If a list is used for choices, then the result will be 2-tuples. If a dictionary is used, then the result will be 3-tuples containing the key for each match. For example, searching for 'bird' in the dictionary {'bard': 'train', 'dog': 'man'} may return ('train', 22, 'bard'), ('man', 0, 'dog') itemsrrr"N) hasattrr#r,rprocess extract_iterrr%intround) r*choicesrrr" is_mapping is_lowereditchoicescorekeys rextractWithoutOrderr<Ps^'*J++J eY /E    w F36"!  B!Fs e %E&0vuc"vuoE FsA=A?c"t|||||S)a Select the best match in a list or dictionary of choices. Find best matches in a list or dictionary of choices, return a list of tuples containing the match and its score. If a dictionary is used, also returns the key for each match. Arguments: query: An object representing the thing we want to find. choices: An iterable or dictionary-like object containing choices to be matched against the query. Dictionary arguments of {key: value} pairs will attempt to match the query against each value. processor: Optional function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. This can be used to match against, say, the first element of a list: lambda x: x[0] Defaults to thefuzz.utils.full_process(). scorer: Optional function for scoring matches between the query and an individual processed choice. This should be a function of the form f(query, choice) -> int. By default, fuzz.WRatio() is used and expects both query and choice to be strings. limit: Optional maximum for the number of elements returned. Defaults to 5. Returns: List of tuples containing the match and its score. If a list is used for choices, then the result will be 2-tuples. If a dictionary is used, then the result will be 3-tuples containing the key for each match. For example, searching for 'bird' in the dictionary {'bard': 'train', 'dog': 'man'} may return [('train', 22, 'bard'), ('man', 0, 'dog')] )rrlimit) extractBests)r*r5rrr?s rextractrAs\ w)FRW XXrct|d}|tv}t||}tj||t ||t |||}t|D]-\} \} } } |rtt| } |r| | | fn| | f|| </|S)a Get a list of the best matches to a collection of choices. Convenience function for getting the choices with best scores. Args: query: A string to match against choices: A list or dictionary of choices, suitable for use with extract(). processor: Optional function for transforming choices before matching. See extract(). scorer: Scoring function for extract(). score_cutoff: Optional argument for score threshold. No matches with a score less than this number will be returned. Defaults to 0. limit: Optional maximum for the number of elements returned. Defaults to 5. Returns: A a list of (match, score) tuples. r.)rrr"r?) r0r#r,r1rArr% enumerater3r4) r*r5rrr"r?r6r7resultsir9r:r;s rr@r@s('*J++J eY /E w F36"! G$-W#5M FE3 e %E-7feS)fe_ M Nrct|d}|tv}t||}tj||t ||t ||}||S|\}} } |rtt| } |r|| | fS|| fS)ad Find the single best match above a score in a list of choices. This is a convenience method which returns the single best choice. See extract() for the full arguments list. Args: query: A string to match against choices: A list or dictionary of choices, suitable for use with extract(). processor: Optional function for transforming choices before matching. See extract(). scorer: Scoring function for extract(). score_cutoff: Optional argument for score threshold. If the best match is found, but it is not greater than this number, then return None anyway ("not a good enough match"). Defaults to 0. Returns: A tuple containing a single match and its score, if a match was found that was above score_cutoff. Otherwise, returns None. r.r/) r0r#r,r1 extractOnerr%r3r4) r*r5rrr"r6r7resr9r:r;s rrGrGs,'*J++J eY /E    w F36"!  C { FE3E%L!#-FE3 BFE?BrFct}|D]2}t||||d}|jt|dd4t |t |k7r t |S|S)a This convenience function takes a list of strings containing duplicates and uses fuzzy matching to identify and remove duplicates. Specifically, it uses process.extract to identify duplicates that score greater than a user defined threshold. Then, it looks for the longest item in the duplicate list since we assume this item contains the most entity information and returns that. It breaks string length ties on an alphabetical sort. Note: as the threshold DECREASES the number of duplicates that are found INCREASES. This means that the returned deduplicated list will likely be shorter. Raise the threshold for dedupe to be less sensitive. Args: contains_dupes: A list of strings that we would like to dedupe. threshold: the numerical value (0,100) point at which we expect to find duplicates. Defaults to 70 out of 100 scorer: Optional function for scoring matches between the query and an individual processed choice. This should be a function of the form f(query, choice) -> int. By default, fuzz.token_set_ratio() is used and expects both query and choice to be strings. Returns: A deduplicated list. For example: In: contains_dupes = ['Frodo Baggin', 'Frodo Baggins', 'F. Baggins', 'Samwise G.', 'Gandalf', 'Bilbo Baggins'] In: dedupe(contains_dupes) Out: ['Frodo Baggins', 'Samwise G.', 'Bilbo Baggins', 'Gandalf'] N)rr"r?c(t|d|dfS)Nr)r')xs rzdedupe..7sAaD 1Q4/@r)r;r)setr@addmaxr'list)contains_dupes thresholdrdedupeditemmatchess rdeduperWsj:eGFt^FQZbfg C%@A!DEF LC,??4=S^Sr)#rrlogging rapidfuzzrfuzzrr1 functoolsr getLogger__name__r(rdefault_scorerrdefault_processorrratio partial_ratiorrrrrrrr#r%r,r<rAr@rGrWr rrrcsG#) '  H %&&> JJ ++%//511  %"?"?!!5#A#AKKKKLL%,,LL%,, 13DNij>FB'8VW.Yb,=^bckl&R*;>`a)CX&(0D0D"Tr