String evaluator interface.
Grade, tag, or otherwise evaluate predictions relative to their inputs and/or reference labels.
Compute a regex match between the prediction and the reference.
evaluator = RegexMatchStringEvaluator(flags=re.IGNORECASE) evaluator.evaluate_strings( prediction="Mindy is the CTO", reference="^mindy.*cto$", ) # This will return {'score': 1.0} due to the IGNORECASE flag
evaluator = RegexMatchStringEvaluator() evaluator.evaluate_strings( prediction="Mindy is the CTO", reference="^Mike.*CEO$", ) # This will return {'score': 0.0}
evaluator.evaluate_strings( prediction="Mindy is the CTO", reference="^Mike.*CEO$|^Mindy.*CTO$", ) # This will return {'score': 1.0} as the prediction matches the second pattern in the union
