machetli.evaluator

Machetli evaluators are Python scripts that are started with the path to a file that represents a state in Machetli’s search. They check if a certain behavior occurs for that input and communicate this back to the search with their exit code. The user documentation contains more information on how to write an evaluator. This file defines the exit codes and offers a general convenience function for implementing evaluators. Additional convenience functions come with specific packages.

machetli.evaluator.EXIT_CODE_BEHAVIOR_NOT_PRESENT = 33

Exit code returned by an evaluator if the given state does not exhibit the behavior that the evaluator is checking for.

machetli.evaluator.EXIT_CODE_BEHAVIOR_PRESENT = 42

Exit code returned by an evaluator if the given state exhibits the behavior that the evaluator is checking for.

machetli.evaluator.EXIT_CODE_CRITICAL = 35

Exit code returned by an evaluator if a critical error occurred. Any recognized exit code is also interpreted as a critical error. In particular, 0 is treated as an error, because it means that the evaluator completed without communicating a result.

machetli.evaluator.EXIT_CODE_RESOURCE_LIMIT = 34

Exit code returned by an evaluator if it ran out of time or memory while checking the state.

machetli.evaluator.run_evaluator(evaluate)[source]

Load the state passed to the script via its only command line arguments, then run the given function evaluate on it, and exit the program with the appropriate exit code. If the function returns True, use EXIT_CODE_BEHAVIOR_PRESENT, otherwise use EXIT_CODE_BEHAVIOR_NOT_PRESENT.

This function is meant to be used as the main function of an evaluator script. Package-specific overloads are available for more convenient evaluation functions and for testing the evaluator.

Parameters

evaluate – is a function taking the filename of a state as input and returning True if the specified behavior occurs for the given instance, and False if it doesn’t. Other ways of exiting the function (exceptions, sys.exit with exit codes other than EXIT_CODE_BEHAVIOR_PRESENT or EXIT_CODE_BEHAVIOR_NOT_PRESENT) are treated as failed evaluations by the search.