Module core.evaluator

Functions

def init_driver() ‑> selenium.webdriver.remote.webdriver.WebDriver

Classes

class Evaluator

Helper class that provides a standard way to create an ABC using inheritance.

Ancestors

  • abc.ABC

Subclasses

Methods

def compare(self, results: Dict[str, pandas.core.frame.DataFrame], metrics: list) ‑> matplotlib.figure.Figure
def evaluate(self, dataset: pandas.core.frame.DataFrame) ‑> pandas.core.frame.DataFrame
class LLMEvaluator

Helper class that provides a standard way to create an ABC using inheritance.

Ancestors

Methods

def compare(self, results: Dict[str, pandas.core.frame.DataFrame], metrics: list = ['precision', 'recall', 'time']) ‑> matplotlib.figure.Figure
def evaluate(self, navigation_engine: lavague.core.navigation.NavigationEngine, retrieved_dataset: pandas.core.frame.DataFrame, csv_out_name: str, max_retry: int = 1, eval_mode: Literal['exec', 'json', 'yaml', 'json_old'] = 'yaml') ‑> pandas.core.frame.DataFrame
class RetrieverEvaluator

Helper class that provides a standard way to create an ABC using inheritance.

Ancestors

Methods

def compare(self, results: Dict[str, pandas.core.frame.DataFrame], metrics: list = ['precision', 'recall', 'time']) ‑> matplotlib.figure.Figure
def evaluate(self, retriever: lavague.core.retrievers.BaseHtmlRetriever, rephrased_dataset: pandas.core.frame.DataFrame, csv_out_name: str) ‑> pandas.core.frame.DataFrame
def rephrase_dataset(self, dataset: pandas.core.frame.DataFrame, csv_out_name: str, llm: llama_index.core.llms.llm.LLM = None) ‑> pandas.core.frame.DataFrame
class SeleniumDriverForEval (url: Optional[str] = None, get_selenium_driver: Optional[Callable[[], selenium.webdriver.remote.webdriver.WebDriver]] = None, headless: bool = True, user_data_dir: Optional[str] = None, width: int = 1080, height: int = 1080, no_load_strategy: bool = False, options: Optional[selenium.webdriver.chrome.options.Options] = None, driver: Optional[selenium.webdriver.remote.webdriver.WebDriver] = None)

Helper class that provides a standard way to create an ABC using inheritance.

Init the driver with the init funtion, and then go to the desired url

Ancestors

  • lavague.drivers.selenium.base.SeleniumDriver
  • lavague.core.base_driver.BaseDriver
  • abc.ABC