scandeval.model_cache

source module scandeval.model_cache

ModelCache class for caching model outputs.

Classes

• ModelCache — A cache for model outputs.

Functions

• split_dataset_into_cached_and_non_cached — Split a dataset into a cached and non-cached part.

• load_cached_model_outputs — Load the cached model outputs.

source class ModelCache(cache_name: str, max_generated_tokens: int)

A cache for model outputs.

Initialize the model output cache.

Attributes

• model_cache_dir —

  The directory to store the cache in.

• cache_path —

  The path to the cache file.

• cache —

  The model output cache.

• max_generated_tokens —

  The maximum number of tokens to generate for each example.

Parameters

• model_cache_dir : Path —

  The directory to store the cache in.

• cache_name : str —

  The name of the cache file.

• max_generated_tokens : int —

  The maximum number of tokens to generate for each example.

Methods

• load — Load the model output cache.

• save — Save the model output cache to disk.

• remove — Remove the cache from memory and delete it from disk.

• add_to_cache — Add the model input/output to the cache.

source method ModelCache.load() → None

Load the model output cache.

source method ModelCache.save() → None

Save the model output cache to disk.

source method ModelCache.remove() → None

Remove the cache from memory and delete it from disk.

source method ModelCache.add_to_cache(model_inputs: dict, model_output: GenerativeModelOutput) → None

Add the model input/output to the cache.

Parameters

• model_inputs : dict —

  The model inputs.

• model_output : GenerativeModelOutput —

  The model output.

source split_dataset_into_cached_and_non_cached(dataset: Dataset, cache: ModelCache) → tuple[Dataset, Dataset]

Split a dataset into a cached and non-cached part.

Parameters

• dataset : Dataset —

  The dataset to split.

• cache : ModelCache —

  The model output cache.

Returns

• tuple[Dataset, Dataset] — The cached and non-cached parts of the dataset.

source load_cached_model_outputs(cached_dataset: Dataset, cache: ModelCache) → GenerativeModelOutput

Load the cached model outputs.

Parameters

• cached_dataset : Dataset —

  The dataset containing the cached examples.

• cache : ModelCache —

  The model output cache.

Returns

• GenerativeModelOutput — The model output containing the cached sequences.