flexmeasures.data.services.generator_results

Checking and saving what a data generator returns, in one way for every kind of data generator.

A data generator, such as a scheduler, a reporter, a forecaster or a plugin’s automation type, returns its results as dicts naming the “sensor” to record on and holding the “data” to record there. Whichever kind of generator it is, and whether it runs in a worker or from the CLI, its results are checked against the sensors it may record on and saved the same way here.

Functions

flexmeasures.data.services.generator_results.check_generator_results(results: list[dict], permitted_sensor_ids: set[int] | None, generator: str, automation_id: int | None = None) → None

Refuse results for any sensor outside the permitted ones, judging the whole set before any of it is saved or handed back.

A result without a sensor, such as a scheduler’s own bookkeeping, records on no sensor, so it is not judged. Pass None for permitted_sensor_ids where no such check applies, as on the CLI, whose user is trusted.

Raises:

GeneratorWritesUncheckedSensor – naming the generator, the refused sensors and the permitted ones.

flexmeasures.data.services.generator_results.check_results_of_current_job(results: list[dict], generator) → None

Refuse results for any sensor outside those the automation behind the current job was checked against.

Outside a job, or in a job that no automation queued, nothing is checked, as for every caller of check_generator_results.

Raises:

GeneratorWritesUncheckedSensor – naming the generator, the automation, the refused sensors and the permitted ones.

flexmeasures.data.services.generator_results.describe_generator(generator) → str

Name a data generator for a message: its class, and its data source if it already has one.

The data source is only named when the generator already holds it, since asking for it could create one.

flexmeasures.data.services.generator_results.save_generator_results(results: list[dict]) → list[dict]

Save what a data generator returns, all of it or none of it, and say how many beliefs each result saved.

The results are saved within a savepoint, so a save that fails halfway leaves none of them staged, while whatever the caller staged before stays as it was. Committing is left to the caller, as it is for save_to_db, so that the results can be part of a larger transaction. Every result saved must name its sensor, unlike for the check: a result without one, such as a scheduler’s own bookkeeping, holds no beliefs, so the caller leaves it out, as make_schedule does. A belief that repeats the belief right before it is not saved again, as for any other data, which also makes saving results a generator already saved itself cost nothing extra.

Returns:

per result, the sensor id and the number of beliefs saved (“n_rows”), which leaves out NaN values and beliefs that were already on record.

flexmeasures.data.services.generator_results.save_results_not_yet_saved(results: list[dict]) → None

Save the results that do not say how many of their beliefs were saved, and record that number on each of them as “n_saved”.

A generator that saves its results as it goes, such as the built-in forecasting pipeline, says so with “n_saved” on each result, and is not saved again. One that only returns its results has them saved here, and one that saved them itself anyway costs nothing extra, since its beliefs are then already on record. Committing is left to the caller.

Exceptions

exception flexmeasures.data.services.generator_results.GeneratorWritesUncheckedSensor(generator: str, refused_sensor_ids: Iterable[int], permitted_sensor_ids: Iterable[int], automation_id: int | None = None)

Raised when a data generator returns results for a sensor that nobody’s permissions were checked against.

It is a refusal rather than a passing failure: the same results are refused again on every retry, until the generator’s configuration or the sensors it was checked against change. It derives from Exception rather than PermissionError, because PermissionError is the operating system’s (an OSError), which the same jobs raise when they cannot write a file. The facts are kept as attributes, so that a handler need not read them from the message.