tbp.monty.frameworks.loggers

tbp.monty.frameworks.loggers.exp_logger

class BaseMontyLogger(handlers)

Bases: object

Basic logger that logs or saves information when logging is called.

__init__(handlers)

close(logger_args, output_dir, model)

flush()

post_episode(logger_args, output_dir, model)

post_epoch(logger_args, output_dir, model)

post_eval(logger_args, output_dir, model)

post_step(logger_args, output_dir, model)

post_train(logger_args, output_dir, model)

pre_episode(logger_args, output_dir, model)

pre_epoch(logger_args, output_dir, model)

pre_eval(logger_args, output_dir, model)

pre_step(logger_args, output_dir, model)

pre_train(logger_args, output_dir, model)

class LoggingCallbackHandler(loggers, model, output_dir)

Bases: object

Calls a list of loggers on an event (e.g., post_train).

Each logger receives: logger_args: dict with time stamps (steps, epochs, etc.) and

env_interface.primary_target which contains object id and pose

output_dir: Full path of the directory to store log files

Note

This logger handler is intended primarily for logging.

__init__(loggers, model, output_dir)

call_event(event, logger_args)

close(logger_args)

post_episode(logger_args)

post_epoch(logger_args)

post_eval(logger_args)

post_step(logger_args)

post_train(logger_args)

pre_episode(logger_args)

pre_epoch(logger_args)

pre_eval(logger_args)

pre_step(logger_args)

pre_train(logger_args)

property logger_list

class TestLogger(handlers)

Bases: BaseMontyLogger

__init__(handlers)

close(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

post_epoch(logger_args, output_dir, model)

post_eval(logger_args, output_dir, model)

post_train(logger_args, output_dir, model)

pre_episode(logger_args, output_dir, model)

pre_epoch(logger_args, output_dir, model)

pre_eval(logger_args, output_dir, model)

pre_train(logger_args, output_dir, model)

tbp.monty.frameworks.loggers.graph_matching_loggers

class BasicGraphMatchingLogger(handlers)

Bases: BaseMontyLogger

Basic logger that logs or saves information when logging is called.

__init__(handlers)

Initialize logger.

flush()

get_formatted_overall_stats(mode, episode)

log_episode(logger_args, output_dir, model)

maybe_log(logger_args, output_dir, model)

Left here in case we go back to size based logging.

Remove if not used after code has stabilized.

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Run get_stats_per_lm and add to overall stats.

Store stats ~

1 (episode)

lm_0 (which lm)

stats

update_overall_stats(mode: ExperimentMode, episode, episode_steps, monty_matching_steps) → None

Update overall run stats for mode.

Return type:

None

class DetailedGraphMatchingLogger(handlers)

Bases: BasicGraphMatchingLogger

Log detailed stats as .json file by saving data for each LM and SM.

__init__(handlers)

Initialize stats dicts.

log_episode(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Add episode data to overall buffer_data dict.

Store stats ~

1 (episode)

lm_0 (which lm)

stats

class SelectiveEvidenceLogger(handlers)

Bases: BasicGraphMatchingLogger

Log evidences as .json file by saving data for each LM and SM.

This is slimmed down to only log data needed for object similarity analysis. Data logged:

• evidences for each object and pose at the end of an episode

• first frame of the view finder

__init__(handlers)

Initialize stats dicts.

log_episode(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Add episode data to overall buffer_data dict.

tbp.monty.frameworks.loggers.monty_handlers

class BasicCSVStatsHandler

Bases: MontyHandler

Grab any logs at the BASIC level and append to train or eval CSV files.

__init__()

Initialize with empty dictionary to keep track of writes per file.

We only want to include the header the first time we write to a file. This keeps track of writes per file so we can format the file properly.

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

move_columns_to_front(df, columns)

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class DetailedJSONHandler(detailed_episodes_to_save: Container[int] | Literal['all'] = 'all', detailed_save_per_episode: bool = False) → None

Bases: MontyHandler

Grab any logs at the DETAILED level and append to a json file.

__init__(detailed_episodes_to_save: Container[int] | Literal['all'] = 'all', detailed_save_per_episode: bool = False) → None

Initialize the DetailedJSONHandler.

Parameters:

• detailed_episodes_to_save (Container[int] | Literal[‘all’]) – Container of episodes to save or the string "all" (default) to include every episode.

• detailed_save_per_episode (bool) – Whether to save individual episode files or consolidate into a single detailed_run_stats.json file. Defaults to False.

close()

get_detailed_stats(data, global_episode_id: int, local_episode: int, mode: ExperimentMode) → dict

Get detailed episode stats.

Returns:

Episode stats.

Return type:

stats

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, local_episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

Report episode data.

class MontyHandler

Bases: object

abstract close()

abstract classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

abstract report_episode(**kwargs)

tbp.monty.frameworks.loggers.wandb_handlers

class BasicWandbChartStatsHandler

Bases: WandbHandler

Log LM episode stats to wandb with one chart per measure.

get_safe_columns_per_lm(stats)

Format each episode by looping over learning modules and formatting each one.

Parameters:

stats – dict ~ {LM_0: dict, LM_1: dict}

Returns:

The formatted stats.

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class BasicWandbTableStatsHandler

Bases: WandbHandler

Log LM episode stats to wandb as tables.

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class DetailedWandbHandler

Bases: WandbHandler

Make animations from sequences of observations on wandb.

NOTE: Not yet generalized for different model architectures. This assumes SM_0 is the patch and SM_1 is the view finder.

get_episode_frames(episode_stats)

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initialization for subclasses not covered by WandbHandler.

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class DetailedWandbMarkedObsHandler

Bases: DetailedWandbHandler

Just like DetailedWandbHandler, but use fancier observations.

NOTE: This assumes SM_1 and SM_0 are the view finder and patch modules respectively,

meaning this logger is specific to the model architecture.

NOTE: This is slow, adding a few seconds per function call. The intended use

case is for debugging and error analysis, so speed should not be an issue when the number of episodes is small, but probably do not use this if you are running a large number of experiments.

get_episode_frames(episode_stats)

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initialization for subclasses not covered by WandbHandler.

class DetailedWandbTableStatsHandler

Bases: BasicWandbTableStatsHandler

Log LM stats and actions to wandb as tables.

This modified version of BasicWandbTableStatsHandler also logs the actions executed in each episode to wandb as tables (one table per episode).

__init__()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class WandbHandler

Bases: MontyHandler

Parent class for wandb loggers.

__init__()

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initialization for subclasses not covered by WandbHandler.

report_episode(data, output_dir, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)

class WandbWrapper(wandb_handlers: list, run_name: str, wandb_group: str | None = None, config: dict | None = None, resume_wandb_run: bool = False, wandb_id: str | None = None)

Bases: MontyHandler

Container for wandb handlers.

Loops over a series of handlers which log different information without committing (sending it to wandb).

The wrapper finally commits all logs at once. This allows us to maintain control over the wandb global step. This class assumes reporting takes place once per episode; hence the wandb handlers have report_episode methods.

__init__(wandb_handlers: list, run_name: str, wandb_group: str | None = None, config: dict | None = None, resume_wandb_run: bool = False, wandb_id: str | None = None)

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode: ExperimentMode = ExperimentMode.TRAIN, **kwargs)