tbp.monty.frameworks.loggers

tbp.monty.frameworks.loggers.exp_logger

class BaseMontyLogger(handlers)

Bases: object

Basic logger that logs or saves information when logging is called.

__init__(handlers)

close(logger_args, output_dir, model)

flush()

post_episode(logger_args, output_dir, model)

post_epoch(logger_args, output_dir, model)

post_eval(logger_args, output_dir, model)

post_step(logger_args, output_dir, model)

post_train(logger_args, output_dir, model)

pre_episode(logger_args, output_dir, model)

pre_epoch(logger_args, output_dir, model)

pre_eval(logger_args, output_dir, model)

pre_step(logger_args, output_dir, model)

pre_train(logger_args, output_dir, model)

class LoggingCallbackHandler(loggers, model, output_dir)

Bases: object

Calls a list of loggers on an event (eg post_train).

Each logger receives: logger_args: dict with time stamps (steps, epochs, etc.) and

env_interface.primary_target which contains object id and pose

output_dir: Full path of the directory to store log files

Note

This logger handler is intended primarily for logging

__init__(loggers, model, output_dir)

call_event(event, logger_args)

close(logger_args)

post_episode(logger_args)

post_epoch(logger_args)

post_eval(logger_args)

post_step(logger_args)

post_train(logger_args)

pre_episode(logger_args)

pre_epoch(logger_args)

pre_eval(logger_args)

pre_step(logger_args)

pre_train(logger_args)

property logger_list

class TestLogger(handlers)

Bases: BaseMontyLogger

__init__(handlers)

close(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

post_epoch(logger_args, output_dir, model)

post_eval(logger_args, output_dir, model)

post_train(logger_args, output_dir, model)

pre_episode(logger_args, output_dir, model)

pre_epoch(logger_args, output_dir, model)

pre_eval(logger_args, output_dir, model)

pre_train(logger_args, output_dir, model)

tbp.monty.frameworks.loggers.graph_matching_loggers

class BasicGraphMatchingLogger(handlers)

Bases: BaseMontyLogger

Basic logger that logs or saves information when logging is called.

__init__(handlers)

Initialize logger.

flush()

get_formatted_overall_stats(mode, episode)

log_episode(logger_args, output_dir, model)

maybe_log(logger_args, output_dir, model)

Left here in case we go back to size based logging.

Remove if not used after code has stabilized.

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Run get_stats_per_lm and add to overall stats.

Store stats ~

1 (episode)

lm_0 (which lm)

stats

update_overall_stats(mode, episode, episode_steps, monty_matching_steps)

Update overall run stats for mode.

class DetailedGraphMatchingLogger(handlers)

Bases: BasicGraphMatchingLogger

Log detailed stats as .json file by saving data for each LM and SM.

__init__(handlers)

Initialize stats dicts.

log_episode(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Add episode data to overall buffer_data dict.

Store stats ~

1 (episode)

lm_0 (which lm)

stats

class SelectiveEvidenceLogger(handlers)

Bases: BasicGraphMatchingLogger

Log evidences as .json file by saving data for each LM and SM.

This is slimmed down to only log data needed for object similarity analysis. Data logged:

• evidences for each object and pose at the end of an episode

• first frame of the view finder

__init__(handlers)

Initialize stats dicts.

log_episode(logger_args, output_dir, model)

post_episode(logger_args, output_dir, model)

update_episode_data(logger_args, model)

Add episode data to overall buffer_data dict.

tbp.monty.frameworks.loggers.monty_handlers

class BasicCSVStatsHandler

Bases: MontyHandler

Grab any logs at the BASIC level and append to train or eval CSV files.

__init__()

Initialize with empty dictionary to keep track of writes per file.

We only want to include the header the first time we write to a file. This keeps track of writes per file so we can format the file properly.

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

move_columns_to_front(df, columns)

report_episode(data, output_dir, episode, mode='train', **kwargs)

class DetailedJSONHandler(detailed_episodes_to_save: Container[int] | Literal['all'] = 'all', detailed_save_per_episode: bool = False, episode_id_parallel: int | None = None) → None

Bases: MontyHandler

Grab any logs at the DETAILED level and append to a json file.

__init__(detailed_episodes_to_save: Container[int] | Literal['all'] = 'all', detailed_save_per_episode: bool = False, episode_id_parallel: int | None = None) → None

Initialize the DetailedJSONHandler.

Parameters:

• detailed_episodes_to_save (Container[int] | Literal[‘all’]) – Container of episodes to save or the string "all" (default) to include every episode.

• detailed_save_per_episode (bool) – Whether to save individual episode files or consolidate into a single detailed_run_stats.json file. Defaults to False.

• episode_id_parallel (int | None) – Episode id associated with current run, used to identify the episode when using run_parallel.

close()

get_detailed_stats(data, global_episode_id: int, local_episode: int, mode: Literal['train', 'eval']) → dict

Get detailed episode stats.

Returns:

Episode stats.

Return type:

stats

get_episode_id(local_episode, mode: Literal['train', 'eval'], **kwargs) → int

Get global episode id corresponding to a mode-local episode index.

This function is needed to determine correct episode id when using run_parallel.

Returns:

Combined train+eval episode id.

Return type:

global_episode_id

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, local_episode, mode='train', **kwargs)

Report episode data.

class MontyHandler

Bases: object

abstract close()

abstract classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

abstract report_episode(**kwargs)

class ReproduceEpisodeHandler

Bases: MontyHandler

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode='train', **kwargs)

tbp.monty.frameworks.loggers.wandb_handlers

class BasicWandbChartStatsHandler

Bases: WandbHandler

Log LM episode stats to wandb with one chart per measure.

get_safe_columns_per_lm(stats)

Format each episode by looping over learning modules and formatting each one.

Parameters:

stats – dict ~ {LM_0: dict, LM_1: dict}

Returns:

The formatted stats.

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode='train', **kwargs)

class BasicWandbTableStatsHandler

Bases: WandbHandler

Log LM episode stats to wandb as tables.

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode='train', **kwargs)

class DetailedWandbHandler

Bases: WandbHandler

Make animations from sequences of observations on wandb.

NOTE: not yet generalized for different model architectures. This assumes SM_0 is the patch, SM_1 is the view finder.

get_episode_frames(episode_stats)

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initializations for subclasses not covered by init of WandbHandler.

report_episode(data, output_dir, episode, mode='train', **kwargs)

class DetailedWandbMarkedObsHandler

Bases: DetailedWandbHandler

Just like DetailedWandbHandler, but use fancier observations.

NOTE: this assumes sm1 and sm0 are the view finder and patch modules respectively,

meaning this logger is specific to the model architecture

NOTE: this is slow, adding ~ a few seconds per function call. The intended use

case is for debugging and error analysis, so speed should not be an issue when the number of episodes is small. But probably do not use this fi you are running a large number of experiments.

get_episode_frames(episode_stats)

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initializations for subclasses not covered by init of WandbHandler.

class DetailedWandbTableStatsHandler

Bases: BasicWandbTableStatsHandler

Log LM stats and actions to wandb as tables.

This is a modified version of BasicWandbTableStatsHandler that, in addition to the stats, logs the actions exectuted in each episode to wandb as tables (one table per episode).

__init__()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode='train', **kwargs)

class WandbHandler

Bases: MontyHandler

Parent class for wandb loggers.

__init__()

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

post_init()

Handle additional initialization for subclasses.

Call this to handle any additional initializations for subclasses not covered by init of WandbHandler.

report_episode(data, output_dir, mode='train', **kwargs)

class WandbWrapper(wandb_handlers: list, run_name: str, wandb_group: str | None = None, config: dict | None = None, resume_wandb_run: bool = False, wandb_id: str | None = None)

Bases: MontyHandler

Container for wandb handlers.

Loops over a series of handlers which log different information without commiting (sending it to wandb).

The wrapper finally commits all logs at once. This allows us to maintain control over the wandb global step. This class assumes reporting takes place once per episode, hence the wandb handlers have report_episode methods.

__init__(wandb_handlers: list, run_name: str, wandb_group: str | None = None, config: dict | None = None, resume_wandb_run: bool = False, wandb_id: str | None = None)

close()

classmethod log_level()

Handlers filter information from the data they receive.

This class method specifies the level they filter at.

report_episode(data, output_dir, episode, mode='train', **kwargs)