tbp.monty.frameworks.models.evidence_matching#

tbp.monty.frameworks.models.evidence_matching.graph_memory#

class EvidenceGraphMemory(max_nodes_per_graph, max_graph_size, num_model_voxels_per_dim, *args, **kwargs)[source]#

Bases: GraphMemory

Custom GraphMemory that stores GridObjectModel instead of GraphObjectModel.

__init__(max_nodes_per_graph, max_graph_size, num_model_voxels_per_dim, *args, **kwargs)[source]#

Initialize a graph memory structure. This can then be filled with graphs.

Parameters:

  • k – integer k as in KNN, used for creating edges between observations

  • graph_delta_thresholds – thresholds for determining if two observations are sufficiently different to both be added to the object model.

Examples:

graph_memory = GraphMemory()
graph_memory._add_graph_to_memory(cup_model, "cup")
graph_memory.reset() # Call at beginning of episode
get_initial_hypotheses()[source]#
get_rotation_features_at_all_nodes(graph_id, input_channel)[source]#

Get rotation features from all N nodes. shape=(N, 3, 3).

Returns:

The rotation features from all N nodes. shape=(N, 3, 3).

tbp.monty.frameworks.models.evidence_matching.hypotheses#

class ChannelHypotheses(evidence: numpy.ndarray, locations: numpy.ndarray, poses: numpy.ndarray, input_channel: str) → None[source]#

Bases: Hypotheses

A set of hypotheses for a single input channel.

__init__(evidence: numpy.ndarray, locations: numpy.ndarray, poses: numpy.ndarray, input_channel: str) → None#
input_channel: str#
class Hypotheses(evidence: numpy.ndarray, locations: numpy.ndarray, poses: numpy.ndarray) → None[source]#

Bases: object

Set of hypotheses consisting of evidence, locations, and poses.

The three arrays are expected to have the same shape. Each index corresponds to a hypothesis.

__init__(evidence: numpy.ndarray, locations: numpy.ndarray, poses: numpy.ndarray) → None#
evidence: ndarray#
locations: ndarray#
poses: ndarray#

tbp.monty.frameworks.models.evidence_matching.hypotheses_displacer#

class DefaultHypothesesDisplacer(feature_weights: dict, graph_memory: EvidenceGraphMemory, max_match_distance: float, tolerances: dict, use_features_for_matching: dict[str, bool], feature_evidence_calculator: Type[FeatureEvidenceCalculator] = <class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment: int = 1, max_nneighbors: int = 3, past_weight: float = 1, present_weight: float = 1)[source]#

Bases: object

__init__(feature_weights, graph_memory, max_match_distance, tolerances, use_features_for_matching, feature_evidence_calculator=<class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment=1, max_nneighbors=3, past_weight=1, present_weight=1)[source]#

Initializes the DefaultHypothesesDisplacer.

Parameters:

  • feature_weights – How much should each feature be weighted when calculating the evidence update for hypothesis. Weights are stored in a dictionary with keys corresponding to features (same as keys in tolerances).

  • graph_memory – The graph memory to read graphs from.

  • max_match_distance – Maximum distance of a tested and stored location to be matched.

  • tolerances – How much can each observed feature deviate from the stored features to still be considered a match.

  • use_features_for_matching – Dictionary mapping input channels to booleans indicating whether to use features for matching.

  • feature_evidence_calculator – Class to calculate feature evidence for all nodes. Defaults to the default calculator.

  • feature_evidence_increment – Feature evidence (between 0 and 1) is multiplied by this value before being added to the overall evidence of a hypothesis. This factor is only multiplied with the feature evidence (not the pose evidence as opposed to the present_weight). Defaults to 1.

  • max_nneighbors – Maximum number of nearest neighbors to consider in the radius of a hypothesis for calculating the evidence. Defaults to 3.

  • past_weight – How much should the evidence accumulated so far be weighted when combined with the evidence from the most recent observation. Defaults to 1.

  • present_weight – How much should the current evidence be weighted when added to the previous evidence. If past_weight and present_weight add up to 1, the evidence is bounded and can’t grow infinitely. Defaults to 1. NOTE: right now this doesn’t give as good performance as with unbounded evidence since we don’t keep a full history of what we saw. With a more efficient policy and better parameters that may be possible to use though and could help when moving from one object to another and to generally make setting thresholds etc. more intuitive.

displace_hypotheses_and_compute_evidence(channel_displacement: numpy.ndarray, channel_features: dict, evidence_update_threshold: float, graph_id: str, possible_hypotheses: ChannelHypotheses, total_hypotheses_count: int) → ChannelHypotheses[source]#
Return type:

ChannelHypotheses

class HypothesesDisplacer(*args, **kwargs)[source]#

Bases: Protocol

__init__(*args, **kwargs)#
displace_hypotheses_and_compute_evidence(channel_displacement: numpy.ndarray, channel_features: dict, evidence_update_threshold: float, graph_id: str, possible_hypotheses: ChannelHypotheses, total_hypotheses_count: int) → ChannelHypotheses[source]#

Updates evidence by comparing features after applying sensed displacement.

This function applies the sensor displacement to the existing hypothesis and uses the result as search locations for comparing the sensed features. This comparison is used to update the evidence scores of the existing hypotheses. The hypotheses locations are updated to the new locations (i.e., after displacement)

Parameters:

  • channel_displacement (ndarray) – Channel-specific sensor displacement.

  • channel_features (dict) – Channel-specific input features.

  • evidence_update_threshold (float) – Evidence update threshold.

  • graph_id (str) – The ID of the current graph

  • possible_hypotheses (ChannelHypotheses) – Channel-specific possible hypotheses.

  • total_hypotheses_count (int) – Total number of hypotheses in the graph.

Return type:

ChannelHypotheses

Returns:

Displaced hypotheses with computed evidence.

tbp.monty.frameworks.models.evidence_matching.hypotheses_updater#

class DefaultHypothesesUpdater(feature_weights: dict, graph_memory: EvidenceGraphMemory, max_match_distance: float, tolerances: dict, feature_evidence_calculator: type[FeatureEvidenceCalculator] = <class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment: int = 1, features_for_matching_selector: type[FeaturesForMatchingSelector] = <class 'tbp.monty.frameworks.models.evidence_matching.features_for_matching.selector.DefaultFeaturesForMatchingSelector'>, initial_possible_poses: Literal[('uniform', 'informed')] | list[Rotation] = 'informed', max_nneighbors: int = 3, past_weight: float = 1, present_weight: float = 1, umbilical_num_poses: int = 8)[source]#

Bases: object

__init__(feature_weights, graph_memory, max_match_distance, tolerances, feature_evidence_calculator=<class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment=1, features_for_matching_selector=<class 'tbp.monty.frameworks.models.evidence_matching.features_for_matching.selector.DefaultFeaturesForMatchingSelector'>, initial_possible_poses='informed', max_nneighbors=3, past_weight=1, present_weight=1, umbilical_num_poses=8)[source]#

Initializes the DefaultHypothesesUpdater.

Parameters:

  • feature_weights – How much should each feature be weighted when calculating the evidence update for hypothesis. Weights are stored in a dictionary with keys corresponding to features (same as keys in tolerances).

  • graph_memory – The graph memory to read graphs from.

  • max_match_distance – Maximum distance of a tested and stored location to be matched.

  • tolerances – How much can each observed feature deviate from the stored features to still be considered a match.

  • feature_evidence_calculator – Class to calculate feature evidence for all nodes. Defaults to the default calculator.

  • feature_evidence_increment – Feature evidence (between 0 and 1) is multiplied by this value before being added to the overall evidence of a hypothesis. This factor is only multiplied with the feature evidence (not the pose evidence as opposed to the present_weight). Defaults to 1.

  • features_for_matching_selector – Class to select if features should be used for matching. Defaults to the default selector.

  • initial_possible_poses – Initial possible poses that should be tested for. Defaults to “informed”.

  • max_nneighbors – Maximum number of nearest neighbors to consider in the radius of a hypothesis for calculating the evidence. Defaults to 3.

  • past_weight – How much should the evidence accumulated so far be weighted when combined with the evidence from the most recent observation. Defaults to 1.

  • present_weight – How much should the current evidence be weighted when added to the previous evidence. If past_weight and present_weight add up to 1, the evidence is bounded and can’t grow infinitely. Defaults to 1. NOTE: right now this doesn’t give as good performance as with unbounded evidence since we don’t keep a full history of what we saw. With a more efficient policy and better parameters that may be possible to use though and could help when moving from one object to another and to generally make setting thresholds etc. more intuitive.

  • umbilical_num_poses – Number of sampled rotations in the direction of the plane perpendicular to the surface normal. These are sampled at umbilical points (i.e., points where PC directions are undefined).

update_hypotheses(hypotheses: Hypotheses, features: dict, displacements: dict | None, graph_id: str, mapper: ChannelMapper, evidence_update_threshold: float) → list[ChannelHypotheses][source]#

Update hypotheses based on sensor displacement and sensed features.

Updates existing hypothesis space or initializes a new hypothesis space if one does not exist (i.e., at the beginning of the episode). Updating the hypothesis space includes displacing the hypotheses possible locations, as well as updating their evidence scores. This process is repeated for each input channel in the graph.

Parameters:

  • hypotheses (Hypotheses) – Hypotheses for all input channels in the graph_id

  • features (dict) – Input features

  • displacements (dict | None) – Given displacements

  • graph_id (str) – Identifier of the graph being updated

  • mapper (ChannelMapper) – Mapper for the graph_id to extract data from evidence, locations, and poses based on the input channel

  • evidence_update_threshold (float) – Evidence update threshold.

Return type:

list[ChannelHypotheses]

Returns:

The list of hypotheses updates to be applied to each input channel.

class HypothesesUpdater(*args, **kwargs)[source]#

Bases: Protocol

__init__(*args, **kwargs)#
update_hypotheses(hypotheses: Hypotheses, features: dict, displacements: dict | None, graph_id: str, mapper: ChannelMapper, evidence_update_threshold: float) → list[ChannelHypotheses][source]#

Update hypotheses based on sensor displacement and sensed features.

Parameters:

  • hypotheses (Hypotheses) – Hypotheses for all input channels for the graph_id

  • features (dict) – Input features

  • displacements (dict | None) – Given displacements

  • graph_id (str) – Identifier of the graph being updated

  • mapper (ChannelMapper) – Mapper for the graph_id to extract data from evidence, locations, and poses based on the input channel

  • evidence_update_threshold (float) – Evidence update threshold

Return type:

list[ChannelHypotheses]

Returns:

The list of channel hypotheses updates to be applied.

all_usable_input_channels(features: dict, all_input_channels: list[str]) → list[str][source]#

Determine all usable input channels.

Parameters:

  • features – Input features.

  • all_input_channels – All input channels that are stored in the graph.

Returns:

All input channels that are usable for matching.

tbp.monty.frameworks.models.evidence_matching.learning_module#

class EvidenceGraphLM(max_match_distance, tolerances: dict, feature_weights: dict, feature_evidence_increment=1, evidence_threshold_config: float | str = 'all', vote_evidence_threshold=0.8, past_weight=1, present_weight=1, vote_weight=1, object_evidence_threshold=1, x_percent_threshold=10, path_similarity_threshold=0.1, pose_similarity_threshold=0.35, required_symmetry_evidence=5, graph_delta_thresholds=None, max_graph_size=0.3, max_nodes_per_graph=2000, num_model_voxels_per_dim=50, use_multithreading=True, gsg_class=<class 'tbp.monty.frameworks.models.goal_state_generation.EvidenceGoalStateGenerator'>, gsg_args=None, hypotheses_updater_class: type[HypothesesUpdater] = <class 'tbp.monty.frameworks.models.evidence_matching.hypotheses_updater.DefaultHypothesesUpdater'>, hypotheses_updater_args: dict | None = None, *args, **kwargs)[source]#

Bases: GraphLM

Learning module that accumulates evidence for objects and poses.

Matching Attributes:
max_match_distance: Maximum distance of a tested and stored location to

be matched.

tolerances: How much can each observed feature deviate from the stored

features to still be considered a match.

feature_weights: How much should each feature be weighted when calculating

the evidence update for hypotheses. Weights are stored in a dictionary with keys corresponding to features (same as keys in tolerances)

feature_evidence_increment: Feature evidence (between 0 and 1) is multiplied

by this value before being added to the overall evidence of a hypothesis. This factor is only multiplied with the feature evidence (not the pose evidence as opposed to the present_weight).

evidence_threshold_config: How to decide which hypotheses

should be updated. When this parameter is either ‘[int]%’ or ‘x_percent_threshold’, then this parameter is applied to the evidence for the Most Likely Hypothesis (MLH) to determine a minimum evidence threshold in order for other hypotheses to be updated. Any hypotheses falling below the resulting evidence threshold do not get updated. The other options set a fixed threshold that does not take MLH evidence into account. In [int, float, ‘[int]%’, ‘mean’, ‘median’, ‘all’, ‘x_percent_threshold’]. Defaults to ‘all’.

vote_evidence_threshold: Only send votes that have a scaled evidence above

this threshold. Vote evidences are in the range of [-1, 1] so the threshold should not be outside this range.

past_weight: How much should the evidence accumulated so far be weighted

when combined with the evidence from the most recent observation.

present_weight: How much should the current evidence be weighted when added

to the previous evidence. If past_weight and present_weight add up to 1, the evidence is bounded and can’t grow infinitely. NOTE: right now this doesn’t give as good performance as with unbounded evidence since we don’t keep a full history of what we saw. With a more efficient policy and better parameters that may be possible to use though and could help when moving from one object to another and to generally make setting thresholds etc. more intuitive.

vote_weight: Vote evidence (between -1 and 1) in multiplied by this value

when being added to the overall evidence of a hypothesis. If past and current_weight add up to 1, it is use as weight in np.average to keep the evidence in a fixed range.

Terminal Condition Attributes:
object_evidence_threshold: Minimum required evidence for an object to be

recognized. We additionally check that the evidence for this object is significantly higher than for all other objects.

x_percent_threshold: Used in two places:
  1. All objects whose highest evidence is greater than the most likely

    objects evidence - x_percent of the most like objects evidence are considered possible matches. That means to only have one possible match, no other object can have more evidence than the candidate match’s evidence - x percent of it.

  2. Within one object, possible poses are considered possible if their

    evidence is larger than the most likely pose of this object - x percent of this poses evidence.

# TODO: should we use a separate threshold for within and between objects? If this value is larger, the model is usually more robust to noise and reaches a better performance but also requires a lot more steps to reach a terminal condition, especially if there are many similar object in the data set.

path_similarity_threshold: How similar do paths have to be to be

considered the same in the terminal condition check.

pose_similarity_threshold: difference between two poses to be considered

unique when checking for the terminal condition (in radians).

required_symmetry_evidence: number of steps with unchanged possible poses

to classify an object as symmetric and go into terminal condition.

Model Attributes:
graph_delta_thresholds: Thresholds used to compare nodes in the graphs being

learned, and thereby whether to include a new point or not. By default, we only consider the distance between points, using a threshold of 0.001 (determined in remove_close_points). Can also specify thresholds based on e.g. surface normal angle difference, or principal curvature magnitude difference.

max_graph_size: Maximum size of a graph in meters. Any observations that fall

out of this range will be discarded/used for building a new model. This constraints the size of models that an LM can learn and enforces learning models of sub-components of objects.

max_nodes_per_graph: Maximum number of nodes in a graph. This will be k when

picking the k-winner voxels to add their content into the graph used for matching.

num_model_voxels_per_dim: Number of voxels per dimension in the model grid.

This constraints the spatial resolution that the model can represent. max_graph_size/num_model_voxels_per_dim = how much space is lumped into one voxel. All locations that fall into the same voxel will be averaged and represented as one value. num_model_voxels_per_dim should not be too large since the memory requirements grow cubically with this number.

gsg_class: The type of goal-state-generator to associate with the LM. gsg_args: Dictionary of configuration parameters for the GSG. hypotheses_updater_class: The type of hypotheses updater to associate with the

LM.

hypotheses_updater_args: Dictionary of configuration parameters for the

hypotheses updater.

Debugging Attributes:
use_multithreading: Whether to calculate evidence updates for different

objects in parallel using multithreading. This can be done since the updates to different objects are completely independent of each other. In general it is recommended to use this but it can be useful to turn it off for debugging purposes.

__init__(max_match_distance, tolerances: dict, feature_weights: dict, feature_evidence_increment=1, evidence_threshold_config: float | str = 'all', vote_evidence_threshold=0.8, past_weight=1, present_weight=1, vote_weight=1, object_evidence_threshold=1, x_percent_threshold=10, path_similarity_threshold=0.1, pose_similarity_threshold=0.35, required_symmetry_evidence=5, graph_delta_thresholds=None, max_graph_size=0.3, max_nodes_per_graph=2000, num_model_voxels_per_dim=50, use_multithreading=True, gsg_class=<class 'tbp.monty.frameworks.models.goal_state_generation.EvidenceGoalStateGenerator'>, gsg_args=None, hypotheses_updater_class: type[HypothesesUpdater] = <class 'tbp.monty.frameworks.models.evidence_matching.hypotheses_updater.DefaultHypothesesUpdater'>, hypotheses_updater_args: dict | None = None, *args, **kwargs)[source]#

Initialize general Learning Module based on graphs.

Parameters:

initialize_base_modules – Provides option to not intialize the base modules if more specialized versions will be initialized in child LMs. Defaults to True.

collect_stats_to_save()[source]#

Get all stats that this LM should store in the buffer for logging.

Returns:

The stats dictionary.

get_all_evidences()[source]#

Return evidence for each pose on each graph (pointer).

get_current_mlh()[source]#

Return the current most likely hypothesis of the learning module.

Returns:

graph_id, location, rotation, scale, evidence

Return type:

dict with keys

get_evidence_for_each_graph()[source]#

Return maximum evidence count for a pose on each graph.

get_mlh_for_object(object_id)[source]#

Get mlh for a specific object ID.

Note

When trying to retrieve the MLH for the current most likely object and not any other object, it is better to use self.current_mlh

Returns:

The most likely hypothesis for the object ID.

get_output()[source]#

Return the most likely hypothesis in same format as LM input.

The input to an LM at the moment is a dict of features at a location. The output therefor has the same format to keep the messaging protocol consistent and make it easy to stack multiple LMs on top of each other.

If the evidence for mlh is < object_evidence_threshold, interesting_features == False

get_possible_hypothesis_ids(object_id)[source]#
get_possible_matches()[source]#

Return graph ids with significantly higher evidence than median.

get_possible_poses(as_euler=True)[source]#

Return possible poses for each object (for logging).

Here this list doesn’t get narrowed down. This is not really used for evidence matching since we threshold in other places.

get_top_two_mlh_ids()[source]#

Retrieve the two most likely object IDs for this LM.

Returns:

The two most likely object IDs.

get_top_two_pose_hypotheses_for_graph_id(graph_id)[source]#

Return top two hypotheses for a given graph_id.

get_unique_pose_if_available(object_id)[source]#

Get the most likely pose of an object if narrowed down.

If there is not one unique possible pose or symmetry detected, return None

Returns:

The pose and scale if a unique pose is available, otherwise None.

receive_votes(vote_data)[source]#

Get evidence count votes and use to update own evidence counts.

Weighted by distance to votes and their evidence. TODO: also take into account rotation vote

vote_data contains:

pos_location_votes: shape=(N, 3) pos_rotation_votes: shape=(N, 3, 3) pose_evidences: shape=(N,)

reset()[source]#

Reset evidence count and other variables.

send_out_vote() → dict | None[source]#

Send out hypotheses and the evidence for them.

Votes are a dict and contain the following:

pose_hypotheses: locations (V, 3) and rotations (V, 3, 3) pose_evidence: Evidence (V) for each location-rotation pair in the

pose hypotheses. Scaled into range [-1, 1] where 1 is the hypothesis with the largest evidence in this LM and -1 the one with the smallest evidence. When thresholded, pose_evidence will be in range [self.vote_evidence_threshold, 1]

sensed_pose_rel_body: sensed location and rotation of the input to this

LM. Rotation is represented by the pose vectors (surface normal and curvature directions) for the SMs. For input from LMs it is also represented as 3 unit vectors, these are calculated from the estimated rotation of the most likely object. This pose is used to calculate the displacement between two voting LMs and to translate the votes between their reference frames. Shape=(4,3).

Where V is the number of votes (V=number of hypotheses if not thresholded) If none of the hypotheses of an object are > vote_evidence_threshold, this object will not send out a vote.

Return type:

dict | None

Returns:

Dictionary with possible states and sensed poses relative to the body, or None if we don’t want the LM to vote.

set_detected_object(terminal_state)[source]#

Set the current graph ID.

If we didn’t recognize the object this will be new_object{n} where n is len(graph_memory) + 1. Otherwise it is the id of the graph that we recognized. If we timed out it is None and we will not update the graph memory.

tbp.monty.frameworks.models.evidence_matching.model#

class MontyForEvidenceGraphMatching(*args, **kwargs)[source]#

Bases: MontyForGraphMatching

Monty model for evidence based graphs.

Customize voting and union of possible matches.

__init__(*args, **kwargs)[source]#

Initialize and reset LM.

switch_to_exploratory_step()[source]#

Switch to exploratory step.

Also, set mlh evidence high enough to generate output during exploration.

tbp.monty.frameworks.models.evidence_matching.resampling_hypotheses_updater#

class ResamplingHypothesesUpdater(feature_weights: dict, graph_memory: EvidenceGraphMemory, max_match_distance: float, tolerances: dict, feature_evidence_calculator: Type[FeatureEvidenceCalculator] = <class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment: int = 1, features_for_matching_selector: Type[FeaturesForMatchingSelector] = <class 'tbp.monty.frameworks.models.evidence_matching.features_for_matching.selector.DefaultFeaturesForMatchingSelector'>, hypotheses_count_multiplier: float = 1.0, hypotheses_existing_to_new_ratio: float = 0.1, initial_possible_poses: Literal[('uniform', 'informed')] | list[Rotation] = 'informed', max_nneighbors: int = 3, past_weight: float = 1, present_weight: float = 1, umbilical_num_poses: int = 8)[source]#

Bases: object

Hypotheses updater that resamples hypotheses at every step.

This updater enables updating of the hypothesis space by resampling and rebuilding the hypothesis space at every step. We resample hypotheses from the existing hypothesis space, as well as new hypotheses informed by the sensed pose.

The resampling process is governed by two main parameters:

  • hypotheses_count_multiplier: scales the total number of hypotheses every step.

  • hypotheses_existing_to_new_ratio: controls the proportion of existing vs.

    informed hypotheses during resampling.

To reproduce the behavior of DefaultHypothesesUpdater sampling a fixed number of hypotheses only at the beginning of the episode, you can set hypotheses_count_multiplier=1.0 and hypotheses_existing_to_new_ratio=0.0.

Note

It would be better to decouple the amount of hypotheses added from the amount deleted in each step. At the moment, this is decided by the hypotheses_count_multiplier. For example, when the multiplier is set to 1.0, the hypotheses sampled is equal to the hypotheses removed. We ideally can decouple theses then use a slope threshold to decide on which hypotheses to remove, and use prediction error or other heuristics to decide to how many hypotheses to resample.

__init__(feature_weights, graph_memory, max_match_distance, tolerances, feature_evidence_calculator=<class 'tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator.DefaultFeatureEvidenceCalculator'>, feature_evidence_increment=1, features_for_matching_selector=<class 'tbp.monty.frameworks.models.evidence_matching.features_for_matching.selector.DefaultFeaturesForMatchingSelector'>, hypotheses_count_multiplier=1.0, hypotheses_existing_to_new_ratio=0.1, initial_possible_poses='informed', max_nneighbors=3, past_weight=1, present_weight=1, umbilical_num_poses=8)[source]#

Initializes the ResamplingHypothesesUpdater.

Parameters:

  • feature_weights – How much should each feature be weighted when calculating the evidence update for hypothesis. Weights are stored in a dictionary with keys corresponding to features (same as keys in tolerances).

  • graph_memory – The graph memory to read graphs from.

  • max_match_distance – Maximum distance of a tested and stored location to be matched.

  • tolerances – How much can each observed feature deviate from the stored features to still be considered a match.

  • feature_evidence_calculator – Class to calculate feature evidence for all nodes. Defaults to the default calculator.

  • feature_evidence_increment – Feature evidence (between 0 and 1) is multiplied by this value before being added to the overall evidence of a hypothesis. This factor is only multiplied with the feature evidence (not the pose evidence as opposed to the present_weight). Defaults to 1.

  • features_for_matching_selector – Class to select if features should be used for matching. Defaults to the default selector.

  • hypotheses_count_multiplier – Scales the total number of hypotheses every step. Defaults to 1.0.

  • hypotheses_existing_to_new_ratio – Controls the proportion of the existing vs. newly sampled hypotheses during resampling. Defaults to 0.0.

  • initial_possible_poses – Initial possible poses that should be tested for. Defaults to “informed”.

  • max_nneighbors – Maximum number of nearest neighbors to consider in the radius of a hypothesis for calculating the evidence. Defaults to 3.

  • past_weight – How much should the evidence accumulated so far be weighted when combined with the evidence from the most recent observation. Defaults to 1.

  • present_weight – How much should the current evidence be weighted when added to the previous evidence. If past_weight and present_weight add up to 1, the evidence is bounded and can’t grow infinitely. Defaults to 1. NOTE: right now this doesn’t give as good performance as with unbounded evidence since we don’t keep a full history of what we saw. With a more efficient policy and better parameters that may be possible to use though and could help when moving from one object to another and to generally make setting thresholds etc. more intuitive.

  • umbilical_num_poses – Number of sampled rotations in the direction of the plane perpendicular to the surface normal. These are sampled at umbilical points (i.e., points where PC directions are undefined).

update_hypotheses(hypotheses: Hypotheses, features: dict, displacements: dict | None, graph_id: str, mapper: ChannelMapper, evidence_update_threshold: float) → list[ChannelHypotheses][source]#

Update hypotheses based on sensor displacement and sensed features.

Updates existing hypothesis space or initializes a new hypothesis space if one does not exist (i.e., at the beginning of the episode). Updating the hypothesis space includes displacing the hypotheses possible locations, as well as updating their evidence scores. This process is repeated for each input channel in the graph.

Parameters:

  • hypotheses (Hypotheses) – Hypotheses for all input channels in the graph_id

  • features (dict) – Input features

  • displacements (dict | None) – Given displacements

  • graph_id (str) – Identifier of the graph being updated

  • mapper (ChannelMapper) – Mapper for the graph_id to extract data from evidence, locations, and poses based on the input channel

  • evidence_update_threshold (float) – Evidence update threshold.

Return type:

list[ChannelHypotheses]

Returns:

The list of hypotheses updates to be applied to each input channel.