tbp.monty#

tbp.monty.cmp#

class Goal(location: np.ndarray | None, morphological_features: dict[str, Any] | None, non_morphological_features: dict[str, Any] | None, confidence: float, use_state: bool, sender_id: str, sender_type: str, goal_tolerances: dict[str, Any] | None, info: dict[str, Any] | None = None)[source]#

Bases: Message

Specialization of Message for goals with null (None) values allowed.

Specialized form of message that still adheres to the cortical messaging protocol, but can have null (None) values associated with the location and morphological features.

Used by goal generators (GSGs) to communicate goals to other GSGs, and to motor actuators.

The message variables generally have the same meaning as for the base Message class, and they represent the target values for the receiving system. Thus if a goal specifies a particular object ID (non-morphological feature) in a particular pose (location and morphological features), then the receiving system should attempt to achieve that state.

Note however that for the goal, the confidence corresponds to the conviction with which a GSG believes that the current goal should be acted upon. Float bound in [0.0, 1.0].

__init__(location: np.ndarray | None, morphological_features: dict[str, Any] | None, non_morphological_features: dict[str, Any] | None, confidence: float, use_state: bool, sender_id: str, sender_type: str, goal_tolerances: dict[str, Any] | None, info: dict[str, Any] | None = None)[source]#

Initialize a goal.

Parameters:

location (np.ndarray | None) – the location to move to in global/body-centric coordinates, or None if the location is not specified as part of the goal. For example, this may be a point on an object’s surface or a location nearby from which a sensor would have a good view of the target point.
morphological_features (dict[str, Any] | None) – dictionary of morphological features or None. For example, it may include pose vectors, whether the pose is fully defined, etc.
non_morphological_features (dict[str, Any] | None) – a dictionary containing non-morphological features at the target location or None.
confidence (float) – a float between 0 and 1 representing the confidence in the goal.
use_state (bool) – a boolean indicating whether the goal should be used.
sender_id (str) – the ID of the sender of the goal (e.g., “LM_0”).
sender_type (str) – the type of sender of the goal (e.g., “GSG”).
goal_tolerances (dict[str, Any] | None) – Dictionary of tolerances that GSGs use when determining whether the current state of the LM matches the driving goal or None. As such, a GSG can send a goal with more or less strict tolerances if certain elements of the message (e.g. the location of a mug vs its orientation) are more or less important.
info (dict[str, Any] | None) – Optional metadata for logging purposes.

class Message(location, morphological_features, non_morphological_features, confidence, use_state, sender_id, sender_type)[source]#

Bases: object

Message class used as message packages passed in Monty using CMP.

The cortical messaging protocol (CMP) is used to pass messages between Monty components and makes sure we can easily set up arbitrary configurations of them. This class makes it easier to define the CMP in one place and defines the content and structure of messages passed between Monty components. It also contains some helper functions to access and modify the message content.

States are represented in this format but can be interpreted by the receiver in different ways:

Observed states: states output by sensor modules Hypothesized states: states output by learning modules Goals: motor output of learning modules

location#: 3D vector representing the location

morphological_features#: dictionary of morphological features. Should include pose_vectors of shape (3,3) and pose_fully_defined (bool).

non_morphological_features#: dictionary of non-morphological features.

confidence#: message confidence. In range [0,1].

use_state#: boolean indicating whether the message should be used or not.

sender_id#: string identifying the sender of the message.

sender_type#: string identifying the type of sender. Can be “SM” or “LM”.

__init__(location, morphological_features, non_morphological_features, confidence, use_state, sender_id, sender_type)[source]#: Initialize a message.

get_curvature_directions()[source]#

Return the curvature direction vectors.

Raises:: ValueError – If self.sender_type is not SM

get_feature_by_name(feature_name)[source]#

get_nth_pose_vector(pose_vector_index)[source]#

Return the nth pose vector.

When self.sender_type == “SM”, the first pose vector is the surface normal and the second and third are the curvature directions. When self.sender_type == “LM”, the pose vectors correspond to the rotation of the object relative to the model learned of it.

get_on_object()[source]#

Return whether we think we are on the object or not.

This is currently used in the policy to stay on the object.

get_pose_vectors()[source]#: Return the pose vectors.

get_surface_normal()[source]#

Return the surface normal vector.

Raises:: ValueError – If self.sender_type is not SM

set_displacement(displacement, ppf=None)[source]#

Add displacement (represented as dict) to message.

TODO S: Add this to message or in another place?

transform_morphological_features(translation=None, rotation=None)[source]#: Apply translation and/or rotation to morphological features.

encode_goal(goal: Goal) → dict[str, Any][source]#

Encode a goal into a dictionary.

Parameters:: goal – The goal to encode.
Returns:: A dictionary containing the goal’s attributes.

tbp.monty.context#

class RuntimeContext(rng: numpy.random.RandomState, suppress_runtime_errors: bool = False) → None[source]#

Bases: object

Monty’s runtime context.

The RuntimeContext carries runtime-scoped values used throughout Monty.

rng#: The random number generator.

suppress_runtime_errors#: Whether to suppress runtime errors. Runtime errors can be raised when goal is None or invalid. When in an experimental mode, we want to raise runtime errors by default. When in a production mode, we want to suppress runtime errors by default. Currently, we run a lot of experiments, so the current default is to raise runtime errors.

__init__(rng: numpy.random.RandomState, suppress_runtime_errors: bool = False) → None#

rng: RandomState#

suppress_runtime_errors: bool = False#

tbp.monty.geometry#

class Rotation(scipy_rotation: scipy.spatial.transform.Rotation) → None[source]#

Bases: object

Rotation in 3 dimensions.

This class was created to be a (nearly) drop-in replacement for (scipy.spatial.transform.Rotation)[https://docs.scipy.org/doc/scipy-1.10.1/reference/generated/scipy.spatial.transform.Rotation.html]. that better conforms to our conventions. Primarily, we wanted to be consistent about using scalar-first (wxyz) order for quaternions, but scalar-last (xyzw) is scipy’s default mode. Consequently, this class’s from_quat and as_quat implementations assume and return scalar-first quaternion components.

Since Rotation is a thin wrapper around scipy.spatial.transform.Rotation, its API is largely inherited from scipy. The main exceptions are:

from_quat and as_quat assume scalar-first order.

The approx_equal method has been backported from future scipy versions.

For consistency, from_scipy_rotation and as_scipy_rotation methods have been added.

Any missing scipy methods can be added as needed.

__init__(scipy_rotation: scipy.spatial.transform.Rotation) → None[source]#

static align_vectors(a: npt.ArrayLike, b: npt.ArrayLike, weights: npt.ArrayLike | None = None, return_sensitivity: bool = False) → tuple[Rotation, float] | tuple[Rotation, float, np.ndarray][source]#

Estimate a rotation to optimally align two sets of vectors.

For full details, see (scipy.spatial.transform.Rotation.align_vectors)[https://docs.scipy.org/doc/scipy-1.10.1/reference/generated/scipy.spatial.transform.Rotation.align_vectors.html].

Parameters:

a (npt.ArrayLike) – Array-like of shape (3,) or (N, 3).
b (npt.ArrayLike) – Array-like of shape (3,) or (N, 3).
weights (npt.ArrayLike | None) – Weights describing the relative importance of the vector observations. If None (default), then all values in weights are assumed to be 1. One and only one weight may be infinity, and weights must be positive.
return_sensitivity (bool) – Whether to return the sensitivity matrix. See Notes for details. Default is False.

Returns:

Best estimate of the Rotation that transforms b to a. rssd : Square root of the weighted sum of the squared distances between

the given sets of vectors.

sensitivity_matrixSensitivity matrix of the estimated rotation estimate.: See scipy documentation (link above) for details.

Return type:

rotation

apply(vectors: numpy.typing.ArrayLike, inverse: bool = False) → numpy.ndarray[source]#

Apply this rotation to a set of vectors.

If the original frame rotates to the final frame by this rotation, then its application to a vector can be seen in two ways:

As a projection of vector components expressed in the final frame to the original frame.

As the physical rotation of a vector being glued to the original frame as it rotates. In this case the vector components are expressed in the original frame before and after the rotation.

Parameters:

vectors (ArrayLike) – Array-like of shape (3,) or (N, 3) of xyz coordinates to rotate.
inverse (bool) – If True then apply the inverse of this rotation. Equivalent to rotation.inv().apply(vectors).

Return type:

ndarray

Returns:

An array of shape (3,) or (N, 3) of rotated xyz coordinates.

Examples

>>> # pitch up 180°, then roll counter-clockwise 90°
>>> a_rotation = Rotation.from_euler("XZ", [np.pi, np.pi/2])
>>> a_rotation.apply([3, 5, 8])
array([-5., -3., -8.])

>>> vectors = np.array([
...     [3, 5, 8],
...     [3, 5, -8],
...     [3, -5, -8],
...     [-3, -5, -8],
...     [-3, -5, 8],
...     [-3, 5, 8]
... ], dtype=float)
>>> a_rotation.apply(vectors)
array([[-5., -3., -8.],
       [-5., -3.,  8.],
       [ 5., -3.,  8.],
       [ 5.,  3.,  8.],
       [ 5.,  3., -8.],
       [-5.,  3., -8.]])

approx_equal(other: Rotation, tol: float = 1e-06) → bool | np.ndarray[source]#

Check if this rotation is approximately equal to another rotation.

Parameters:

other (Rotation) – The other rotation to compare to.
tol (float) – Absolute tolerance, expressed in radians.

Return type:

bool | np.ndarray

Returns:

True if the angular delta between a and b is within tolerance. False otherwise. If a and b are non-single, returns an array of booleans.

as_euler(seq: str, degrees: bool = False) → numpy.ndarray[source]#

Return type:: ndarray

as_matrix() → numpy.ndarray[source]#

Return type:: ndarray

as_quat() → numpy.ndarray[source]#

The (scalar-first) quaternion representation.

This methods differs substantially from scipy.spatial.transform.Rotation.as_quat. Here, we return quaternions in scalar-first (wxyz) order, whereas scipy returns them in scalar-last (xyzw) order. Scalar-last ordering will not be supported.

Return type:: ndarray
Returns:: Array of shape (4,) or (N, 4) in scalar-first (wxyz) order.

as_rotvec(degrees: bool = False) → numpy.ndarray[source]#

Return type:: ndarray

as_scipy_rotation() → scipy.spatial.transform.Rotation[source]#

Return type:: Rotation

static concatenate(rotations: Iterable[Rotation]) → Rotation[source]#

Concatenate a sequence of Rotation objects into a single object.

This is useful if you want to, for example, take the mean of a set of rotations and need to pack them into a single object to do so.

Parameters:: rotations (Iterable[Rotation]) – The rotations to concatenate. If a single Rotation object is passed in, a copy is returned.
Return type:: Rotation
Returns:: The Rotation instance containing the concatenated rotations.

static from_euler(seq: str, angles: float | npt.ArrayLike, degrees: bool = False) → Rotation[source]#

Return type:: Rotation

static from_matrix(matrix: numpy.typing.ArrayLike) → Rotation[source]#

Return type:: Rotation

static from_quat(quat: numpy.typing.ArrayLike) → Rotation[source]#

Build from (scalar-first) quaternion data.

This methods differs substantially from scipy.spatial.transform.Rotation.from_quat. Here, we expect quaternions in scalar-first (wxyz) order, where SciPy expects them in scalar-last (xyzw) order. Scalar-last ordering will not be supported.

Parameters:: quat (ArrayLike) – Array-like of shape (4,) or (N, 4) in scalar-first (wxyz) order.
Return type:: Rotation
Returns:: A Rotation instance.
Raises:: ValueError – If quat does not have shape (4,) or (N, 4).

static from_rotvec(rotvec: numpy.typing.ArrayLike, degrees: bool = False) → Rotation[source]#

Return type:: Rotation

static from_scipy_rotation(rot: scipy.spatial.transform.Rotation) → Rotation[source]#

Return type:: Rotation

static identity(num: int | np.integer | None = None) → Rotation[source]#

Return type:: Rotation

inv() → Rotation[source]#

Create a new Rotation that is the inverse of this Rotation.

Composition of a rotation with its inverse is an identity transformation.

Return type:: Rotation
Returns:: The new Rotation object.

Examples

>>> fwd = Rotation.from_euler("y", -np.pi/6)  # yaw right 30°
>>> inv = fwd.inv()
>>> inv.inv().approx_equal(fwd)
True

magnitude() → float | np.ndarray[source]#

Return type:: float | np.ndarray

mean(weights: npt.ArrayLike | None = None) → Rotation[source]#

Return type:: Rotation

Return type:: Rotation

property single: bool#: Whether this instance represents a single rotation.

scipy_rotations_approx_equal(a: ScipyRotation, b: ScipyRotation, tol: float = 1e-06) → bool | np.ndarray[source]#

Backport of scipy.spatial.transform.Rotation.approx_equal.

Parameters:

a (ScipyRotation) – First scipy rotation.
b (ScipyRotation) – Second scipy rotation.
tol (float) – Absolute tolerance, expressed in radians.

Return type:

bool | np.ndarray

Returns:

True if the angular delta between a and b is within tolerance. False otherwise. If a and b are non-single, returns an array of booleans.

to_scalar_first(xyzw: numpy.typing.ArrayLike) → numpy.ndarray[source]#

Convert a quaternion from scalar-last (xyzw) to scalar-first (wxyz) order.

This is a helper function for the Rotation class extracted for testing purposes.

Parameters:: xyzw (ArrayLike) – Array-like of shape (4,) or (N, 4) in scalar-last (xyzw) order.
Return type:: ndarray
Returns:: Array of shape (4,) or (N, 4) in scalar-first (wxyz) order.

to_scalar_last(wxyz: numpy.typing.ArrayLike) → numpy.ndarray[source]#

Convert a quaternion from scalar-first (wxyz) to scalar-last (xyzw) order.

This is a helper function for the Rotation class extracted for testing purposes.

Parameters:: wxyz (ArrayLike) – Array-like of shape (4,) or (N, 4) in scalar-first (wxyz) order.
Return type:: ndarray
Returns:: Array of shape (4,) or (N, 4) in scalar-last (xyzw) order.

tbp.monty.hydra#

monty_class_resolver(class_name: str) → type[source]#

Returns a class object by fully qualified path.

Return type:: type

TODO: This is an interim solution to retrieve my_class in: the my_class(**my_args) pattern.

ndarray_resolver(list_or_tuple: list | tuple) → np.ndarray[source]#

Returns a numpy array from a list or tuple.

Return type:: np.ndarray

numpy_list_eval_resolver(expr_list: list) → list[float][source]#

ones_resolver(n: int) → numpy.ndarray[source]#

Returns a numpy array of ones.

Return type:: ndarray

path_expanduser_resolver(path: str) → str[source]#

Returns a path with ~ expanded to the user’s home directory.

Return type:: str

register_resolvers() → None[source]#

Skips resolvers that are already registered rather than raising a ValueError, since multiple entry points (e.g. tests/__init__.py and update_snapshots.py) may call this function in the same process.

Return type:: None

tests_dir_resolver(path: str) → str[source]#

Return type:: str

tbp.monty.math#

tbp.monty.path#

monty_data_path(custom_data_path: str | Path | None, default_subpath: str | Path) → Path[source]#

Get data path, using custom path if provided, or return the default.

Parameters:

custom_data_path (str | Path | None) – Custom data path provided by the user, or None to use default_subpath.
default_subpath (str | Path) – Default subpath within MONTY_DATA to use if no custom path.

Return type:

Path

Returns:

Full data path.