pick_pixel#

NyxCameraSensor.pick_pixel(camera_index, x, y)[source]#

Cast a ray through pixel (x, y) of the given camera and return what it hits.

Parameters:

  • camera_index (int) – Index of the camera within this sensor’s shared metadata. For a single NyxCameraSensor this is 0; multi-camera setups index in registration order.

  • x (int) – Pixel coordinates in image space, with the origin at the top-left of the framebuffer. Must lie within the camera’s configured resolution.

  • y (int) – Pixel coordinates in image space, with the origin at the top-left of the framebuffer. Must lie within the camera’s configured resolution.

Return type:

Optional[NyxPickPixelResult]

Returns:

NyxPickPixelResult or None – None if the ray missed all scene geometry (e.g. background / sky). Otherwise a NyxPickPixelResult with fields:

  • entity: the Genesis Entity that was hit

  • link_name: name of the hit link (URDF) or <link>_<vgeom_idx> (MJCF); empty string for other morph types

  • position: world-space hit point as (x, y, z) in Genesis Z-up coordinates

Example

pick_pixel casts a ray from the camera through the requested pixel (top-left origin) and returns either None if the ray exits the scene (e.g. through the sky / environment map) or a NyxPickPixelResult naming the hit entity, its link, and the world-space hit position in Genesis Z-up:

# ``cam`` is a built NyxCameraSensor returned by ``scene.add_sensor(...)``;
# see the user guide for the surrounding scene-build boilerplate.
res = cam.pick_pixel(camera_index=0, x=320, y=240)
if res is None:
    print("ray hit nothing (background)")
else:
    wx, wy, wz = res.position
    print(f"hit {res.entity} link={res.link_name!r} at ({wx:.2f}, {wy:.2f}, {wz:.2f})")

Note

link_name is morph-dependent: the link name for URDF entities, a "<link>_<vgeom_idx>" string for MJCF entities, and an empty string for every other morph type. camera_index is this sensor’s slot in its shared metadata, 0 for the only camera in the scene; multi-camera setups index in registration order.