paralleldomain.data_lab.config.sensor_rig

class CameraIntrinsic(*, proto=None, width=None, height=None, fov=None, supersample=None, capture_rgb=None, capture_depth=None, capture_normals=None, capture_segmentation=None, capture_instance=None, capture_detections=None, capture_motionvectors=None, lut=None, lut_weight=None, post_process_params=None, post_process=None, distortion_params=None, noise_params=None, enable_streaming=None, transmit_gray=None, distortion_lookup_table=None, capture_basecolor=None, capture_properties=None, time_offset=None, capture_backwardmotionvectors=None)

Parameters which control the intrinsics of an rgb camera sensor.

Parameters:
width

Width of the image in pixels

height

Height of the image in pixels.

fov

Field of view of the camera in degrees. Ignored when fx, fy, cx and cy are set in DistortionParams.

supersample

Supersampling to apply during rendering. Supersampling is ignored on pinhole cameras.

capture_rgb

Flag to control whether RGB data is captured.

capture_depth

Flag to control whether depth data is captured.

capture_normals

Flag to control whether surface normal data is captured.

capture_segmentation

Flag to control whether semantic segmentation data is captured.

capture_instance

Flag to control whether instance segmentation data is captured.

capture_detections

Flag to control whether bounding box data is captured.

capture_motionvectors

Flag to control whether motion vector data is captured.

lut

Path to color adjustment map in unreal syntax.

lut_weight

Scale factor when applying lut. Default: 1.0

post_process_params

Post processing parameters to apply to the rendered image.

post_process

Array of ‘post_process_entries’, each of which has an unreal material path and a weight (to blend with the previous image). “material” is the path in unreal syntax and “weight” is the blend weight (0->1).

distortion_params

The distortion parameters to apply to the image.

noise_params

The noise parameters to apply to the image.

enable_streaming
transmit_gray

Control whether the returned image is grayscale.

distortion_lookup_table

Path to the distortion lookup table if a lookup table is required by DistortionParams.

capture_basecolor

Flag to control whether base color data is captured.

capture_properties

Flag to control whether material propety data is captured.

time_offset

Time offset of captured images.

capture_backwardmotionvectors

Flag to control whether backwards motion vector data is captured.

class DistortionParams(*, proto=None, k1=None, k2=None, k3=None, k4=None, k5=None, k6=None, p1=None, p2=None, skew=None, is_fisheye=None, fx=None, fy=None, cx=None, cy=None, fisheye_model=None)

Parameters that control the lens distortion applied to RGB camera sensors

Parameters:
k1

Radial distortion parameter k1 if the Brown Conrady or OpenCV Fisheye models are specified in fisheye_model, ignored otherwise.

k2

Radial distortion parameter k2 if the Brown Conrady or OpenCV Fisheye models are specified in fisheye_model, ignored otherwise.

k3

Radial distortion parameter k3 if the Brown Conrady or OpenCV Fisheye models are specified in fisheye_model, ignored otherwise.

k4

Radial distortion parameter k4 if the Brown Conrady or OpenCV Fisheye models are specified in fisheye_model, ignored otherwise.

k5

Radial distortion parameter k5 if the Brown Conrady model is specified in fisheye_model. Ignored otherwise.

k6

Radial distortion parameter k6 if the Brown Conrady model is specified in fisheye_model. Ignored otherwise.

p1

Tangential distortion parameter p1 if the Brown Conrady model is specified in fisheye_model. Ignored otherwise.

p2

Tangential distortion parameter p2 if the Brown Conrady model is specified in fisheye_model. Ignored otherwise.

skew

Parameter controlling the skew distortion that is applied to the camera.

is_fisheye

DEPRECATED

fx

The horizontal focal length of the camera.

fy

Vertical focal length of the camera.

cx

Distortion center of the image along the horizontal axis.

cy

Distortion center of the image along the vertical axis.

fisheye_model

The type of fisheye cameras being replicated. The following are valid values:

0 - Pinhole Camera if :attr:`k1`,:attr:`k2`,:attr:`p1`,:attr:`p2`[,:attr:`k3`[,:attr:`k4`,:attr:`k5`,:attr:`k6`]] not specified or Brown Conrady Distortion if distortion parameters are specified
1 - OpenCV Fisheye Distortion
3 - PD Fisheye Distortion
6 - Orthographic Projection
class NoiseParams(*, proto=None, enable_bayer=None, enable_gauss_noise=None, enable_poisson_noise=None, enable_denoise=None, gauss_noise_sigma=None, poisson_noise_lambda=None, denoise_filter=None, denoise_filter_size=None, bilateral_sigma_d=None, bilateral_sigma_r=None, enable_auto_noise=None, signal_amount=None, pre_amplifier_noise=None, post_amplifier_noise=None, is_using_iso=None, iso_level=None, enable_auto_iso=None, fstop=None, max_exposure_time=None, quantum_efficiency=None)
Parameters:
enable_bayer

Flag to enable Bayer noise, also known as RGGB noise, simulates the noise patterns present when using a Bayer filter to capture color images. This filter is a common way to capture color in digital cameras. It shows up as a colors shifting away from pixels. Default: True

enable_gauss_noise

Flag to enable Gaussian noise, also known as white noise, simulates electronic interference caused by thermal vibrations. This type of noise is tied to the temperature of the sensor and the brightness of the scene being captured - this adds noise uniformly on the image Default: True

enable_poisson_noise

Flag to enable Poisson Noise. Simulates the probability of photons being collected by the sensor. This is the most common form of noise in an image, unless it is a low-light capture. Poisson noise is a good way to simulate the natural behavior of photons in an image. This adds more noise to brighter areas, less to darker areas. Default: True

enable_denoise

Flag to add a denoise pass to mimic a real sensor’s stack. Default: True

gauss_noise_sigma

Sigma applied when Gauss Noise is activated. Larger values result in noisier images. Default: 0.025

poisson_noise_lambda

Lambda applied to poisson noise. Smaller values result in noisier images. Default: 1000.0

denoise_filter

Type of denoising filter applied. Default: DenoiseFilter.MEDIAN_FILTER

denoise_filter_size

Only applies to Bilateral denoising filter. Represents how many pixels to sample. Default: 3

bilateral_sigma_d

Spatial sigma_d of bilateral denoising filter. Default: 3.0

bilateral_sigma_r

sigma_r of bilateral denoising filter. Default: 20.0

enable_auto_noise

Boolean to enable automatic setting of noise parameters based on exposure value. Default: True

signal_amount

Signal amount. Default: 5223

pre_amplifier_noise

Pre-amplifier noise. Default: 7.63

post_amplifier_noise

Post-amplifier noise. Default: 247.5

is_using_iso

Boolean flag to control whether ISO is used to adjust noise parameters Default: True

iso_level

ISO level of the camera. Default: 800

enable_auto_iso

Controls whether IDO is used to scale noise parameters.

fstop

F-Stop of the camera. Default: 1.0

max_exposure_time

Inverse of the minimum shutter speed. Default: 0.033

quantum_efficiency

Used when calculating auto ISO defaults. Default: 0.7

class PostProcessNode(*, proto=None, material=None, weight=None)

Node which carries out post processing on a rendered image.

Parameters:
material

The material to be applied in post processing.

weight

The weight of the post processing to be applied.

class PostProcessParams(*, proto=None, exposure_compensation=None, exposure_speed_up=None, exposure_speed_down=None, exposure_min_ev100=None, exposure_max_ev100=None, exposure_metering_mask=None, motion_blur_amount=None, motion_blur_max=None, dof_focal_distance=None, dof_depth_blur_amount=None, dof_depth_blur_radius=None, vignette_intensity=None, tone_curve=None, exposure_compensation_curve=None)

Parameters that control the post processing applied to rgb camera sensor data.

Parameters:
exposure_compensation

Logarithmic adjustment for exposure. When set to 0, there will be no adjustment, -1 is two times darker, -2 is four times darker, 1 is two times brighter, and 2 is four times brighter Default: -100.0

exposure_speed_up

Speeds up eye adaptation in f-stops per second.

exposure_speed_down

Slows down eye adaptation in f-stops per second.

exposure_min_ev100

The minimum brightness limit for auto exposure adaptation, expressed in pixel luminance (cd/m2). Value is typically negative and should be less than or equal to Max EV100. As this value increases, the scene view gets darker. If Min EV100 is equal to Max EV100, auto exposure is disabled. Default: -100.0

exposure_max_ev100

The maximum brightness limit for auto exposure adaptation, expressed in pixel luminance (cd/m2). Value should be positive and should be greater than or equal to Min EV100. As this value decreases, the scene view gets brighter. If Max EV100 is equal to Min EV100, auto exposure is disabled. Default: -100.0

exposure_metering_mask

Texture in unreal syntax. Bright spots on the mask will have high influence on auto exposure metering, and dark spots will have low influence.

motion_blur_amount

Motion blur amount expressed in screen percentage (exposure will affect the final outcome). Default: 1.5

motion_blur_max

Maximum motion blue amount in screen percentage. Default: 5.0

dof_focal_distance

Focal distance (in cm) to near focal plane. Default: -1.0

dof_depth_blur_amount

Distance (in km) where we blur to 50%.

dof_depth_blur_radius

Radius of how many pixels to blur. Default: -1.0

vignette_intensity

Strength of vignette applied.

tone_curve

Type of tone curve applied to the image.

exposure_compensation_curve

Curve in unreal syntax. The X and Y-axis values in the Curve graph translate to the Average Scene EV100 and Exposure Compensation (Curve) values

class SensorConfig(*, proto=None, display_name=None, camera_intrinsic=None, lidar_intrinsic=None, radar_intrinsic=None, sensor_extrinsic=None, render_ego=None)

A configuration of one sensor.

Parameters:
display_name

The name of the sensor.

camera_intrinsic

Instrinsics of a camera sensor.

lidar_intrinsic

Instrinsics of a lidar sensor.

radar_intrinsic

Instrinsics of a radar sensor.

sensor_extrinsic

The extrinsics of the sensor.

render_ego

Control whether or not the attached ego agent is rendered if present

get_sensor_to_ego(coordinate_system='FLU')
Parameters:

coordinate_system (str) – The coordinate system the return transformation is in

Return type:

Transformation

Returns:

The transformation from sensor to ego in the given coordinate system

property ego_to_sensor: Transformation

Returns: The transformation from ego to sensor in FLU

property sensor_to_ego: Transformation

Returns: The transformation from sensor to ego in FLU

class SensorExtrinsic(*, proto=None, yaw=None, pitch=None, roll=None, x=None, y=None, z=None, lock_to_yaw=None, attach_socket=None, follow_rotation=None)

Object to store extrinsic parameters of sensors. All extrinsics are stored in the RFU coordinate system relative to the agent to which the sensor is attached.

Parameters:
yaw

Yaw of the sensor in degrees.

pitch

Pitch of the sensor in degrees.

roll

Roll of the sensor in degrees.

x

x position of the sensor in meters.

y

y position of the sensor in meters.

z

z position of the sensor in meters.

lock_to_yaw

Setting this flag to true will cause the yaw value of the sensor orientation to be locked to the agent to which the sensor is attached, but leave the pitch and roll values as zero with respect to the world frame.

attach_socket

Controls the socket on the agent to which the sensor should be attached.

follow_rotation

Controls whether or not the sensor follows the rotation of the socket to which it is attached if attach_socket is True.

class SensorList(*, proto=None, sensors=None)

A list of sensors.

Parameters:
sensors
class SensorRig(*args, **kwargs)

A configuration of a rig of sensors containing one or more sensors.

Parameters:
sensor_configs

The configuration of sensors within a particular sensor rig.

sensor_rig_artifact_uid

The s3 path to a sensor rig if it is not stored locally.

default_sensor_splits_list

A list of sensor names to control which sensors are rendered in parallel.

class ToneCurveParams(*, proto=None, slope=None, toe=None, shoulder=None, black_clip=None, white_clip=None)

Parameters that control the tone curve applied to the image.

Parameters:
slope

A slope tone curve. Default: 0.66

toe

A toe tone curve. Default: 0.52

shoulder

A shoulder tone curve. Default: 0.49

black_clip

A black_clip tone curve. Default: 0.0

white_clip

A white_clip tone curve. Default: 0.08

class DenoiseFilter(value)

The type of denoising filter applied to an image.

AVERAGE_FILTER = 0

Denoising filter using average method.

BILATERAL_FILTER = 3

Denoising filter using bilateral method.

FAST_MEDIAN_FILTER = 2

Denoising filter using fast median method.

MEDIAN_FILTER = 1

Denoising filter using median method.