paralleldomain.model.dataset¶

class Dataset(decoder)¶

The Dataset object is the entry point for loading any data.

A dataset manages all attached scenes and its sensor data. It takes care of calling the decoder when specific data is required and stores it in the PD SDK model classes and attributes.

Parameters:: decoder (DatasetDecoderProtocol) –

get_scene(scene_name)¶

Allows access to a scene by using its name.

Parameters:: scene_name (str) – Name of scene to be returned
Return type:: Scene
Returns:: Returns the Scene object for a scene name.

get_sensor_frames(sensor_names=None, frame_ids=None, scene_names=None, shuffle=False, concurrent=False, max_queue_size=8, max_workers=4, random_seed=42, only_cameras=False, only_radars=False, only_lidars=False)¶

Returns a generator that yields all sensor frames from the scenes with the given names. If None is passed for scenes_names all scenes in this dataset are used. Same for sensor_names and frame_ids. None means all available will be returned.

Return type:

Generator[SensorFrame[Optional[datetime]], None, None]

Parameters:

sensor_names (Iterable[str] | None) –
frame_ids (Iterable[str] | None) –
scene_names (Iterable[str] | None) –
shuffle (bool) –
concurrent (bool) –
max_queue_size (int) –
max_workers (int) –
random_seed (int) –
only_cameras (bool) –
only_radars (bool) –
only_lidars (bool) –

get_unordered_scene(scene_name)¶

Allows access to a sensor frame set by using its name.

Parameters:: scene_name (str) – Name of sensor frame set to be returned
Return type:: UnorderedScene
Returns:: Returns the SensorFrameSet object for a sensor frame set name.

scene_pipeline(shuffle=False, concurrent=False, endless_loop=False, random_seed=42, scene_names=None, only_ordered_scenes=False, max_queue_size=8, max_workers=4)¶

Returns a generator that yields all scenes from the dataset with the given names. If None is passed for scenes_names all scenes in this dataset are returned.

Return type:

Generator[Union[UnorderedScene, Scene], None, None]

Parameters:

shuffle (bool) –
concurrent (bool) –
endless_loop (bool) –
random_seed (int) –
scene_names (List[str] | None) –
only_ordered_scenes (bool) –
max_queue_size (int) –
max_workers (int) –

sensor_frame_pipeline(shuffle=False, concurrent=False, endless_loop=False, random_seed=42, scene_names=None, frame_ids=None, sensor_names=None, only_ordered_scenes=False, max_queue_size=8, max_workers=4, only_cameras=False, only_radars=False, only_lidars=False)¶

Returns a generator that yields tuples of SensorFrame, Frame, Scene from the dataset. It can also be configured to only return certain scenes, frames, sensors. By setting shuffle = True the data is returned a randomized order. By setting concurrent = True threads are used to speed up the generator.

You can use this as quick access to sensor frames to load images, annotations etc. and feed those to your models or to encode the sensor frames into another data format.

Parameters:

shuffle (bool) – if = True the data is returned a randomized order
concurrent (bool) – if = True threads are used to speed up the generator
endless_loop (bool) – whether to iterate over the scenes of the dataset in an endless loop
random_seed (int) – seed used for shuffling the data
scene_names (Optional[List[str]]) – optional iterable of scene_names, if set, only frames from these scenes are yielded
frame_ids (Union[Dict[str, Iterable[str]], Iterable[str], None]) –
optional, dict/iterable containing frame_ids::
- if frame_ids is None, yield all frames from each scene
- if frame_ids is a dictionary, mapping scene_names to frame_ids:
  
  if a scene_name is among the dictionary keys, the corresponding value defines which frames to yield from that scene
  
  if a scene_name is not in the dictionary, yield all frames from that scene
- if frame_ids is an iterable, yield the same set of frames from each scene
sensor_names (Optional[Iterable[str]]) – optional iterable of sensor_names, if set, only frames recorded with these sensors are yielded
only_ordered_scenes (bool) – if = True, only yield frames from ordered scenes
max_queue_size (int) – maximum queue size
max_workers (int) – max number of worker threads
only_cameras (bool) – only yield camera frames
only_radars (bool) – only yield radar frames
only_lidars (bool) – only yield lidar frames

Return type:

Generator[Tuple[SensorFrame[Optional[datetime]], Frame, Union[UnorderedScene, Scene]], None, None]

property available_annotation_identifiers: List[AnnotationIdentifier]¶: Returns a list of available annotation identifiers for the dataset.

property available_annotation_types: List[Type[Annotation]]¶: Returns a list of available annotation types for the dataset.

property camera_frames: Generator[CameraSensorFrame[datetime | None], None, None]¶: Returns a generator that yields all CameraSensorFrames of all the unordered scenes in this dataset.

property camera_names: Set[str]¶: Returns the names of all camera sensors across all scenes in this dataset.

property decoder_init_kwargs: Dict[str, Any]¶: Returns an optional path to a file or a folder where the dataset is stored.

property format: str¶: Returns a str with the name of the dataset storage format (e.g. dgp, cityscapes, nuscenes).

property lidar_frames: Generator[LidarSensorFrame[datetime | None], None, None]¶: Returns a generator that yields all LidarSensorFrames of all the unordered scenes in this dataset.

property lidar_names: Set[str]¶: Returns the names of all lidar sensors across all scenes in this dataset.

property metadata: DatasetMeta¶: Returns a list of scene names within the dataset.

property name: str¶: Returns the name of the dataset.

property path: AnyPath¶: Returns an optional path to a file or a folder where the dataset is stored.

property radar_frames: Generator[RadarSensorFrame[datetime | None], None, None]¶: Returns a generator that yields all RadarSensorFrames of all the unordered scenes in this dataset.

property radar_names: Set[str]¶: Returns the names of all Radar sensors across all scenes in this dataset.

property scene_names: List[str]¶: Returns a list of scene names within the dataset.

property scenes: Dict[str, Scene]¶: Returns a dictionary of Scene instances with the scene name as key.

property sensor_frames: Generator[SensorFrame[datetime | None], None, None]¶: Returns a generator that yields all SensorFrames (Lidar and Camera) of all the unordered scenes in this dataset.

property unordered_scene_names: List[str]¶: Returns a list of sensor frame set names within the dataset.

property unordered_scenes: Dict[str, UnorderedScene[datetime | None]]¶: Returns a dictionary of SensorFrameSet instances with the scene name as key. This a superset of scenes and hence will also include all scenes (Scenes that have sensor frames with a date time).