event_aug

event_aug.noise

event_aug.noise.downsample(arr: ndarray, reshape_size: Tuple[int, int], crop_size: Optional[Tuple[int, int]] = None) → ndarray

Downsample an array along the spatial dimensions to a given size by either resizing or cropping or both.

Parameters

• arr (np.ndarray) – Input array.

• reshape_size (Tuple[int, int]) – Size to reshape the array along spatial dimensions to using interpolation.

• crop_size (Tuple[int, int]) – Size to crop the array along spatial dimensions to.

Returns

Downsampled array.

Return type

np.ndarray

event_aug.noise.gen_fractal_3d(shape: Tuple[int, int, int], res: Tuple[int, int, int] = (1, 4, 4), tileable: Tuple[bool, bool, bool] = (True, False, False), octaves: int = 1, persistence: float = 0.5, lacunarity: int = 2, reshape_size: Optional[Tuple[int, int]] = None, crop_size: Optional[Tuple[int, int]] = None, return_arr: bool = True, save_arr: bool = False, arr_save_path: Optional[str] = None, save_video: bool = False, video_save_path: Optional[str] = None, out_fps: int = 25) → ndarray

Generates 3D Fractal noise.

Parameters

• shape (Tuple[int, int, int]) – Shape of the 3D fractal noise wanted. This must be a multiple of lacunarity**(octaves-1)*res.

• res (Tuple[int, int, int]) – The number of periods of noise to generate along each axis (tuple of three ints). Note that shape must be a multiple of (lacunarity**(octaves-1)*res).

• tileable (Tuple[bool, bool, bool]) – If the noise should be tileable along each axis.

• octaves (int) – Number of octaves in the noise.

• persistence (float) – The scaling factor between two octaves.

• lacunarity (int) – The frequency factor between two octaves.

• reshape_size (Tuple[int, int]) – Size to reshape the array along spatial dimensions to using interpolation.

• crop_size (Tuple[int, int]) – Size to crop the array along spatial dimensions to.

• return_arr (bool) – If the noise array should be returned.

• save_arr (bool) – If the generated array should be saved to an npy file.

• arr_save_path (str) – Path (.npy) to save the generated array to.

• save_video (bool) – If the generated array should be saved as a video to a .mp4 file.

• video_save_path (str) – Path (.mp4) to save the generated array as a video to.

• out_fps (int) – FPS of the output video.

Returns

3D array of Fractal noise.

Return type

np.ndarray

event_aug.noise.gen_perlin_2d(shape: Tuple[int, int], scale: int = 100, octaves: int = 6, persistence: float = 0.5, lacunarity: float = 2.0, seed: int = 0, save_path: Optional[str] = None, reshape_size: Optional[Tuple[int, int]] = None, crop_size: Optional[Tuple[int, int]] = None) → ndarray

Generates 2D Perlin noise.

Parameters

• shape (Tuple[int, int]) – Shape of the 2D output array.

• scale (int) – Scale of the noise.

• octaves (int) – Number of octaves to use.

• persistence (float) – Scaling factor between two octaves.

• lacunarity (float) – Frequency factor between two octaves.

• seed (int) – Seed for the noise generation.

• save_path (str) – Path to save the generated noise as an image.

• reshape_size (Tuple[int, int]) – Size to reshape the array along spatial dimensions to using interpolation.

• crop_size (Tuple[int, int]) – Size to crop the array along spatial dimensions to.

Returns

2D array of Perlin noise.

Return type

np.ndarray

event_aug.noise.gen_perlin_3d(shape: Tuple[int, int, int], res: Tuple[int, int, int] = (1, 4, 4), tileable: Tuple[bool, bool, bool] = (True, False, False), reshape_size: Optional[Tuple[int, int]] = None, crop_size: Optional[Tuple[int, int]] = None, return_arr: bool = True, save_arr: bool = False, arr_save_path: Optional[str] = None, save_video: bool = False, video_save_path: Optional[str] = None, out_fps: int = 25) → ndarray

Generates 3D Perlin noise.

Parameters

• shape (Tuple[int, int, int]) – Shape of the 3D Perlin noise wanted. This must be a multiple of res.

• res (Tuple[int, int, int]) – The number of periods of noise to generate along each axis (tuple of three ints). Note that shape must be a multiple of res.

• tileable (Tuple[bool, bool, bool]) – If the noise should be tileable along each axis.

• reshape_size (Tuple[int, int]) – Size to reshape the array along spatial dimensions to using interpolation.

• crop_size (Tuple[int, int]) – Size to crop the array along spatial dimensions to.

• return_arr (bool) – If the noise array should be returned.

• save_arr (bool) – If the generated array should be saved to an npy file.

• arr_save_path (str) – Path (.npy) to save the generated array to.

• save_video (bool) – If the generated array should be saved as a video to a .mp4 file.

• video_save_path (str) – Path (.mp4) to save the generated array as a video to.

• out_fps (int) – FPS of the output video.

Returns

3D array of Perlin noise.

Return type

np.ndarray

event_aug.noise.postprocess_3d_noise(arr, reshape_size=None, crop_size=None)

event_aug.noise.shift_range(arr: ndarray)

Shift the range of values in an array to [0, 255]

Parameters

arr (np.ndarray) – Input array

Returns

Array with shifted range

Return type

np.ndarray

event_aug.spike_encoding

event_aug.spike_encoding.delta_intensity_code_arr(arr: Union[ndarray, Tensor], arr_from_file: bool = False, threshold: int = 15, percent_threshold: int = 10, mode: str = 'threshold', use_neg_delta: bool = True, exclude_start: bool = False, return_arr: bool = True, save_arr: bool = False, arr_save_path: Optional[str] = None, save_video: bool = False, video_save_path: Optional[str] = None, video_fps: int = 25, memory_map: bool = True) → Union[ndarray, Tensor]

Converts a video data array to spikes. If the difference in the intensity of a pixel in consecutive frames is greater than the threshold, the pixel is assigned an event.

Parameters

• arr (np.ndarray or torch.Tensor or str) – Array containing the video data (frames). Should be of shape (T x H x W) or (T x H x W x C). If a string is passed, the array is read from a .npy file.

• arr_from_file (bool) – Whether to read the video array from a .npy file.

• threshold (int) – Threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• percent_threshold (int) – Pixel-wise percentage threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• mode (str) – Whether to use ‘threshold’ or ‘percent_threshold’”.

• use_neg_delta (bool) – Whether to consider decreases in intensity as well along with increases for assigning events.

• exclude_start (bool) – Whether to not return the spikes for the first frame which will always be 0 for all pixels.

• return_arr (bool) – Whether to return the spikes array.

• save_arr (bool) – Whether to save the spikes array to a .npy file.

• arr_save_path (str) – Name of the .npy file to save the spikes array to (excluding extension).

• save_video (bool) – Whether to save the spikes as a video (.mp4).

• video_save_path (str) – Name of the .mp4 file to save the spikes video to.

• video_fps (int) – Frames per second of the video.

• memory_map (bool) – Whether to use memory mapping to read the spikes video array.

Returns

Array containing the spikes.

Return type

np.ndarray or torch.Tensor

event_aug.spike_encoding.delta_intensity_code_file(video_path: str, threshold: int = 15, percent_threshold: int = 10, mode: str = 'threshold', out_fps: Optional[int] = None, use_neg_delta: bool = True, exclude_start: bool = False, return_arr: bool = False, save_video: bool = True, video_save_path: Optional[str] = None, save_arr: bool = False, arr_save_path: Optional[str] = None) → None

Reads a video file, convert the video to spiking form and saves to a file. If the difference in the intensity of a pixel in consecutive frames is greater than the threshold, the pixel is assigned an event.

Parameters

• video_path (str) – Path to the input video file.

• threshold (int) – Threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• percent_threshold (int) – Pixel-wise percentage threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• mode (str) – Whether to use ‘threshold’ or ‘percent_threshold’”.

• out_fps (int) – Output video frame rate.

• use_neg_delta (bool) – Whether to consider decreases in intensity as well along with increases for assigning events.

• exclude_start (bool) – Whether to not return the spikes for the first frame which will always be 1 for all pixels.

• return_arr (bool) – Whether to return an array containing frame-wise spikes.

• save_video (bool) – Whether to save the frame-wise spikes as a video (.mp4).

• video_save_path (str) – Name of the .mp4 file to save the spikes video to.

• save_arr (bool) – Whether to save the spikes array to a .npy file.

• arr_save_path (str) – Name of the .npy file to save the spikes array to (excluding extension).

event_aug.spike_encoding.delta_intensity_code_video(video_path: Optional[str] = None, video_arr: Optional[Union[ndarray, Tensor]] = None, threshold: int = 15, percent_threshold: int = 10, mode: str = 'threshold', use_neg_delta: bool = True, out_fps: Optional[int] = None, exclude_start: bool = True, return_arr: bool = False, save_video: bool = True, video_save_path: Optional[str] = None, save_arr: bool = False, arr_save_path: Optional[str] = None) → Optional[Union[ndarray, Tensor]]

Converts a video to spiking form. If the difference in the intensity of a pixel in consecutive frames is greater than the threshold, the pixel is assigned an event. Works with either a video file or a video array.

Parameters

• video_path (str) – Path to the input video file.

• video_arr (np.ndarray or torch.Tensor) – Array containing the video data (frames). Should be of shape (T x H x W) or (T x H x W x C).

• threshold (int) – Threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• percent_threshold (int) – Pixel-wise percentage threshold for the difference in intensities of pixels in consecutive frames for assigning events.

• mode (str) – Whether to use ‘threshold’ or ‘percent_threshold’”.

• use_neg_delta (bool) – Whether to consider decreases in intensity as well along with increases for assigning events.

• out_fps (int) – Output video frame rate (required if input video is a file).

• exclude_start (bool) – Whether to not return the spikes for the first frame which will always be 0 for all pixels (required if input video is a file).

• return_arr (bool) – Whether to return an array containing frame-wise spikes.

• save_video (bool) – Whether to save the frame-wise spikes as a video (.mp4).

• video_save_path (str) – Name of the .mp4 file to save the spikes video to.

• save_arr (bool) – Whether to save the spikes array to a .npy file.

• arr_save_path (str) – Name of the .npy file to save the spikes array to (excluding extension).

event_aug.spike_encoding.rate_code(data: Union[ndarray, Tensor], n_steps: int = 1, gain: int = 1) → ndarray

Converts array of continuous input data to spikes using rate coding. Input array is normalized to [0, 1] and the resulting elements are treated as probabilities for Bernoulli trials.

Parameters

• data (np.ndarray or torch.Tensor) – 2D array of continuous input data. 3D array in case of time-series data (shape: T x H x W).

• n_steps (int) – Number of time-steps to perform rate coding for. Only used for 2D input data. In case of time series data, rate coding is performed for each time-step.

• gain (int) – Factor by which to scale normalized input features.

Returns

2D array of spikes. 3D array in case of time-series data.

Return type

np.ndarray

event_aug.spike_injection

event_aug.spike_injection.inject_event_spikes(event_data_path: str, save_path: str, spikes_video_path: Optional[str] = None, spikes_arr: Optional[Union[ndarray, str]] = None, spikes_arr_from_file: bool = False, memory_map: bool = True, resize_size: Optional[Tuple[int, int]] = None, crop_size: Optional[Tuple[int, int]] = None, fps: Optional[int] = None, label: Optional[int] = None, polarity: Optional[int] = None, timestamp_keys: Optional[Tuple[str]] = None, xy_keys: Optional[Tuple[str]] = None, label_keys: Optional[Tuple[str]] = None, polarity_keys: Optional[Tuple[str]] = None, iterations: int = 1, verbose=False)

Injects specified event spikes into existing event recordings data to serve as augmentation.

Parameters

• event_data_path (str) – Path to the .h5 file containing the event recordings data.

• save_path (str) – Path to save the augmented event data with spikes injected to.

• spikes_video_path (str) – Path to the video containing the event spikes data as frames.

• spikes_arr (np.ndarray) – NumPy Array containing the event spikes data as frames.

• spikes_arr_from_file (bool) – Whether to read the spikes video array from a .npy file.

• memory_map (bool) – Whether to use memory mapping to read the spikes video array.

• resize_size (Tuple[int, int]) – If specified, the size to reshape the event spikes frames to.

• crop_size (Tuple[int, int]) – If specified, the size to crop the event spikes frames to.

• fps (int) – Frame rate to use for the event spikes to be injected.

• label (int) – Label to assign for the event spikes to be injected.

• polarity (int) – Polarity to assign for the event spikes to be injected.

• timestamp_keys (Tuple[str]) – Keys to use to index the event timestamp data.

• xy_keys (Tuple[str]) – Keys to use to index the event xy coordinates data.

• label_keys (Tuple[str]) – Keys to use to index the event label data.

• polarity_keys (Tuple[str]) – Keys to use to index the event polarity data.

• iterations (int) – Number of times to inject event spikes.

• verbose (bool) – Whether to print progress messages.

event_aug.utils

event_aug.utils.array_to_video(arr: ndarray, save_path: str, fps: int)

Converts an array of image data to a video.

Parameters

• arr (np.ndarray) – Array containing the image data.

• save_path (str) – Path to the output video file.

• fps (int) – Output video frame rate.

event_aug.utils.download_from_youtube(urls: Union[Tuple[str], str], start_times: Union[Tuple[int], str], end_times: Union[Tuple[int], str], save_dir: str)

Downloads videos from YouTube.

Parameters

• urls (Union[Tuple[str], str]) – YouTube URLs.

• start_times (Union[Tuple[int], str]) – Start times to trim the videos.

• end_times (Union[Tuple[int], str]) – End times to trim the videos.

• save_dir (str) – Directory to save the videos to.

event_aug.utils.imgs_to_video(save_path: str, img_dir: Optional[str] = None, img_arr: Optional[ndarray] = None, height: Optional[int] = None, width: Optional[int] = None, out_fps: int = 15, numbered_imgs: bool = False, verbose: bool = False)

Converts a directory of images or an array / list of image data to a video.

Parameters

• save_path (str) – Path (.mp4) to the output video file.

• img_dir (str) – Path to the directory containing the images.

• imgs_arr (np.ndarray or list) – Array containing the image data. If it’s a list, it should contain as many images as number of frames wanted. If it’s a multi-dimensional array, it should be of shape (T x H x W) or (T x H x W x C).

• height (int) – Height of the images.

• width (int) – Width of the images.

• out_fps (int) – Output video frame rate.

• numbered_imgs (bool) – Whether the image file names are a number sequence.

• verbose (bool) – Whether to print progress.

event_aug.utils.resize_video(video_path: str, save_path: str, size: Tuple[int])

Resizes a video to a given size.

Parameters

• video_path (str) – Path to the input video.

• save_path (str) – Path to the output video.

• size (Tuple[int]) – Size to resize the video to.

event_aug.utils.save_video_frames_diffs(video_path: str, save_path: str, out_fps: Optional[int] = None, neg_diff: bool = True)

Saves the differences in intensities between video frames as a video.

Parameters

• video_path (str) – Path to the input video.

• save_path (str) – Path to save the output video.

• out_fps (int) – Output video frame rate.

• neg_diff (bool) – Whether to consider negative differences.

event_aug.utils.video_to_array(video_path: str, grayscale: bool = False, return_fps: bool = False) → Union[ndarray, Tuple[ndarray, int]]

Converts a video to an array of frames data.

Parameters

• video_path (str) – Path to the input video file.

• grayscale (bool) – Whether to convert the frames to grayscale.

• return_fps (bool) – Whether to return the video frame rate.

Returns

• arr (np.ndarray) – Array of frames data.

• fps (int) – Video frame rate.