aac_datasets package

Audio Captioning datasets for PyTorch.

class AudioCaps(
root: str | Path | None = None,
subset: str = 'train',
download: bool = False,
transform: Callable[[Dict[str, Any]], Any] | None = None,
verbose: int = 0,
force_download: bool = False,
verify_files: bool = False,
*,
audio_duration: float = 10.0,
audio_format: str = 'flac',
audio_n_channels: int = 1,
download_audio: bool = True,
exclude_removed_audio: bool = True,
ffmpeg_path: str | Path | None = None,
flat_captions: bool = False,
max_workers: int | None = 1,
sr: int = 32000,
with_tags: bool = False,
ytdlp_path: str | Path | None = None,
)[source]

Bases: AACDataset[AudioCapsItem]

Unofficial AudioCaps PyTorch dataset.

Subsets available are ‘train’, ‘val’ and ‘test’.

Audio is a waveform tensor of shape (1, n_times) of 10 seconds max, sampled at 32kHz by default. Target is a list of strings containing the captions. The ‘train’ subset has only 1 caption per sample and ‘val’ and ‘test’ have 5 captions. Download requires ‘yt-dlp’ and ‘ffmpeg’ commands.

AudioCaps paper : https://www.aclweb.org/anthology/N19-1011.pdf

Dataset folder tree
{root}
└── AUDIOCAPS
    ├── train.csv
    ├── val.csv
    ├── test.csv
    └── audio_32000Hz
        ├── train
        │    └── (46231/49838 flac files, ~42G for 32kHz)
        ├── val
        │    └── (465/495 flac files, ~425M for 32kHz)
        └── test
            └── (913/975 flac files, ~832M for 32kHz)
CARD: ClassVar[AudioCapsCard] = <aac_datasets.datasets.functional.audiocaps.AudioCapsCard object>
property download: bool
property exclude_removed_audio: bool
property index_to_name: Dict[int, str]
property root: str
property sr: int
property subset: str
property with_tags: bool
class Clotho(
root: str | Path | None = None,
subset: str = 'dev',
download: bool = False,
transform: Callable | None = None,
verbose: int = 0,
force_download: bool = False,
verify_files: bool = False,
*,
clean_archives: bool = True,
flat_captions: bool = False,
version: str = 'v2.1',
)[source]

Bases: AACDataset[ClothoItem]

Unofficial Clotho PyTorch dataset.

Subsets available are ‘train’, ‘val’, ‘eval’, ‘dcase_aac_test’, ‘dcase_aac_analysis’, ‘dcase_t2a_audio’ and ‘dcase_t2a_captions’.

Audio are waveform sounds of 15 to 30 seconds, sampled at 44100 Hz. Target is a list of 5 different sentences strings describing an audio sample. The maximal number of words in captions is 20.

Clotho V1 Paper : https://arxiv.org/pdf/1910.09387.pdf

Dataset folder tree for version ‘v2.1’, with all subsets
{root}
└── CLOTHO_v2.1
    ├── archives
    |   └── (5 7z files, ~8.9GB)
    ├── clotho_audio_files
    │   ├── clotho_analysis
    │   │    └── (8360 wav files, ~19GB)
    │   ├── development
    │   │    └── (3839 wav files, ~7.1GB)
    │   ├── evaluation
    │   │    └── (1045 wav files, ~2.0GB)
    │   ├── test
    │   |    └── (1043 wav files, ~2.0GB)
    │   ├── test_retrieval_audio
    │   |    └── (1000 wav files, ~2.0GB)
    │   └── validation
    │        └── (1045 wav files, ~2.0GB)
    └── clotho_csv_files
        ├── clotho_captions_development.csv
        ├── clotho_captions_evaluation.csv
        ├── clotho_captions_validation.csv
        ├── clotho_metadata_development.csv
        ├── clotho_metadata_evaluation.csv
        ├── clotho_metadata_test.csv
        ├── clotho_metadata_validation.csv
        ├── retrieval_audio_metadata.csv
        └── retrieval_captions.csv
CARD: ClassVar[ClothoCard] = <aac_datasets.datasets.functional.clotho.ClothoCard object>
INVALID_SOUND_ID: ClassVar[str] = 'Not found'
INVALID_START_END_SAMPLES: ClassVar[str] = ''
property download: bool
property root: str
property sr: int
property subset: str
property version: str
class MACS(
root: str | Path | None = None,
subset: str = 'full',
download: bool = False,
transform: Callable | None = None,
verbose: int = 0,
force_download: bool = False,
verify_files: bool = False,
*,
clean_archives: bool = True,
flat_captions: bool = False,
)[source]

Bases: AACDataset[MACSItem]

Unofficial MACS PyTorch dataset.

Dataset folder tree
{root}
└── MACS
    ├── audio
    │    └── (3930 wav files, ~13GB)
    ├── LICENCE.txt
    ├── MACS.yaml
    ├── MACS_competence.csv
    └── tau_meta
        ├── fold1_evaluate.csv
        ├── fold1_test.csv
        ├── fold1_train.csv
        └── meta.csv
CARD: ClassVar[MACSCard] = <aac_datasets.datasets.functional.macs.MACSCard object>
property download: bool
get_annotator_id_to_competence_dict() Dict[int, float][source]

Get annotator to competence dictionary.

get_competence(
annotator_id: int,
) float[source]

Get competence value for a specific annotator id.

property root: str
property sr: int
property subset: str
class WavCaps(
root: str | Path | None = None,
subset: str = 'as_noac',
download: bool = False,
transform: Callable | None = None,
verbose: int = 0,
force_download: bool = False,
verify_files: bool = False,
*,
clean_archives: bool = False,
hf_cache_dir: str | None = None,
repo_id: str | None = None,
revision: str | None = '85a0c21e26fa7696a5a74ce54fada99a9b43c6de',
zip_path: str | Path | None = None,
)[source]

Bases: AACDataset[WavCapsItem]

Unofficial WavCaps PyTorch dataset.

WavCaps Paper : https://arxiv.org/pdf/2303.17395.pdf HuggingFace source : https://huggingface.co/datasets/cvssp/WavCaps

This dataset contains 4 training subsets, extracted from different sources: - AudioSet strongly labeled (as) - BBC Sound Effects (bbc) - FreeSound (fsd) - SoundBible (sb) - AudioSet strongly labeled without AudioCaps (as_noac) - FreeSound without Clotho (fsd_nocl)

Warning

WavCaps download is experimental ; it requires a lot of disk space and can take very long time to download and extract, so you might expect errors.

Dataset folder tree
{root}
└── WavCaps
    ├── Audio
    │   ├── AudioSet_SL
    │   │    └── (108317 flac files, ~64GB)
    │   ├── BBC_Sound_Effects
    │   │    └── (31201 flac files, ~142GB)
    │   ├── FreeSound
    │   │    └── (262300 flac files, ~1.4TB)
    │   └── SoundBible
    │        └── (1232 flac files, ~884MB)
    ├── Zip_files
    │   ├── AudioSet_SL
    │   │    └── (8 zip files, ~76GB)
    │   ├── BBC_Sound_Effects
    │   │    └── (26 zip files, ~562GB)
    │   ├── FreeSound
    │   │    └── (123 zip? files, ~1.4TB)
    │   └── SoundBible
    │        └── (1 zip? files, ~624GB)
    ├── json_files
    │    ├── AudioSet_SL
    │    │    └── as_final.json
    │    ├── BBC_Sound_Effects
    │    │    └── bbc_final.json
    │    ├── FreeSound
    │    │    ├── fsd_final_2s.json
    │    │    └── fsd_final.json
    │    ├── SoundBible
    │    │    └── sb_final.json
    │    └── blacklist
    │         ├── blacklist_exclude_all_ac.json
    │         ├── blacklist_exclude_test_ac.json
    │         └── blacklist_exclude_ubs8k_esc50_vggsound.json
    ├── .gitattributes
    └── README.md
CARD: ClassVar[WavCapsCard] = <aac_datasets.datasets.functional.wavcaps.WavCapsCard object>
property download: bool
property root: str
property sr: int
property subset: str
get_default_ffmpeg_path() str[source]

Returns the default ffmpeg executable path.

If set_default_ffmpeg_path() has been used before with a string argument, it will return the value given to this function. Else if the environment variable AAC_DATASETS_FFMPEG_PATH has been set to a string, it will return its value. Else it will be equal to “ffmpeg” by default.

get_default_root() str[source]

Returns the default root directory path.

If set_default_root() has been used before with a string argument, it will return the value given to this function. Else if the environment variable AAC_DATASETS_ROOT has been set to a string, it will return its value. Else it will be equal to “.” by default.

get_default_ytdlp_path() str[source]

Returns the default yt-dlp executable path.

If set_default_ytdlp_path() has been used before with a string argument, it will return the value given to this function. Else if the environment variable AAC_DATASETS_YTDLP_PATH has been set to a string, it will return its value. Else it will be equal to “yt-dlp” by default.

set_default_ffmpeg_path(
tmp_path: str | Path | None,
) None[source]

Override default ffmpeg executable path.

set_default_root(
cache_path: str | Path | None,
) None[source]

Override default root directory path.

set_default_ytdlp_path(
java_path: str | Path | None,
) None[source]

Override default yt-dl executable path.

Subpackages

Submodules