aac_datasets.datasets.functional.clotho module

class ClothoCard

Bases: DatasetCard

ANNOTATIONS_CREATORS: Tuple[str, ...] = ('crowdsourced',)

CAPTIONS_PER_AUDIO: Dict[str, int] = {'dcase_aac_analysis': 0, 'dcase_aac_test': 0, 'dcase_t2a_audio': 0, 'dcase_t2a_captions': 1, 'dev': 5, 'eval': 5, 'val': 5}

CITATION: str = '\n    @inproceedings{Drossos_2020_icassp,\n        title        = {Clotho: an Audio Captioning Dataset},\n        author       = {Drossos, Konstantinos and Lipping, Samuel and Virtanen, Tuomas},\n        year         = 2020,\n        booktitle    = {Proc. IEEE Int. Conf. Acoustic., Speech and Signal Process. (ICASSP)},\n        pages        = {736--740},\n        abstract     = {Audio captioning is the novel task of general audio content description using free text. It is an intermodal translation task (not speech-to-text), where a system accepts as an input an audio signal and outputs the textual description (i.e. the caption) of that signal. In this paper we present Clotho, a dataset for audio captioning consisting of 4981 audio samples of 15 to 30 seconds duration and 24 905 captions of eight to 20 words length, and a baseline method to provide initial results. Clotho is built with focus on audio content and caption diversity, and the splits of the data are not hampering the training or evaluation of methods. All sounds are from the Freesound platform, and captions are crowdsourced using Amazon Mechanical Turk and annotators from English speaking countries. Unique words, named entities, and speech transcription are removed with post-processing. Clotho is freely available online (https://zenodo.org/record/3490684).}\n    }\n    '

DEFAULT_SUBSET: str = 'dev'

DEFAULT_VERSION: str = 'v2.1'

HOMEPAGE: str = 'https://zenodo.org/record/3490684'

LANGUAGE: Tuple[str, ...] = ('en',)

LANGUAGE_DETAILS: Tuple[str, ...] = ('en-US',)

NAME: str = 'clotho'

N_CHANNELS: int = 1

PRETTY_NAME: str = 'Clotho'

SAMPLE_RATE: int = 44100

SIZE_CATEGORIES: Tuple[str, ...] = ('1K<n<10K',)

SUBSETS: Tuple[str, ...] = ('dev', 'val', 'eval', 'dcase_aac_test', 'dcase_aac_analysis', 'dcase_t2a_audio', 'dcase_t2a_captions')

TASK_CATEGORIES: Tuple[str, ...] = ('audio-to-text', 'text-to-audio')

VERSIONS: Tuple[str, ...] = ('v1', 'v2', 'v2.1')

download_clotho_dataset(

root: str | Path | None = None,

subset: str = 'dev',

force: bool = False,

verbose: int = 0,

verify_files: bool = True,

clean_archives: bool = True,

version: str = 'v2.1',

) → None

Prepare Clotho data.

Parameters:

• root – Dataset root directory. defaults to “.”.

• subset – The subset of Clotho to use. Can be one of SUBSETS. defaults to “dev”.

• force – If True, force to download again all files. defaults to False.

• verbose – Verbose level. defaults to 0.

• verify_files – If True, check all file already downloaded are valid. defaults to False.

• clean_archives – If True, remove the compressed archives from disk to save space. defaults to True.

• version – The version of the dataset. Can be one of versions. defaults to ‘v2.1’.

download_clotho_datasets(

root: str | Path | None = None,

subsets: str | Iterable[str] = 'dev',

force: bool = False,

verbose: int = 0,

clean_archives: bool = True,

verify_files: bool = True,

version: str = 'v2.1',

) → None

Function helper to download a list of subsets. See download_clotho_dataset() for details.

load_clotho_dataset(

root: str | Path | None = None,

subset: str = 'dev',

verbose: int = 0,

version: str = 'v2.1',

) → Dict[str, List[Any]]

Load Clotho metadata.

Parameters:

• root – Dataset root directory. defaults to “.”.

• subset – The subset of Clotho to use. Can be one of SUBSETS. defaults to “dev”.

• verbose – Verbose level. defaults to 0.

• version – The version of the dataset. Can be one of versions. defaults to ‘v2.1’.

Returns:

A dictionnary of lists containing each metadata.