Skip to content

Beatwise TF matrix (or multibeat) #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Yoz0 wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Beatwise TF matrix (or multibeat) #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
b26c2f9
Add frames per beat option in compute_beat_sync_features
Nov 1, 2024
81ff573
Add (est/ann)_beatsync_features_mfpb to Features
Nov 1, 2024
d9c5783
Rename ***_beastync_features_mfpb to ***_multibeat_features
Nov 4, 2024
84a4edd
Add option to select features for multibeat and option to change fram…
Nov 4, 2024
bf7f36f
Add tests for multibeat
Nov 4, 2024
cc8e345
Add frames per beat to global params
Nov 7, 2024
c873611
Add multibeat times and add proper padding to multibeat
Nov 8, 2024
3f66345
Add frames per beat parameter to all features
Nov 8, 2024
3853aa5
Add frames per beat to config
Nov 8, 2024
578c41b
Add multibeat parameter to msaf example
Nov 8, 2024
b41452e
Add multibeat param to SegmenterInterface
Nov 8, 2024
a988d28
Add multibeat option in eval process
Nov 9, 2024
d84282d
Rework padding beats to original state
Nov 9, 2024
0525186
Add multibeat parameter to test_run
Nov 9, 2024
0afa7a2
Shape multi beat to beatwise matrix
Nov 10, 2024
51fe348
Add documentation for multibeat
Nov 11, 2024
7050a53
Add test for multibeat
Nov 12, 2024
37e3780
Fix typos
Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/config.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@ import MSAF and print the config variable, as in:
The size of the hop size, which should be smaller than the ``n_fft`` value,
such that overlap is allowed.

.. attribute:: frames_per_beat

Positive int value, default: 3

The number of frames kept per beat when using the multibeat features.
Must be lower than the number of computed between two beats.

.. attribute:: features_tmp_file

String value, default ``'.features_msaf_tmp.json'``
Expand Down
18 changes: 17 additions & 1 deletion docs/features.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ The format of the ``json`` file is as follows:
"audio_file": "<path to audio file>",
"dur": "<duration of audio>",
"sample_rate": "<sample rate>",
"hop_length": "<hop lenght>"
"hop_length": "<hop lenght>",
"frames_per_beat": "<frames per beat>"
},
"metadata": {
"timestamp": "<YYYY/MM/DD hh:mm:ss>",
Expand All @@ -47,6 +48,14 @@ The format of the ``json`` file is as follows:
[ 0.0, 0.0, "..." ],
"..."
],
"est_multibeat": [
[0.0, 0.0, "..."],
"..."
],
"ann_multibeat": [
[0.0, 0.0, "..."],
"..."
],
"params": {
"<param_name1>": "<param_value2>",
"<param_name1>": "<param_value2>",
Expand All @@ -55,6 +64,8 @@ The format of the ``json`` file is as follows:
}
"est_beatsync_times": [ 0.0, 1.0, "..." ],
"ann_beatsync_times": [ 0.0, 1.0, "..." ],
"est_multibeat_times": [ 0.0, 1.0, "..." ],
"ann_multibeat_times": [ 0.0, 1.0, "..." ],
"est_beats": [ 0.0, 1.0, "..." ],
"ann_beats": [ 0.0, 1.0, "..." ]
}
Expand All @@ -67,11 +78,16 @@ A brief description for the main keys of this ``json`` file follows:
* ``ann_beats``: contains the set of reference beats, in seconds (only exists if reference beats are available).
* ``est_beatsync_times``: contains the set times associated with each (estimated-)beat-synchronous feature (might differ with `est_beats` in the beginning or end).
* ``ann_beatsync_times``: contains the set times associated with each (annotated-)beat-synchronous feature (might differ with `ann_beats` in the beginning or end).
* ``est_beatsync_times``: contains the set times associated with each (estimated-)beat-synchronous feature when using multibeat (might differ with `est_beatsync_times` in the end).
* ``ann_beatsync_times``: contains the set times associated with each (annotated-)beat-synchronous feature when using multibeat (might differ with `ann_beatsync_times` in the end).

* ``<feature_id>`` (e.g., ``pcp``, ``mfcc``): contains the actual features of the given audio file. Inside this key the following sub-keys can be found:

* ``framesync``: Actual frame-wise features.
* ``est_beatsync``: Features synchronized to the estimated beats.
* ``ann_beatsync``: Features synchronized to the reference beats (only exists if reference beats are available).
* ``est_multibeat``: Features synchronized to the estimated beats (`frames_per_beat` frames are computed for each beat).
* ``ann_multibeat``: Features synchronized to the reference beats (`frames_per_beat` frames are computed for each beat) (only exists if reference beats are available).
* ``params``: A set of parameters of the actual type of features.

Pre-computed features for the `SPAM dataset <https://github.com/urinieto/msaf-data/tree/master/SPAM>`_ can be found `here <https://ccrma.stanford.edu/%7Eurinieto/SPAM/SPAM-features.tgz>`_.
Expand Down
8 changes: 8 additions & 0 deletions examples/run_msaf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,13 @@
help="Use frame-synchronous features",
default=False,
)
parser.add_argument(
"-mb",
action="store_true",
dest="multibeat",
help="Compute mulitple frames per beat",
default=False,
)
parser.add_argument(
"-j",
action="store",
Expand Down Expand Up @@ -113,6 +120,7 @@
"annot_beats": args.annot_beats,
"feature": args.feature,
"framesync": args.framesync,
"multibeat": args.multibeat,
"boundaries_id": args.boundaries_id,
"labels_id": args.labels_id,
"n_jobs": args.n_jobs,
Expand Down
2 changes: 2 additions & 0 deletions msaf/algorithms/interface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ def __init__(
feature="pcp",
annot_beats=False,
framesync=False,
multibeat=False,
features=None,
**config
):
Expand Down Expand Up @@ -80,6 +81,7 @@ def __init__(
self.feature_str = feature
self.annot_beats = annot_beats
self.framesync = framesync
self.multibeat = multibeat
self.config = config
self.features = features

Expand Down
Loading