Skip to content

Add time embeddings, use them for sorting#8

Merged
ruuda merged 19 commits intomasterfrom
daytime-decay
Apr 29, 2025
Merged

Add time embeddings, use them for sorting#8
ruuda merged 19 commits intomasterfrom
daytime-decay

Conversation

@ruuda
Copy link
Copy Markdown
Owner

@ruuda ruuda commented Apr 29, 2025

Background

An idea that I've had for a very long time, and I now finally got to implementing: music is very seasonal for me, on multiple levels. Some albums are good for a hot summer night, while others are good for a rainy autumn day. I listen to more chill music early in the morning, more upbeat music during the day and evening. I listen to different music on weekends than on weekdays.

All of these are cyclic, all of them have a “center” modulo a year, week, and day. So what if we generate vectors, and embed these points in time in ℝ⁶ (technically, S¹×S¹×S¹ embedded in ℝ⁶)? Then adding/averaging them becomes meaningful, and every track, album, and artist can have a “preferred time”. We can use cosine distance to find albums suitable for the current time of the day/week/year.

So I implemented this. It works. It works surprisingly well, actually. I planned to use this as one feature to feed into a neural network, but even the basic cosine distance works quite well.

Performance impact

It sounds kind of expensive to do all these float operations on every listen when replaying the history, and to recompute the score on every request, however it seems fine. On my (reasonably fast) laptop, musium count — which runs the count for the entire history in the database and then ranks in various ways — runs in 280 ms with 45k listens, and it’s probably not even bottlenecked on these new operations. On my Raspberry Pi 4, loading playcounts takes less than a second (the logs only have second granularity), and curling the /api/albums endpoint locally on the Raspberry takes 40.6 ms ± 7.3 ms, so nothing to worry about. Memory usage grows too in theory, but I observe 51–54MB at idle for my collection both before and after, so this is not a problem either.

ruuda added 19 commits April 29, 2025 22:23
I have this idea in my head for months, let's actually try it this time.
Music is very seasonal for me, I have tracks that are better suited for
a hot summer night, tracks that are better suited for a grey autumn day.
Also, time of the week matters, I listen to chill music on a Saturday
or Sunday morning, I listen to bangers more in the evenings. So I
thought, what if we create a moving average for a track (and album, and
artist) to track the time of the year, week, and day? Then we can make
better suggestions for what to listen to based on the current time.

This adds the types for that, and also a mapping of weekdays to the
circle, based on me toying around with Geogebra.
I'm using Helix now and it formats on save, the diff is not too big,
let's accept it.
They don't look right to me, the months are skewed too much towards
spring, and days too much towards Thursday, though maybe that is due
to me meddling and not providing the proper inverse mapping.

I need to write some tests to sort this out.
Okay the weekdays were wrong, but the rest of it was right. Weird. Let's
see again if it makes sense now.
After eyeballing the output, it seems to work better without this.
In the end this piecewise-linear mapping to the circle is a lot simpler
than the manual tweaking I did before, and at least on paper it has very
nice properties for distinguishing between weekends and weekdays, party-
nights and quiet nights.
I was curious about whether this would work, and from looking at the
numbers, it works very well. See also the doc comment.
It does a decent job from what I can tell, but on its own this score is
no good, because it doesn't take popularity into account, so it will
suggest obscure tracks just because I played them once almost exactly a
year ago. But as a multiplier for the discovery ranking, I think it
would do very well. Let me check!
This is needed to expose it through the API, though right now we do not
take it into account, that will be in a follow-up commit. For now I'm
not worrying about memory usage.
So we can use it as a weight in a "for now" ranking.
This integrates the new "for now"-weighed discover scores, and it also
exposes a more raw "for now" score that needs to be integrated into the
UI still.

This includes some rustfmt reformattings, but not everything it would
reformat in the same files; some changes are very invasive and I want
to avoid doing that here.
Albums without plays have a zero norm on their time embedding, but we
should just return zero scores for them, not compute anything. It makes
the call site prettier too.
At least, on my viewport, on narrower screens it may still not fit. Not
a great solution there, I think it would need an (implicit) scrollbar.
Bot for now (haha, see what I did there) it works.
I end up doing only mul_adds.
@ruuda ruuda merged commit 946d135 into master Apr 29, 2025
1 check passed
@ruuda ruuda deleted the daytime-decay branch April 29, 2025 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant