Extended data mapping rules

[muety]

This feature is based on what was requested in #691, #820 and #822.

Current situation

Currently, Wakapi allows to remap a field's value to an alternative value using the aliases functionality (e.g. map foo-frontend and foo-backend to foo).

Moreover, labels can be used for grouping projects (e.g. assign label work to projects foo and bar).

Lastly, custom language mapping can be defined to map from file ending to language names (e.g. map files ending with .gql to "language" GraphQL).

New functionality

We might substitute these by a single, advanced custom rules feature, where users can define custom mappings (including wildcard patterns, see #607) from any (exposed) field to any other (exposed) field.

Implementation considerations

It is still to be defined how this feature could be implemented on a technical level. All of the above mechanism won't actually modify the underlying heartbeat objects, but only apply the mapping during summary computation. I prefer this mode of operation, because it's "lossless" and you can change or delete your custom mappings at any time without having lost data.

However, the feature requested in #752 (comment) would probably require to rewrite field values already at the time of ingest. Or, alternatively, mapping rules could be as elaborate as to feature a time period they apply to, e.g. "remap language field for project=foo to java for all heartbeats between d1 and d2". This way, @MeltedButter77 would be able to (a) have custom mappings defined while (b) still being able to "reuse" project names (in other words, have different mappings / target values for the same source field value).

I'll have to think about this a bit more thoroughly and it's definitely going to be quite a big change.

[taksh108]

All of the above mechanism won't actually modify the underlying heartbeat objects, but only apply the mapping during summary computation. I prefer this mode of operation, because it's "lossless" and you can change or delete your custom mappings at any time without having lost data.

A consideration about this - While I think this is the approach to go for - I think Wakatime does modify original entries - and it says so that changing mappings is Destructive.
Also - How would new heartbeats be handled? Would they follow mapping rules or map based on current logic.
Am I understanding correctly - that the current plan would recalculate and recreate data in summaries and summary_items tables? Also I feel there'll be a need to restrict it to one operation a time - not allowing to run while a mapping is running.|
Also - thank you for doing this! You are making life easier!

[muety]

My preference would be to implement in a non-destructive way, i.e. only apply the mapping "on read" instead of "on write". It would then apply to existing and new heartbeats equally.

Also, if we drop the above mentioned mechanisms (aliases, project labels and language mappings) in favor of this new, generic data mapping functionality, we'll have to migrate any existing rules to the new "format".

[taksh108]

Hi @muety ,

Just wanted to follow up on this thread and was wondering if you've had any further thoughts on the implementation plan. I would be happy to assist in any way I can, whether it's through testing, brainstorming, or contributing.

Thanks for all your work on Wakapi!

[muety]

Hi @taksh108, thanks for offering your support! I think have a pretty good and detailed plan for this refactoring in my head. It's quite a big one with a lot of different pieces needing to be touched, which is why I didn't get to it yet. It's on top of my roadmap for Wakapi, but I'll need ~ two days of spare time for the implementation. Will hopefully get to it soon.

[ngoomie]

Will it also be possible to exclude certain files/etc from showing up in my Wakapi stats? It seems the WakaTime extension for Neovim reports when I'm looking at documentation or interacting with Tutor files. For my WakaTime I have certain paths like /usr/share/nvim/runtime/doc/ set to be entirely excluded so this doesn't falsely pad out my programming time

[muety]

Data mapping rules will probably only be about renaming / relabeling data. The easiest way to exclude certain files or paths would be to put according configuration via .wakatime.cfg or .wakatime-project in place, see How do I disable WakaTime on certain folders?

[ngoomie]

Oh sweet, I should've checked for something like that. Thanks!