Summary
The feature to edit documents and data from inside the application itself allows for easier access and convenience, however it has side effects that question its purpose.
Extend the documents system to handle syncing data across servers, while also allowing for independent server data that should stay localized.
The current implementation meets the basic needs and provides a blueprint of what the core should strive to be. However it lacks host to host interaction and host separation.
The result is a system that does what it should until it gets a lot of use and attention; it's capabilities are ahead of its infrastructure.
Current implementation
Documents and categories are all globalized, meaning any change made will be out of sync from the repo. There is no interface to communicate with the repo once data has been modified.
Servers can have their own transcribers that can edit documents, however servers will have conflicting versions if the same files are edited on multiple hosts. Thus the cleanest solution is to appoint one host as the sender and all others as receivers.
When admins create new documents and categories, there is no way to distinguish if the content is intended for every server or only for that single host.
How it works
The documents system is composed of documents (title plus content), and categories. (names). The data in this system is not handled by the Fluent localization module, instead it is user generated and read from disc.
The application does more than simply viewing documents via the interface, it has an entire suite of tools for managing data.
Users set a list of fluent languages, and admins can appoint them as transcribers for specific languages.
Transcribers can edit document content, add translations for documents, and localize names of documents and categories.
Admins have additional operations such as creating and removing data.
The issue is that these changes do not do anything beyond saving to disc; the server owner would then have to commit the data to the repo manually.
This might not sound like a big deal, but there are some hosts who will leave their server outdated for weeks or months. Then when they finally do decide to update the server, they then would have to push their changes, which may be outdated by that point.
Shared vs independent data
The easier of the two problems is determining what couns as shared data and what should be independent to a single host. Some questions to ask:
- How should data be identified by both machine and users? What does it mean for each?
- How would the system detect the proper scope that the data belongs to?
- How should the system treat these scopes differently? When is it safe to assume that they are alike?
- Should it be possible to change the scope of data, and if so how would that be handled?
The solution being proposed is separating server/documents into two subfolders: Shared and Independent. The Independent folder would be included in .gitignore.
The _metadata.json file currently only contains categories. Thus for this reason, it would be safe to put this file in both of these folders to have their own categoies.
The system would group shared and independent documents / categories together. A user wouldn't be able to tell whether a document or category is shared or server specific.
The backend would treat something as shared or independent strictly by what subfolder it is present. If a document were to move from the independent folder into the shared folder, that would make it shared automatically.
The ui would allow for admins to move independent data to be shared, but not the other way around. Meaning once independent data is shared, it cannot be undone. This will have to be revised or scrapped entirely to prevent clutter or abuse.
The ui would allow for admins to make a copy of a shared document as an independent version. This is useful for updates that only apply to that server. This feature would only be available for documents, not categories.
Mixed scopes Problem
One of the more less obvious and tricky problems is modifying existing global data to be more appropriate for a single host.
For example, if a host adds new options or variations to an existing game, the documentation for the game rules must be updated to include these new additions.
However, game rules are considered global data, which would naturally apply to all servers. How should this issue be solved?
Possible solutions:
- Allow for copying global data as an independent document. The first issue with this solution is having two versions of the same document. This could be confusing for users, and annoying to maintain. The second issue is whenever the global data gets updated later, the independent copy stays outdated. Would there then be a feature to merge the independent copy with the updated global data? That's a lot of steps, and would require file comparison software, adding more dependencies.
Syncing data across servers
The more challenging aspect of this system is the lack of data syncing.
Without syncing, contributions are not being put together, instead they are fragmented and could even be outdated. If syncing is ignored, the purpose of the documents system is little to no use.
The simplest but worst solution is to disallow editing in the app entirely, requiring all modifications to go through the backend via the repo. While this ensures that all translations will be up to date, it requires using git directly and editing json files by hand.
Furthermore, this solution is not friendly for live server runtime. The server would either need to restart to load the new changes, or it would need functionality to reload the documents system while running. Lastly, data would only be updated whenever the server owner pulls from the repo, which is not ideal.
Discussion
Any suggestions, comments, or refinements to this document are appreciated and will be amended to this post. For all I know, there is a simple solution for this exact situation that I don't know about, so anything is welcome.
Summary
The feature to edit documents and data from inside the application itself allows for easier access and convenience, however it has side effects that question its purpose.
Extend the documents system to handle syncing data across servers, while also allowing for independent server data that should stay localized.
The current implementation meets the basic needs and provides a blueprint of what the core should strive to be. However it lacks host to host interaction and host separation.
The result is a system that does what it should until it gets a lot of use and attention; it's capabilities are ahead of its infrastructure.
Current implementation
Documents and categories are all globalized, meaning any change made will be out of sync from the repo. There is no interface to communicate with the repo once data has been modified.
Servers can have their own transcribers that can edit documents, however servers will have conflicting versions if the same files are edited on multiple hosts. Thus the cleanest solution is to appoint one host as the sender and all others as receivers.
When admins create new documents and categories, there is no way to distinguish if the content is intended for every server or only for that single host.
How it works
The documents system is composed of documents (title plus content), and categories. (names). The data in this system is not handled by the Fluent localization module, instead it is user generated and read from disc.
The application does more than simply viewing documents via the interface, it has an entire suite of tools for managing data.
Users set a list of fluent languages, and admins can appoint them as transcribers for specific languages.
Transcribers can edit document content, add translations for documents, and localize names of documents and categories.
Admins have additional operations such as creating and removing data.
The issue is that these changes do not do anything beyond saving to disc; the server owner would then have to commit the data to the repo manually.
This might not sound like a big deal, but there are some hosts who will leave their server outdated for weeks or months. Then when they finally do decide to update the server, they then would have to push their changes, which may be outdated by that point.
Shared vs independent data
The easier of the two problems is determining what couns as shared data and what should be independent to a single host. Some questions to ask:
The solution being proposed is separating server/documents into two subfolders: Shared and Independent. The Independent folder would be included in .gitignore.
The _metadata.json file currently only contains categories. Thus for this reason, it would be safe to put this file in both of these folders to have their own categoies.
The system would group shared and independent documents / categories together. A user wouldn't be able to tell whether a document or category is shared or server specific.
The backend would treat something as shared or independent strictly by what subfolder it is present. If a document were to move from the independent folder into the shared folder, that would make it shared automatically.
The ui would allow for admins to move independent data to be shared, but not the other way around. Meaning once independent data is shared, it cannot be undone. This will have to be revised or scrapped entirely to prevent clutter or abuse.
The ui would allow for admins to make a copy of a shared document as an independent version. This is useful for updates that only apply to that server. This feature would only be available for documents, not categories.
Mixed scopes Problem
One of the more less obvious and tricky problems is modifying existing global data to be more appropriate for a single host.
For example, if a host adds new options or variations to an existing game, the documentation for the game rules must be updated to include these new additions.
However, game rules are considered global data, which would naturally apply to all servers. How should this issue be solved?
Possible solutions:
Syncing data across servers
The more challenging aspect of this system is the lack of data syncing.
Without syncing, contributions are not being put together, instead they are fragmented and could even be outdated. If syncing is ignored, the purpose of the documents system is little to no use.
The simplest but worst solution is to disallow editing in the app entirely, requiring all modifications to go through the backend via the repo. While this ensures that all translations will be up to date, it requires using git directly and editing json files by hand.
Furthermore, this solution is not friendly for live server runtime. The server would either need to restart to load the new changes, or it would need functionality to reload the documents system while running. Lastly, data would only be updated whenever the server owner pulls from the repo, which is not ideal.
Discussion
Any suggestions, comments, or refinements to this document are appreciated and will be amended to this post. For all I know, there is a simple solution for this exact situation that I don't know about, so anything is welcome.