Roadmap

[YannickB]

Hello everyone,

Ok so after Odoo Experience I have a bigger idea of what still need to be done in order to create an awesome service, here are the features planned in the next months, by order or priority :

• Docker Swarm. We really need to have a clustering system and isolate component inside an overlay network for security.
  Also note that a partner want to use OpenShift instead, we're gonna make it so you can have different runner for Clouder, regular Docker Engine, Docker Swarm, Amazon ECS or OpenShift.
• Third party providers : Clouder will always be able to manage internally DNS, Load balancing, Mails and other stuffs like that, but for critical infrastructures it is often best to rely on big companies like Amazon.
  We will integrate the libcloud library in Clouder so we can provision instances/containers/dns/other stuffs directly from Clouder. The link logic will be the same, but for example when you deploy a new base it'll not be linked to a DNS bind container but to a dummy container which on deploy will create a DNS record in Route53 or another provider.
• New instance creation features : We need to improve the frontend, with important features such as trial period, expiration, email verification, transactional email, manage invoicing based on the number of users in deployed base.
• Resource management : We should be able to specify in the container form a limit on the ressource (CPU/RAM/Disk) which can be assigned to this container. This resources shall be inherited from application.
• Improve subscription form : We need to be able to select a list of features (module to install, add more CPU etc...) defined in the application which can then be selected in the front. The price will be displayed and automatically updated according to the selected features.
  Also, the visitor shoud be able to specify or buy his domain from the front.
  And the design need to be greatly improved.
• Monitoring : Clouder currently use Shinken which is probably not the best tool today. We are considering the use of Elastic Stack for this purpose and for better log management in the Clouder infrastructures.
• Access right and CI : We need to make sure that access right are correctly configured so customer can manage their instance without having access to other customer instance. They will also have access to the gitlab repo of their instance.
  Salt log need to be improved (we want to see each result in clouder.job object) and the deployment log should also be visible from gitlab-ci (right now we just call a webhook)
• Use Alpine as base image : We currently use the official ubuntu image as base, which weight around 200Mo. With all image and container to install, this means a complete oneclick deployment weight around 20-30Go which is very impressive for newcomers.
  I think it's really important to use a light base image like Alpine (recommanded by Docker), we probably can lessen the Clouder installation to around 5Go thanks to it, if not less. Remember that Dockerfile of Clouder images are here : https://github.com/clouder-images
• Download backup from Clouder interface : In order to give customer better control on their data, it'd be really good to have a download button on the save object, which will connect to the backup container and download the backup.
  Having an import feature would be also really interesting for migration from existing infrastructure, so we don't have to do the operation manually in the terminal. We'd just need to import the backup in the backup container and, provided the files are in the correct format, the Clouder restore feature will take care of the rest.
• Self installation of Clouder : Create a Docker Compose file which install a Clouder, capable of auto-backing himself. To give you an idea, on our main site we have an initialisation Clouder, which deploy and backup goclouder.net, which itself deploy and backup the others websites. We need to find a way for Clouder to be easier to deploy and autonomous.
• SSO/Portal : Centralize authentification in application hosted by Clouder in a central SSO platform
• Documentation : The current doc.goclouder.net is now vastly outdated (since summer 2015), it's really important to update it to easy the pain for newcomers who want to contribute.

[lasley]

While abstracting out the container management, it would be nice if we also considered Amazon ECS as a potential runner.

[YannickB]

+10 you're right for Amazon ECS

[lasley]

I haven't actually dug into the DNS controllers yet, but it would be nice to support Route53 instead of just bind. Check these out:

• infrastructure-dns/connector_dns
• preliminary Route53 connector based on connector_dns

I would be happy to make one for Bind9 - should be pretty simple.

[YannickB]

Hum.. +1, I'm in favor to use closed service for ensuring proper quality if the embedded open-source application is not enough.

According to me we shall deploy a dummy "Route53" container (which is of course not a real one) and make it so it can be selected in the place of the bind container in the base. I'll try to make a POC asap, without the code for route53.

Also, I think it's time to revive the master branch, most change which will be done starting from now will break existing infrastructure.

[lasley]

Sweet I look forward to the PoC.

Also agree on the need for an incubator branch, what if we consider it as a v10 incubator? Major version increments are the best time to make these sorts of changes IMO

[YannickB]

Yes it's going to be the incubator of 0.10.0 (remember we don't follow the Odoo version anymore.

[lasley]

Maybe the completion of the incubator is the right time for the OCA migration too? Starting with a fresh production in a fresh repo will make it even more clear that the products are incompatible.

[YannickB]

Yep, well we can migrate there anytime we want anyway. Any news on this side ?

[lasley]

We have incredible buy-in (3-4 delegates), so I'm certain it's happening.

We won't be able to move it in before we fix things like the #141 & some other OCA things like naming standards for columns and ReadMes, so it makes sense to migrate over after the incubator.

My team knows these OCA standards pretty well, so we can handle that for the most part. It's a lot of grunt work, so I'll have some juniors handle it for us most likely. I also have a set of migration regexes that help out quite a bit.

It seems a lot of devs have been working towards similar goals on their own, and all agree that what you have here wins.

Most recent message in the thread - https://odoo-community.org/groups/contributors-15/contributors-46578?mode=thread&date_begin=&date_end=

[YannickB]

Wow ok I wasn't aware that this thread was gotten so big o_o, will read it later.

I'm already working on #141, as I said I think it's not as big as it seems.

[KaloyanNaumov]

For DNS management, I would suggest you to consider the CloudFlare DNS services, as they are free and available over APIs.
There is a Route53 implementation in the IT-projects-LLC's SaaS set of modules.

Also, it would be valuable to have an SSO/Identity Management, which could be Gluu. They are working on a Docker version, which can be adopted.

Security Standards Compliance and Hardening belongs in a different issue, IMO.

[lasley]

Hmmm so maybe an implementation using LibCloud, then a simple selection to determine the driver for the intended service

[YannickB]

Wow LibCloud looks like a golden mine to me, that's exactly what I hoped to find.

I think we are going to have more and more demand for using external service so here is my position :

• For each component (DNS, Load Balancer, monitoring, etc...) we should have one and only one open-source component inside Clouder templates (Bind, Postfix, Shinken etc...). This open-source component shall be chosen and elected among all as being the best available and will be deployed if you want all your infrastructure on your server (as do Clouder today).
• If you want to use external service, they should be proposed as alternative to the internal open-source component. I though we'd need to make one connector for each services, but I hoped that one project would centralize the connector and it seems that's exactly what LibCloud is doing so I am for using it.
  In general, external service will greatly expand the reliability of a Clouder infrastructure until all internal component become mature and secure, so I consider them as really important during the first steps.

Finally, to me SSO/Identity Management/Portal are really important and will be a core feature of Clouder. We don't have a template for them yet but know that's a component which I think about since a really long time, it will be the core of Clouder for the user, where they log into, where they will be redirected to the applications hosted in the Clouder infrastructure (not just odoo, but gitlab, owncloud, rainloop, mattermost and so on). I am aiming for the same user experience we have when we use google applications, nothing less and SSO/Portal are key for this.
I am still looking for the right internal component for this part, it seems Gluu is open-source behind so it may be a choice to consider.

[lasley]

Regarding external services, I think the way to manage them is via the Connector libraries. We can be naive of the external service itself, simply providing a dummy model like you did.

That dummy model can then be inherited by a module to glue a connector into it. The best part is, it doesn't even need to be glued by service type because the connectors themselves are abstract.

For example - given the connector_dns repo, the glue module would create a polymorphic _inherits between the Clouder Domain with the Connector's DNS Record. Add a bit of logic to map the two records, then the DNS synchronization is handled via the external connector library instead. This allows us to maintain a service-agnostic approach to the whole situation - all we do is write to a dummy model.

[KaloyanNaumov]

The research I did on IDP/SSO solutions shows Gluu as the better open source SSO/IDP, compared with OpenAM and Shibboleth among others.

This seems to be good addition:
Sentry There's an Odoo module too.

And was wondering what would be your choice for centralised logs collection solution:
http://www.fluentd.org - https://github.com/fluent/fluentd/
https://www.elastic.co/products/logstash - https://github.com/elastic/logstash
https://www.graylog.org - https://github.com/Graylog2/graylog2-server