Skip to content

Give the option of expecting tables to be present instead of creating them #675

@Simon-Will

Description

@Simon-Will

Description of the issue

We're trying to load the MaStR data for solar plants only into an existing MySQL database. Since we want to retain control of our database state and have changes be done via alembic migrations, we would like to create the tables before instantiating the Mastr object.

We created the tables necessary for the solar download (solar_extended, solar_eeg) ourselves, but of course Mastr tries to create the others. Since we (unfortunately) use MySQL, which unfortunately requires an explicit length for varchar fields (i.e. String(length=some_len) instead of bare String), instantiating Mastr fails with:

sqlalchemy.exc.CompileError: (in table 'basic_units', column 'EinheitMastrNummer'): VARCHAR requires a length on dialect mysql

Ideas of solution

Our issue has two aspects:

  • MySQL is not supported.
  • One cannot prevent Mastr from trying to create tables.

One easy fix for us would be making it possible to skip the table creation step. This is in general a good idea in my opinion to give the user more control over their database if desired. In some environments, this is also necessary because runtime database users aren’t granted permission to use any DDL statements for security reasons. (One can exploit things like function creation in Postgres to get access to the filesystem, arbitrary commands, etc.)

Another step would be supporting MySQL. This requires filling out all the length attributes of the String fields, which is definitely annoying work and may not be very easy to do in a future-proof way.

A third step would be instructing Mastr to only create those tables that are necessary for the intended download.

Workflow checklist

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions