NORMA as a graph database modeling tool?

[eastshores]

Hey there! I'm really thrilled to see this project has stayed alive and that there is an easily added extension that works in VS 2022! That's a testament to the maintainers for sure, as I know this project goes back a long ways. My career has taken me away from standing up new data models so I only recently started thinking about ORM and NORMA after looking at graph databases and thinking to myself, "this feels like ORM would be a perfect fit"

I am still ramping up on graph databases theory wise, so I don't have a firm grasp on direct concept translations when it comes to edges vs. foreign keys, or how the NORMA schema model might translate, but I was wondering if you are aware of ORM being used to design graph db schemas? I came across some posts by Victor Morgante discussing this that seemed to reinforce the notion.

I would be interested in looking into possibly generating Apache Gremlin API commands that could establish a schema using vertexes, edges, and properties. There are some higher level concepts like labels that I don't think ORM has a representation for - but I've been away from ORM for so long I really don't know. Would be interested to know your thoughts.

[k-grosser]

Hi!

there is definitively some research going on, but I'm not aware of a complete and out of the box solution (maybe someone else nows more).

There are two Master's thesis from 2017 that deal with the topic:

Alicia Owen. “Mapping ORM to TGraph”. MA thesis. University of Koblenz-Landau, 2017.
https://kola.opus.hbz-nrw.de/index.php/frontdoor/index/index/year/2017/docId/1537

Marios Braimniotis. “A Transformation from ORM Conceptual Models to Neo4j Graph Database”. MA thesis. Radboud University Nijmegen, 2017.
https://www.ru.nl/publish/pages/769526/4322843_braimniotis_marios_-1a.pdf

I was involved in the first one. However, there we used TGraphs, which are a nice fit, as they are schema-based, but sadly this technology is a research prototype and not maintained anymore. Nevertheless, some general concepts should be applicable for more modern graph databases, too. In some other project, we did a manual mapping to a Neo4J database and used Cypher queries to check constraints and derivations - for sure, most of this should be possible to generate automatically.

[eastshores]

That's great info. I will take a look at the links. As I've been reading more I am starting to understand most graph dbs don't actually have a defined schema intentionally, it's more that you can reverse generate a schema at any given time based on what data is in the graph. There are also limited constraints support in many graph dbs .. for instance in cosmos graph db, the constraints are basically limited to a uniqueness constraint for entity ID within one partition. Not exactly the type of physical model that would benefit from the expressive constraints that ORM generates. Still might be useful to generate the API based on the ORM entities (nodes) and facts (edges). Thanks for the reply!

[mcurland]

Hi John,

I'm glad you're happy to find that NORMA is still alive and kicking.

I'll be the first to admit that I have minimal exposure to graph databases, effectively limited to a couple of hours reading with a focus on potentials for mapping to ORM. My initial impression is that a mapping could be achieved with annotation directly on native ORM elements (object types, fact types and roles). Direct annotation is an important distinction when compared to a relational database, which has a much different structure than an ORM model and cannot be sufficiently customized by marking up ORM elements. Columns are composed of chains of roles, so a fact type or role is frequently associated with multiple columns, making direct ORM markup insufficient for relational mapping.

I think the first question to ask is whether the notion of a 'graph database' is sufficiently defined that we can create a sufficient logical representation of a generic graph database that could then be rendered to multiple physical implementations. This is the approach taken with relational mapping, where there are multiple levels of representation: the data in the .ORM file has basic tables, columns and key/ref constraints, this is mapped to a DCIL XML format (adds data types, domains, check constraints), then DDIL XML (a generic XML representation of DDL), then finally RDBMS-specific DDL statements. (You can see all of these intermediate representations as children of the .orm model node in the VS Solution Explorer). The physical implementation is really not even considered until the final rendering step. The question here is whether a similar approach can be taken with a generic logical graph database structure that can be rendered to a specific API/schema as a final step.

The costs of adding NORMA features (the cost-scale on this is logarithmic, not linear).

1. Define a set of custom properties and produce a generator transform to interpret the data (and give feedback, since custom properties are simply dumb data holders).
2. Define an extension model that collects customization data for a generator as annotations on existing elements. This can move validation and feedback much closer to the user. Some automation is also possible here, such as name generation.
3. Create a more sophisticated model that has an ORM Model Browser representation of the output structure. This allows things like multiple representations of the structure, easier customization and an initial virtual data structure for a generator.
4. A custom graphical representation of the construct.

Similar Mapping

(This is off topic, but related to the above list). If you look on the https://youtube.com/@ORMSolutions channel you'll see some videos that were done for a recent ESA (European Space Agency) conference. They currently have a large initiative using ORM to (re)model large space systems.

Much of the content focuses on work done for what I call the JavaScript Fact Engine (JFE), which is a framework designed to represent an ORM model, along with corresponding back-end generated code to read a model-based JSON query and write a JSON-ized StateChange. The two main objectives are:

1. Implement a full-stack application with minimal code that is not generated directly from the .ORM model.
2. Communicate between layers in terms of model elements (objects and facts), not physical artifacts (tables and columns).

The generated code and related analysis pieces are already advanced (I can import any JFE StateChange data to a relational database with a single web method, including changes as large as the import of an entire ORM model to the database), I am still on stage 1 (custom property definitions) in the list above, meaning that I need to manually add JFE custom properties to include fact types and object types in the system. I obviously plan to move this to stage 2 (custom extension and automatic model generation and naming), but it is unlikely I will push it to stage 3 simply because the structure closely mirrors the ORM model itself.

To circle back, as this is a JavaScript system, the framework runs outside the browser (NodeJS, etc.). However, while there some relational mappings systems I could attempt to target, the relational database backend is not necessarily preferred in this environment, so having a mapping that can define/read/write alternate systems (graph, document, etc.) would be a welcome enhancement.

Cooperation?

I'd be happy to do a call on this where we can discuss possible ways to move this forward.

[git-ogotribe]

Hi Matthew,

I just accidentally bumped onto this thread ;-) Have you ever looked at TypeDB as a potential storage back-end? It's schema approach seems very similar to the ORM2 metaphor.

Have a good one

Rob

[mcurland]

Hi Rob,

No, I haven't looked at TypeDB. The number of different potential targets seems to grow monthly and can be a little overwhelming. Basically, the same analysis process applies as above: see if there is a mapping and if it can be automated.

Of course, one desirable effect is to enable one set of markup to be used for multiple targets. If the markup can be applied directly to object types, fact types and roles--instead of producing a derived customizable structure like a relational model--then the costs of producing it from NORMA become much lower.

-Matt

PS Naturally as an ORM guy I'm not all that partial to ER-styled systems. Some time in the ER-based FHIR system (for medical data) really just strengthened my distaste as there were a ton of modeling biases that seeped into the final data structures. For example, querying data for forward and reverse relationships is much different in the system, so you had to know the owning end for EVERY link in the system. This was frankly absurd as it made basic navigation heavily dependent on the endpoint you started at (doctor->patient was much different than patient->doctor). I'd be curious if TypeDB has similar biases.

PPS If you know the system well, the same invitation to sit down and review possible markup is open here as well. I just need something a little more concrete than curiosity to invest time into new targets.