Additional 232 enhancements #98

tanaya-mankad · 2025-10-01T22:26:32Z

Changed "Data Type" to "Type" inside schema and schema parsing, but not inside core.schema.yaml, so that we can still have the syntax
String: Object Type: "Data Type"
Allow 232-compatible descriptor-based type declarations alongside shorthand symbolic declarations. This change majorly benefitted from adding named groups in schema.py's group types and constraint types, and using those named groups in the json and cpp generators.
In order to support repeated group names, we'll use the regex module instead of the built-in re.

…on, cpp.

dodo.py

tanaya-mankad · 2025-10-01T22:28:45Z

lattice/schema.py



+# Module functions
+def unname_group(expression: RegularExpressionPattern) -> str:


Not a great function name, but short.

remove_named_regex_groups?

...or maybe this should be a member function of RegularExpressionPattern called normalize, cleaned, remove_groups?

I can make it a static or class method, then we can make it a member if/when the "base class" is implemented.

lattice/schema.py

tanaya-mankad · 2025-10-01T22:31:19Z

lattice/cpp/header_entries.py

            if this < other:
                dependency_graph.add_edge(this, other)
-    dependency_graph.remove_nodes_from(list(networkx.isolates(dependency_graph)))
+    # dependency_graph.remove_nodes_from(list(networkx.isolates(dependency_graph)))


Not sure why I was removing unconnected nodes; this was causing the Courier declaration to go missing!

tanaya-mankad · 2025-10-01T22:32:52Z

lattice/cpp/header_entries.py

            entry += f"{self._indent}\t{e},\n"
-        entry += f"{self._indent}\tUNKNOWN\n{self._indent}{self._closure}"
+        if "UNKNOWN" not in self._enumerants:
+            entry += f"{self._indent}\tUNKNOWN\n"


ClimateInformation::DataSourceType already has an UNKNOWN enumerant in the schema. We've been adding one in all the other enums just for the cpp code. So, now we check first.

hmmm...do we remember why? This might be worth looking into more later. We can make an issue for now.

My answer is usually, "Chip".

tanaya-mankad

See comments on changes.

tanaya-mankad · 2025-10-06T16:44:34Z

lattice/schema.py

                )
        #    else:
        #        raise Exception(f"Unrecognized Object Type, \"{object_type}\" in {self.file_path}")
+        for group in self.data_groups.values():


Separated the extraction of elements into its own loop, in order to have all DataGroup objects available when any element is populated. This allows for an element's type to find its defining DataGroup and see if custom attributes should be applied based on that DataGroup's parents and/or templates.

tanaya-mankad · 2025-11-25T16:01:29Z

@nealkruis, this is ready for re-review. We should merge this before the recent PR I started with custom-cpp-gen-path.

nealkruis · 2025-10-09T15:21:58Z

lattice/schema.py

+# Module functions
+
+


Looks like a remnant?

nealkruis · 2025-10-09T15:23:42Z

lattice/schema.py


-        self.data_element_name = match.group(1)  # TODO: Named groups?
-        self.data_element_value = match.group(5)
+        self.data_element_name = match.group("DataElementName")


nealkruis · 2025-10-09T15:41:32Z

lattice/schema.py

        self.dictionary = data_group_dictionary
        self.parent_schema = parent_schema
+        self.parent_template: DataGroupTemplate | None = self._assign_template(
+            self.dictionary.get("Data Group Template", "")


If a template name doesn't exist in the schema (even ""), we should raise an exception or log an error.

I'm not sure I understand this - self.parent_template is allowed to be None because not every DataGroup has one.

What happens if a user has a Data Group Template called ""? That should raise an error since that Data Group Template doesn't exist.

Or if they user references a Data Group Template name that doesn't exist, I think the current code would just assign it to None and go along its happy way. Right?

Oh, I see - None isn't the right answer. Fixed!

nealkruis · 2025-11-25T21:51:29Z

lattice/lattice.py

        submodule_urls: list[str] = []

-        config_file = Path(self.root_directory / "cpp" / "config.yaml").resolve()
+        config_file = Path(self.root_directory / "cpp" / "0config.yaml").resolve()


Looks like a typo?

nealkruis · 2025-11-25T21:53:50Z

dodo.py

            "name": name,
            "task_dep": [f"validate_schemas:{name}", f"generate_json_schemas:{name}", f"validate_example_files:{name}"],
-            "file_dep": [schema.file_path for schema in example.schemas]
+            "file_dep": [schema.schema.file_path for schema in example.schemas]


I'm slightly bothered by "schema.schema". Why is it necessary?

Yeah, it's unfortunate. We can do it a different way, but the reason this exists is because of how lattice.py changed its interaction with what's now schema.py. Used to be that the schema class held its own json_schema_path, meta_schema_path, and cpp_path. Those no longer exist, so I encapsulated them with the schema in a dataclass called SchemaSupport.
Lattice.schema is now a list[SchemaSupport] instead of a list[Schema].
It's a hack for sure, but it's also structurally sound!

I see. We need a structure to relate the various paths with the Schema since the Schema class doesn't actually know about these paths. I think we just need better names then.

What about Lattice.schema_supports?

Ooops. I see you already changed it to schema_info. I'm okay with that. But the CI is now angry.

nealkruis and others added 4 commits September 17, 2025 16:15

Switch "Req" to "Required".

f2ee8ed

Change 'Data Type' to 'Type' (not in Object Type).

5d17132

Fix mypy errors.

e60b54b

Added named groups to schema, enabled multiple parsing methods for js…

7128104

…on, cpp.

tanaya-mankad requested a review from nealkruis October 1, 2025 22:26

tanaya-mankad self-assigned this Oct 1, 2025

tanaya-mankad commented Oct 1, 2025

View reviewed changes

dodo.py Show resolved Hide resolved

tanaya-mankad commented Oct 1, 2025

View reviewed changes

lattice/schema.py Show resolved Hide resolved

tanaya-mankad commented Oct 1, 2025

View reviewed changes

lattice/schema.py Show resolved Hide resolved

tanaya-mankad commented Oct 1, 2025

View reviewed changes

tanaya-mankad added 2 commits October 1, 2025 17:25

Put regex package under dependencies.

7efaec4

Add Custom Attributes, fix my errors.

1e56673

tanaya-mankad commented Oct 6, 2025

View reviewed changes

tanaya-mankad added 2 commits November 14, 2025 15:30

Add more schema.py support.

983ddbd

Import fix from later branch.

185d31c

nealkruis requested changes Nov 25, 2025

View reviewed changes

tanaya-mankad added 3 commits December 1, 2025 18:16

Address PR comments.

62ea627

Raise error if template can't be found.

c97f9de

Add self to list of schemas referenced for Data Group Templates.

78b8206

tanaya-mankad requested a review from nealkruis December 3, 2025 22:17

tanaya-mankad added 3 commits December 3, 2025 16:02

Change name of schema info list.

dcff49a

Make sure f-string formatting is correct.

4174840

Format f-string correctly for python <3.12.

24fa684

nealkruis merged commit 9aa8adc into main Dec 23, 2025
14 checks passed

nealkruis deleted the additional-232-enhancements branch December 23, 2025 16:33



		# Module functions
		def unname_group(expression: RegularExpressionPattern) -> str:

Additional 232 enhancements #98

Additional 232 enhancements #98

Uh oh!

Conversation

tanaya-mankad commented Oct 1, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tanaya-mankad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tanaya-mankad commented Nov 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants