Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
205 commits
Select commit Hold shift + click to select a range
5cc49e5
fix setup.py confict
aaraney Jan 25, 2023
7ad9700
add PydanticEnum. validated by enum member name.
aaraney Jan 9, 2023
f931db6
add PydanticEnum unittests
aaraney Jan 9, 2023
899b8a0
Serializable subclasses pydantic.BaseModel and update docs
aaraney Jan 9, 2023
6566833
add deprecated decorator function
aaraney Jan 9, 2023
e53390d
add unit test to verify deprecated warning is raised by decorator
aaraney Jan 9, 2023
5a877f1
add deprecation warning to parse_simple_serialized
aaraney Jan 9, 2023
c399880
fix docstring typo
aaraney Jan 9, 2023
d92bfb3
refactor SerializedDict, ResultIndicator, and BasicResultIndicator
aaraney Jan 9, 2023
7f6b917
refactor AllocationParadigm
aaraney Jan 9, 2023
744d62b
refactor meta_data enums
aaraney Jan 9, 2023
9b415b2
refactor ContinuousRestriction to use pydantic
aaraney Jan 9, 2023
b633f08
add default implimentation for factory_init_from_deserialized_json
aaraney Jan 9, 2023
e7673ff
add comment to SerializedDict
aaraney Jan 9, 2023
775e642
refactor BasicResultIndicator
aaraney Jan 9, 2023
56dc334
refactor DiscreteRestriction
aaraney Jan 9, 2023
d5d0f09
refactor DataDomain
aaraney Jan 9, 2023
9924c14
refactor DataRequirement
aaraney Jan 9, 2023
8bd316b
remove unnecessary factory_init_from_deserialized_json's
aaraney Jan 9, 2023
f61d5dc
remove unneeded class variables
aaraney Jan 9, 2023
665c415
remove ellipse
aaraney Jan 9, 2023
adb5c03
refactor dataset module enums
aaraney Jan 9, 2023
3fe7c46
refactor Dataset
aaraney Jan 9, 2023
12010b6
add bug fix todo's
aaraney Jan 9, 2023
f71e965
fix **kwargs bug
aaraney Jan 9, 2023
4f2fbe3
clean up imports
aaraney Jan 9, 2023
87401b4
update unittests. no breaking changes made
aaraney Jan 9, 2023
7339714
serialize UUID using explict cast to str
aaraney Jan 9, 2023
215de3b
update dataset unittest
aaraney Jan 9, 2023
e8dd16a
adjust factory init logic in ContinuousRestriction
aaraney Jan 9, 2023
48308b0
add meta_data module unit tests
aaraney Jan 9, 2023
1f8a240
remove unnecessary return by_alias check. now handled by superclass, …
aaraney Jan 10, 2023
8e691f9
handle coercing bool str repr into python type
aaraney Jan 10, 2023
82053ea
add method that encodes python built in types as str (i.e. 'int') and…
aaraney Jan 10, 2023
50d1b8b
correctly deserialize DataDomain's custom_data_fields field
aaraney Jan 10, 2023
25ad089
ensure restriction fields have empty list by default
aaraney Jan 10, 2023
ad5caca
add data domain unit tests
aaraney Jan 10, 2023
c0df060
add pydantic to root requirements.txt
aaraney Jan 10, 2023
7a63bc8
DataDomain, handle case when custom_data_fields=None
aaraney Jan 10, 2023
a5292ab
remove erroneous TODO comments
aaraney Jan 10, 2023
6332e96
@robertbartel confirmed in #239 that category and data_domain are not…
aaraney Jan 11, 2023
1394783
remove commented out Dataset factory_init_from_deserialized_json
aaraney Jan 11, 2023
22ed48b
fix DataDomain's dict method. related to #245
aaraney Jan 18, 2023
2ecec46
revert DataRequirement's __eq__ implimentation
aaraney Jan 19, 2023
d58536f
add note to remove DataDomain.custom_data_fields in future. relates n…
aaraney Jan 19, 2023
f4758f1
exclude custom_data_fields during serialization if empty T variant.
aaraney Jan 19, 2023
ec617ff
add field_serializers Config option to Serializable
aaraney Jan 23, 2023
80a8e22
fix type hints
aaraney Jan 23, 2023
070b57a
test Serializable field_serializer Config option
aaraney Jan 23, 2023
2cce097
simplify Dataset's dict override with config field_serializer
aaraney Jan 23, 2023
8995070
refactor ContinuousRestriction to use config field_serializers
aaraney Jan 23, 2023
e291007
DataDomain now factory inits continuous_restrictions
aaraney Jan 23, 2023
4ac8800
remove commented code
aaraney Jan 23, 2023
489b787
fix test that was failing for the right reason
aaraney Jan 23, 2023
34f712e
bump dmod.core patch version to 0.4.2
aaraney Jan 26, 2023
0b80282
fix comm setup.py merge conflict
aaraney Jan 24, 2023
208675b
dmod.communication enum's now subclass PydanticEnum
aaraney Jan 11, 2023
7734ac8
refactor Message
aaraney Jan 11, 2023
c5c1333
remove unnessesary __init__ in AbstractInitRequest
aaraney Jan 11, 2023
b0da87d
refactor Response
aaraney Jan 11, 2023
f790e27
add TODO. fill in Union of data field types once refactored
aaraney Jan 11, 2023
7f726e3
refactor InvalidMessage
aaraney Jan 11, 2023
b29751a
refactor Response and its subtypes
aaraney Jan 11, 2023
96e4fce
refactor MetadataMessage
aaraney Jan 11, 2023
facf9a1
add MetadataSignal type
aaraney Jan 11, 2023
99d3393
refactor metadata message and response
aaraney Jan 11, 2023
7958201
add generate_secret helper function
aaraney Jan 12, 2023
8f7a1a9
refactor Sesssion
aaraney Jan 12, 2023
ea3634b
refactor FullAuthSession
aaraney Jan 12, 2023
76b13a7
refactor SessionInitMessage
aaraney Jan 12, 2023
06bc81f
refactor FailedSessionInitInfo
aaraney Jan 12, 2023
0c7aff3
refactor SessionInitResponse
aaraney Jan 12, 2023
05f1853
refactor UnsupportedMessageTypeResponse
aaraney Jan 12, 2023
ad87928
format unsupported_message module
aaraney Jan 12, 2023
c108eef
refactor UpdateMessage
aaraney Jan 12, 2023
aabf8c7
add UpdateMessageData; inner type of UpdateMessageResponse
aaraney Jan 12, 2023
0eadac1
refactor UpdateMessageResponse
aaraney Jan 12, 2023
0238e95
refactor SchedulerRequestMessage
aaraney Jan 12, 2023
f6bd552
refactor parameter module
aaraney Jan 12, 2023
787d9b0
refactor DmodJobRequest
aaraney Jan 12, 2023
7b948b1
refactor distribution module
aaraney Jan 12, 2023
70e1035
refactor ExternalRequestResponse
aaraney Jan 12, 2023
358e5da
refactor DatasetQuery
aaraney Jan 17, 2023
1377def
refactor DatasetManagementMessage
aaraney Jan 17, 2023
9969f32
refactor MaaSDatasetManagementMessage
aaraney Jan 18, 2023
6267ab9
refactor PartitionRequest
aaraney Jan 18, 2023
1d1449b
refactor PartitionExternalRequest
aaraney Jan 18, 2023
3bf30e7
fix bug in PartitionRequest __init__
aaraney Jan 18, 2023
d9376b5
refactor EvaluationRequest
aaraney Jan 18, 2023
702ea32
refactor EvaluationConnectionRequest
aaraney Jan 18, 2023
5e9c241
fix typing hints, default value, and retrieval of action in Evaluatio…
aaraney Jan 18, 2023
56ec508
add ActionParameters class. encapsulates evaluation_name and instruct…
aaraney Jan 18, 2023
7ae2d07
refactor StartEvaluationRequest
aaraney Jan 18, 2023
66a28ec
refactor DataTransmitMessage
aaraney Jan 18, 2023
a0e1c88
refactor dict method logic to follow pattern foundelse where in the c…
aaraney Jan 18, 2023
37cac10
complete refactor of UpdateMessageResponse
aaraney Jan 18, 2023
56716a6
add missing parameters field to StartEvaluationRequest
aaraney Jan 18, 2023
6c875d8
fix bug in validating Sesssion dates
aaraney Jan 18, 2023
77c22d3
explicity cast from IPAddress type into str.
aaraney Jan 18, 2023
ad8833d
fix validation of success
aaraney Jan 18, 2023
0ffca37
fix FailedSessionInitInfo's fail_time serialization formatting
aaraney Jan 18, 2023
5ce9223
try and coerce SessionInitResponse' data field in correct order
aaraney Jan 18, 2023
1163238
add NWMRequestBody and NWMInnerRequestBody
aaraney Jan 19, 2023
25ed174
refactor NWMRequest
aaraney Jan 19, 2023
92efb6f
adds ModelExecRequestBody. refactored abstract encapsulation of model…
aaraney Jan 19, 2023
c5495d5
improve get_available_model type hints
aaraney Jan 19, 2023
90f42f0
refactor ModelExecRequest.
aaraney Jan 19, 2023
b25abf0
refactor NGENRequest
aaraney Jan 19, 2023
45586a1
refactor SchedulerRequestResponse. split it's data property into its …
aaraney Jan 20, 2023
5e09154
add NGENRequestBody. encapsulates NGENRequest's `model` field.
aaraney Jan 20, 2023
014dc30
add class var type hints to NGENRequestResponse
aaraney Jan 20, 2023
d511310
specify ExternalRequest aliases in Config class.
aaraney Jan 20, 2023
119bb31
remove, now, unncessary ExternalRequest initializer
aaraney Jan 20, 2023
8a7ac31
add class that encapsulates a ModelExecRequestResponseBody
aaraney Jan 20, 2023
dc350b6
refactor ModelExecRequestResponse
aaraney Jan 20, 2023
38db387
add DataTransmitUUID class
aaraney Jan 20, 2023
16e1926
refactor DataTransmitMessage to use DataTransmitUUID
aaraney Jan 20, 2023
a5cdd69
refactor DataTransmitResponse
aaraney Jan 20, 2023
687e317
refactor DatasetManagementResponse. speperated its body into own data…
aaraney Jan 20, 2023
170a66f
add type hints to MaaSDatasetManagementResponse class vars
aaraney Jan 20, 2023
dfa3776
refactor PartitionResponse
aaraney Jan 21, 2023
5a44b34
fix faulty logic in SchedulerRequestMessage dict method
aaraney Jan 23, 2023
604c2b3
fix typo in DatasetQuery field name
aaraney Jan 23, 2023
0c41a64
update dataset query tests
aaraney Jan 23, 2023
1996266
factory init model exec requests handles nwm and ngen cases. this sho…
aaraney Jan 23, 2023
c8ad6ee
SchedulerRequestMessage factory inits ModelExecRequest correctly
aaraney Jan 23, 2023
ec2691e
allow SchedulerRequestResponse instances to be treated as dictionary
aaraney Jan 23, 2023
2680683
allow SchedulerRequestResponseBody to be treated like it is a dictionary
aaraney Jan 23, 2023
6c77e74
add missing required fields in decorated interface tests
aaraney Jan 23, 2023
aa5c579
add missing required fields to websocket interface tests
aaraney Jan 23, 2023
c4d1af4
use relative imports in tests. this avoids, potentially unexpected be…
aaraney Jan 24, 2023
6809ecf
change assert eq types in ngen request response tests
aaraney Jan 24, 2023
c943006
change assert eq types in nwn request response tests
aaraney Jan 24, 2023
2ef41ab
add classvar type hints to FieldedMessage
aaraney Jan 24, 2023
f994fdc
refactor Session dict override with Config field_serializers
aaraney Jan 24, 2023
e0ca55c
refactor FailedSessionInitInfo dict override with Config field_serial…
aaraney Jan 24, 2023
50382e4
refactor DataTransmitUUID dict override with Config field_serializers
aaraney Jan 24, 2023
1cbb85f
remove unused imports
aaraney Jan 24, 2023
3911c4f
refactor UpdateMessage dict override with Config field_serializers
aaraney Jan 24, 2023
3f35a60
bump minimum required version of dmod.core to 0.4.2
aaraney Jan 26, 2023
dd6b36a
dekabob (i.e. this-that) SchedulerRequestMessage allocation_paradigm …
aaraney Feb 1, 2023
15378db
fix property name that does not exist
aaraney Feb 3, 2023
5208dc8
fix instance where DataRequirement's init params were not specified a…
aaraney Feb 3, 2023
2e4d9c8
fix NWMRequest get_model_name classmethod. see note
aaraney Feb 6, 2023
8e3cd62
Response subtypes are now SerializableDict subtypes. This is more fle…
aaraney Feb 10, 2023
245c34d
metadata_follows _defaults_ to False. This was pre-pydantic behavior
aaraney Feb 10, 2023
8c56cae
remove, now, unnecessary PartitionResponseBody magic method overrides…
aaraney Feb 10, 2023
4518788
fix, could be none, error in SchedulerRequestResponse
aaraney Feb 10, 2023
ec3658a
add session_secret validator. handles None as input. pre-pydantic han…
aaraney Feb 10, 2023
da4d0c5
fix UpdateMessage object_type validator. _guards_ str types
aaraney Feb 10, 2023
02890f7
add pydantic as dep to dmod.scheduler
aaraney Jan 27, 2023
c752133
refactor enum variants to use dmod.core.enum.PydanticEnum
aaraney Jan 27, 2023
5cb52d7
refactor JobStatus
aaraney Jan 31, 2023
a5412d0
resource enum's now are based on PydanticEnum
aaraney Jan 31, 2023
69bc26b
refactor AbstractProcessingAssetPool to use pydantic
aaraney Feb 1, 2023
2715287
refactor SingleHostProcessingAssetPool
aaraney Feb 1, 2023
473114f
refactor Resource
aaraney Feb 1, 2023
45f6032
refactor ResourceAllocation
aaraney Feb 1, 2023
840c36b
resource fields accepted case insensitively
aaraney Feb 1, 2023
160af12
add accidentally remove total_cpu_count prop
aaraney Feb 1, 2023
521abd8
validate that cpu and memory total cannot be larger than cpu and memory
aaraney Feb 1, 2023
8a7bccc
refactor RsaKeyPair
aaraney Feb 2, 2023
4071df0
add RsaKeyPair unit tests
aaraney Feb 2, 2023
6327246
restore unit test assertion condition
aaraney Feb 3, 2023
755b254
validate RsaKeyPair property assignment. Required to retain previous …
aaraney Feb 3, 2023
dbce96a
testing: verify modifying RsaKeyPair directory through a setter behav…
aaraney Feb 3, 2023
005fbe4
add class var type hint to JobStatus
aaraney Feb 6, 2023
c5407cd
JobStatus `name` property now uses fstring
aaraney Feb 6, 2023
c57f4be
ignore type hints that static checker cannot catch
aaraney Feb 6, 2023
15c16bd
refactor Job to use pydantic
aaraney Feb 6, 2023
615a6aa
deprecate Job property setters. set through setter method instead.
aaraney Feb 6, 2023
263b9ca
import validator and cache
aaraney Feb 6, 2023
b90ee6c
refactor JobImp to use pydantic
aaraney Feb 6, 2023
2043459
refactor RequestedJob to use pydantic
aaraney Feb 6, 2023
ecf7ef4
fix rsa test bug. was not cleaning up
aaraney Feb 6, 2023
5778284
add type hints to tests
aaraney Feb 6, 2023
2fa2621
fix missing field in test setup
aaraney Feb 6, 2023
c197181
cannot cache _setter_methods. if this becomes bottleneck, we can cach…
aaraney Feb 6, 2023
57267cd
change Job's allocations field to optional tuple from optional list
aaraney Feb 6, 2023
ec72deb
Resource unit tests
aaraney Feb 6, 2023
6d5c4ca
add JobImpl setter tests
aaraney Feb 7, 2023
2f811b5
format modeldata's setup.py
aaraney Jan 26, 2023
ce92d01
add missing dep, gitpython to modeldata's setup.py
aaraney Jan 26, 2023
1669805
add pydantic dep
aaraney Jan 26, 2023
bbb2cb5
refactor SubsetDefinition
aaraney Jan 26, 2023
0919041
refactor Partition
aaraney Jan 26, 2023
a4be802
refactor PartitionConfig
aaraney Jan 26, 2023
58e55d6
add kwargs argument to SubsetDefinition initilizer
aaraney Jan 26, 2023
fbcb8ed
refactor HydrofabricSubset
aaraney Jan 26, 2023
9a28d4e
refactor SimpleHydrofabricSubset
aaraney Jan 26, 2023
681e6d4
move HydrofabricSubset initializer
aaraney Jan 26, 2023
0ee0140
add private attr class level declarations to SimpleHydrofabricSubset
aaraney Jan 26, 2023
2ec8307
update Tuple type hints to be variadic
aaraney Jan 26, 2023
ad241c1
fix reference to non-existent field
aaraney Feb 7, 2023
300ce9b
add missing PrivateAttr field to Partition
aaraney Feb 7, 2023
58f0341
typo in Partition's fields alias map
aaraney Feb 7, 2023
1362e2a
serialize Partition fields as list and fix it's intializer fn to allo…
aaraney Feb 7, 2023
cc57f91
add Partition unit tests
aaraney Feb 7, 2023
ed2556e
fix invalid reference
aaraney Feb 7, 2023
9815c3a
add PartionConfig unittests. format with black
aaraney Feb 7, 2023
f6379ee
add PartionConfig field_serializer
aaraney Feb 7, 2023
0a93138
override PartitionConfig dict method.
aaraney Feb 7, 2023
ff06874
replace private reference to Dataset field name
aaraney Feb 10, 2023
060e076
replace private reference to Dataset field name in test
aaraney Feb 10, 2023
4e20d27
no longer use a double wrapped classmethod property. is not supported…
aaraney Feb 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 38 additions & 89 deletions python/lib/communication/dmod/communication/data_transmit_message.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,23 @@
from dmod.core.serializable import Serializable
from pydantic import Extra
from dmod.core.serializable_dict import SerializableDict
from .message import AbstractInitRequest, MessageEventType, Response
from typing import Dict, Optional, Union
from numbers import Number
from pydantic import Field
from typing import ClassVar, Type, Union
from uuid import UUID


class DataTransmitMessage(AbstractInitRequest):
class DataTransmitUUID(Serializable):
series_uuid: UUID = Field(description="A unique id for the collective series of transmission message this instance is a part of.")
"""
The expectation is that a larger amount of data will be broken up into multiple messages in a series.
"""

class Config:
field_serializers = {"series_uuid": lambda s: str(s)}


class DataTransmitMessage(DataTransmitUUID, AbstractInitRequest):
"""
Specialized message type for transmitting data.

Expand All @@ -18,64 +31,14 @@ class DataTransmitMessage(AbstractInitRequest):
::class:`str` object. However, instances can be initialized using either ::class:`str` or ::class:`bytes` data.
"""

_KEY_SERIES_UUID = 'series_uuid'
event_type: ClassVar[MessageEventType] = MessageEventType.DATA_TRANSMISSION

event_type: MessageEventType = MessageEventType.DATA_TRANSMISSION
data: str = Field(description="The data carried by this message, in decoded string form.")
is_last: bool = Field(False, description="Whether this is the last data transmission message in this series.")

@classmethod
def factory_init_from_deserialized_json(cls, json_obj: dict) -> Optional['DataTransmitMessage']:
try:
return cls(data=json_obj['data'], series_uuid=UUID(json_obj[cls._KEY_SERIES_UUID]),
is_last=json_obj['is_last'])
except Exception as e:
return None

def __init__(self, data: Union[str, bytes], series_uuid: UUID, is_last: bool = False, *args, **kwargs):
super(DataTransmitMessage, self).__init__(*args, **kwargs)
self._data: str = data if isinstance(data, str) else data.decode()
self._series_uuid = series_uuid
self._is_last: bool = is_last

@property
def data(self) -> str:
"""
The data carried by this message, in decoded string form.

Returns
-------
str
The data carried by this message, in decoded string form.
"""
return self._data

@property
def is_last(self) -> bool:
"""
Whether this is the last data transmission message in this series.

Returns
-------
bool
Whether this is the last data transmission message in this series.
"""
return self._is_last

@property
def series_uuid(self) -> UUID:
"""
A unique id for the collective series of transmission message this instance is a part of.

The expectation is that a larger amount of data will be broken up into multiple messages in a series.

Returns
-------
UUID
A unique id for the collective series of transmission message this instance is a part of.
"""
return self._series_uuid

def to_dict(self) -> Dict[str, Union[str, Number, dict, list]]:
return {'data': self.data, self._KEY_SERIES_UUID: str(self.series_uuid), 'is_last': self.is_last}
class DataTransmitResponseBody(SerializableDict, DataTransmitUUID):
...


class DataTransmitResponse(Response):
Expand All @@ -86,38 +49,24 @@ class DataTransmitResponse(Response):
series of which it is a part.
"""

response_to_type = DataTransmitMessage

_KEY_SERIES_UUID = response_to_type._KEY_SERIES_UUID

@classmethod
def factory_init_from_deserialized_json(cls, json_obj: dict) -> 'DataTransmitResponse':
"""
Factory create a new instance of this type based on a JSON object dictionary deserialized from received JSON.

Parameters
----------
json_obj

Returns
-------
response_obj : Response
A new object of this type instantiated from the deserialize JSON object dictionary, or none if the provided
parameter could not be used to instantiated a new object.
"""
try:
return cls(success=json_obj['success'], reason=json_obj['reason'], message=json_obj['message'],
series_uuid=json_obj['data'][cls._KEY_SERIES_UUID], data=json_obj['data'])
except Exception as e:
return None

def __init__(self, series_uuid: Union[str, UUID], *args, **kwargs):
if 'data' not in kwargs:
kwargs['data'] = dict()
kwargs['data'][self._KEY_SERIES_UUID] = str(series_uuid)
super(DataTransmitResponse, self).__init__(*args, **kwargs)
response_to_type: ClassVar[Type[AbstractInitRequest]] = DataTransmitMessage

data: DataTransmitResponseBody

# `series_uuid` required in prior version of code
def __init__(self, series_uuid: Union[str, UUID] = None, **kwargs):
# assume no need for backwards compatibility
if series_uuid is None:
super().__init__(**kwargs)
return

if "data" not in kwargs:
kwargs["data"] = dict()

kwargs["data"]["series_uuid"] = series_uuid
super().__init__(**kwargs)

@property
def series_uuid(self) -> UUID:
return UUID(self.data[self._KEY_SERIES_UUID])
return self.data.series_uuid

Loading