Unchecked access to nonrequired items in AWS responses may result in runtime errors

[maouw]

A large number of expressions in the codebase make assumptions about the keys present in AWS responses and do not check if those keys exist before accessing them. This can lead to runtime exceptions if the key does not exist in the response.

For example, in the function get_s3_params() in aws/base_classes.py, it is assumed that PolicyName and Arn are present in the response obtained from IAMClient.get_paginator("list_policies"). These keys are not required in the response, so the assignment may result in a runtime exception. Refer to the AWS API documentation for IAM's Policy response, or to the boto3 stub documentation:

# PolicyTypeDef definition

class PolicyTypeDef(TypedDict):
    PolicyName: NotRequired[str],
    PolicyId: NotRequired[str],
    Arn: NotRequired[str],
    Path: NotRequired[str],
    DefaultVersionId: NotRequired[str],
    AttachmentCount: NotRequired[int],
    PermissionsBoundaryUsageCount: NotRequired[int],
    IsAttachable: NotRequired[bool],
    Description: NotRequired[str],
    CreateDate: NotRequired[datetime],
    UpdateDate: NotRequired[datetime],
    Tags: NotRequired[List[TagTypeDef]],  # (1)