Feature/Parameterize Bedrock model ID in CloudFormation templates#238
Feature/Parameterize Bedrock model ID in CloudFormation templates#238oussamahansal wants to merge 2 commits intomainfrom
Conversation
| ], | ||
| "Resource": [ | ||
| { | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.anthropic.*" |
There was a problem hiding this comment.
This doesn't work for the workshop, because the workshop ModelId parameter can also accept a profile, which already includes the prefix. We need this profile to carry all the way through, because a) Many models will now only work when invoked with a profile - sonnet-4-6, for example, b) the workshop's cached responses are keyed for the specific modelId/profile that is suplpied.
There was a problem hiding this comment.
I believe removing the *. would work, right?
iansrobinson
left a comment
There was a problem hiding this comment.
My preference would be for ModelId to accept a model ID or profile. Many models, sonnet-4-6 for example, have to be invoked via a profile. Therefore, I suspect the configuration of environment variables will fail unless supplied with a profile.
In the end, I liked the use of *.anthropic.* in the IAM profiles. It's not as locked to specific models as we'd like, but it works in the face of the inconsistencies in Bedrock.
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| "Default": "https://github.com/awslabs/graphrag-toolkit/releases/latest/download/lexical-graph-examples-latest.zip" | ||
| }, | ||
| "BedrockModelId": { | ||
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", |
There was a problem hiding this comment.
| "Description": "Bedrock model ID used for extraction, response, and evaluation LLMs", | |
| "Description": "Bedrock model ID or inference profile used for extraction, response, and evaluation LLMs", |
| }, | ||
| { | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.anthropic.*" | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" |
There was a problem hiding this comment.
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" | |
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*${BedrockModelId}" |
| }, | ||
| { | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.anthropic.*" | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" |
There was a problem hiding this comment.
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" | |
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*${BedrockModelId}" |
| ], | ||
| "Resource": [ | ||
| { | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.anthropic.*" |
There was a problem hiding this comment.
I believe removing the *. would work, right?
| }, | ||
| { | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.anthropic.*" | ||
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" |
There was a problem hiding this comment.
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/*.${BedrockModelId}" | |
| "Fn::Sub": "arn:${AWS::Partition}:bedrock:*:${AWS::AccountId}:inference-profile/${BedrockModelId}" |
There was a problem hiding this comment.
I don't think we need the wildcard for profiles, since the profile passed in would contain the prefix.
There was a problem hiding this comment.
We might need to go with something like *${BedrockModelId}*?
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.