upgraded dependencies, upgraded s3mock lib, extended redshift support#78
upgraded dependencies, upgraded s3mock lib, extended redshift support#78AndrewChubatiuk wants to merge 5 commits intoopt-tech:masterfrom
Conversation
There was a problem hiding this comment.
Change "jp.ne.opt.redshiftfake.postgres.Driver" to "jp.ne.opt.redshiftfake.postgresql.Driver"
updated class name
| AwsBasicCredentials.create(accessKeyId, secretAccessKey) | ||
| ) | ||
| ) | ||
| case Credentials.WithRole(roleName) => |
There was a problem hiding this comment.
Haven't been able to get this to run. I keep getting:
software.amazon.awssdk.services.sts.model.StsException: The security token included in the request is invalid. (Service: Sts, Status Code: 403, Request ID: fb539513-99dc-4506-a908-bffb45e766f1)
anytime I supply iam_role '<role_arn>'
There was a problem hiding this comment.
Could you share an example of query you're using?
There was a problem hiding this comment.
This query works perfectly(using aws_keys):
UNLOAD ('select * from pg_catalog.pg_tables')
TO 'http://127.0.0.1:9444/test_bucket_1/tt/test.csv'
credentials 'aws_access_key_id=AKIAXXXX;aws_secret_access_key=YYYYYY'
But this doesn't work(iam_role):
UNLOAD ('select * from pg_catalog.pg_tables')
TO 'http://127.0.0.1:9444/test_bucket_1/tt/test.csv'
iam_role 'arn:aws:iam::123456789012:role/some-role'
The error I get is: software.amazon.awssdk.services.sts.model.StsException: The security token included in the request is invalid. (Service: Sts, Status Code: 403, Request ID: 53594081-d02c-40e4-a42b-ca8a8e6c0f1a).
For context, I'm using this driver through python (using jaydebeapi and jpype). But the error seems to come from the scala code.
There was a problem hiding this comment.
are you running this in pod, on EC2 instance or do you provide credentials either with env variables or ~/.aws/credentials?
There was a problem hiding this comment.
At the moment, I'm running this locally. I'll be running this within aws codebuild, which is essentially a docker container.
There was a problem hiding this comment.
to assume role you still need to run it with IAM user creds (from env variables or config) or with IAM role (running inside pod or EC2 instance)
There was a problem hiding this comment.
I did that. In my ~/.aws/credentials file, I have
[default]
aws_access_key_id = foobar_key
aws_secret_access_key = secret
Tried it with an actual set of keys too.
|
@ocadaruma are you still maintaining this repo? |
2 participants
Thank you for this driver
In this PR: