Open
Conversation
Branch 2.0
Added logging factor to groupByKeyAndTake
make provided s3 dependencies
classpath fix revert
Added Future.withTimeout
Moved async http to core
retry on cluster health check
Set host with correct port and ssl support
Allow a granular fallback to external configuration
…nto telemetry_cache
Move telemetry cache to core
Make retry configurable
…te a new cluster.
Feat/destroy function
- update the current branch with the latest commits in master necessary for fleet work properly with the new destroy function
- it is a module inside core/tools which is called with subprocess of python2 (cluster.py) due to the script (revoke_sg_rules.py) be written in python3.
Co-authored-by: Iury Krieger <iury.krieger@linx.com.br>
- add a boolean function to return true if a given cidr exists in a given security group id. - extends the function _get_security_group() to return an specific security group through the param `sg_name`. - add a logic to verifie if the rules were deleted before end the script.
- it is due to requests that are made with boto3
Feat/fleet-options
feat: add a function to delete Flintrock SG rules in dev environment
…code style - now the function is to destroy by fleet id
- it is called inside the util.py (python2) through subprocess.Popen(). - this script returns a message for the fleet deleted and a list containing the instance ids.
Fix/destroy function
hotfix: fix the output when there is no json file with fleet id
…unch-user [DIREEN-2034] Change installation and launch user from root to ec2-user
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problema/Motivação
Configuração do Spark para que o Prometheus colete maior número de métricas.
O que foi feito
Modificada a linha de comando que executa o Job (spark-submit) removendo o parâmetro javaagent, pois a partir de agora a métricas serão expostas via httpserver
Como testar
-Atualizar o submódulo CORE do mail-ignition para a branch do PR (feat/prometheus-script)
-Através do run test-cluster
make image-bash
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
/app/mail-devops/deploy/roles/mail-ignition/files/scripts/job_runner.py run test-cluster /tmp