Open
Conversation
iseessel
commented
Oct 11, 2021
| @@ -7,9 +7,9 @@ config: | |||
| DATA: | |||
iseessel
commented
Oct 11, 2021
vissl/meters/precision_list_meter.py
Outdated
| ) | ||
| val = 100.0 * round(float(val_dict[ind]["val"][top_k_str]), 6) | ||
| val = 100.0 * round(float(val_dict[ind]["val"][f"top_{k}"]), 6) | ||
| # val = 1000 |
iseessel
commented
Oct 11, 2021
vissl/meters/accuracy_list_meter.py
Outdated
| self._meter_names[ind] if (len(self._meter_names) > 0) else ind | ||
| ) | ||
| val = 100.0 * round(float(val_dict[ind]["val"][top_k_str]), 6) | ||
|
|
iseessel
commented
Oct 11, 2021
vissl/trainer/trainer_main.py
Outdated
| for meter in task.meters: | ||
| meter.sync_state() | ||
| logging.info("Meters synced") | ||
| # for meter in task.meters: |
iseessel
commented
Oct 11, 2021
| """ | ||
| meter_names = self.config["METERS"].get("names", "") | ||
| meter_names = self.config["METERS"].get("names", []) | ||
|
|
iseessel
force-pushed
the
zlapp-precision-recall
branch
from
91d4d48 to
78ae59f
Compare
iseessel
commented
Oct 11, 2021
| constructed and each output has a meter associated. | ||
|
|
||
| Example: | ||
| target = [0 0 0 1 1] # Correct classes are 0, 3 |
iseessel
force-pushed
the
zlapp-precision-recall
branch
from
d3cde11 to
9d49c40
Compare
Summary: Pull Request resolved: facebookresearch#447 Reviewed By: QuentinDuval Differential Revision: D31541855 Pulled By: iseessel fbshipit-source-id: bb8648e4c5d010e2e21274dbdd28207de9a1ca8c
Summary: Pull Request resolved: facebookresearch#450 Reviewed By: prigoyal Differential Revision: D31575588 Pulled By: iseessel fbshipit-source-id: 470c2a7a8208a7cc22ffc8ef5c73bb5977a947e6
Summary: Pull Request resolved: facebookresearch#451 Reviewed By: QuentinDuval Differential Revision: D31588441 Pulled By: prigoyal fbshipit-source-id: 67fbff136caff436366ece8c3fbb77287121a9f2
Summary: Pull Request resolved: facebookresearch#449 Upstream hydra now is at "1.2.0dev1" which the last component throws an exception with the current parsing scheme. Here we restrict to the numeric portion which should suffice. Another option may be to depend on packaging.version.parse(). Reviewed By: QuentinDuval Differential Revision: D31557803 fbshipit-source-id: b6d84e92088520f6b6b26a06ac28d0b0730f8045
…rch#191) Summary: Pull Request resolved: fairinternal/ssl_scaling#191 Reviewed By: iseessel Differential Revision: D31662929 Pulled By: QuentinDuval fbshipit-source-id: 7945d717d34f6e270d710372d2371e7f2ea0894e
Summary: We assert python version > 3.7, but only catch the ImportError. Pull Request resolved: facebookresearch#455 Reviewed By: QuentinDuval Differential Revision: D31729200 Pulled By: iseessel fbshipit-source-id: f767cf1c57f7d646eda13ee32035f738aee511a2
Summary: This pull request is related to issue facebookresearch#411 Pull Request resolved: facebookresearch#416 Reviewed By: prigoyal, QuentinDuval Differential Revision: D31608966 Pulled By: iseessel fbshipit-source-id: 221c4d4ee9c08484917b564a885036d2546b698f
…search#189) Summary: Allow to enable or disable activation checkpointing for each block of the RegNet FSDP, instead of relying on a single flag to enable it for all block or none. - [x] added integration tests - [x] all tests are green - [x] ready for merge Pull Request resolved: fairinternal/ssl_scaling#189 Reviewed By: prigoyal Differential Revision: D31616502 Pulled By: QuentinDuval fbshipit-source-id: 5e234f4cce1cd9e50ef521e59190831549275fd3
…facebookresearch#456) Summary: Pull Request resolved: facebookresearch#456 Reviewed By: bottler, prigoyal Differential Revision: D31769847 Pulled By: iseessel fbshipit-source-id: e97b7ef149dade450aa1cf26abf19e3f49ef5f5b
Summary: This fixes the VISSL docker containers. - Bumps pytorch to 1.9.1 - Builds apex from pip/conda packages instead of from source (much, much faster), - Build vissl from a specific branch as opposed to main TODO: Build classy commit. Pull Request resolved: facebookresearch#458 Reviewed By: prigoyal Differential Revision: D31802202 Pulled By: iseessel fbshipit-source-id: c88e0ce77d96219dec70313f25cbf79f03933608
…#408) Summary: Pull Request resolved: facebookresearch#408 ## Main Goal Of PR: Create the ability to mask model outputs to negative infinity based on the classes present. I went back and forth on approach a few times, so lmk what you think -- imo all the possible approaches had drawbacks. Guiding principles: 1. Make the masking as isolated as possible from `train_task.py` and `standard_train_step.py` 2. Automatically create mask based on labels that are present, instead of supplying predetermined mask. Implementation details. 1. Create hook to alter model_output based on labels present. Load labels from within hook, as labels are only loaded when an image is fetched (these datasets are 3. Move Nan check to own hook. ## Secondary Changes 1. Add all currently available Imagenet OOD models (adapt configs). Note: configs are not ready to be run -- Priya is working on In1k hypertuning, but I want to keep them checked in for now. 2. Add inference time resize transforms. 3. Fix some small bugs in extra script utils. Reviewed By: prigoyal Differential Revision: D30521019 fbshipit-source-id: e83e5ee8190141405420a8bd50eb6becaf94ea71
Summary: Pull Request resolved: fairinternal/ssl_scaling#193 Reviewed By: iseessel, prigoyal Differential Revision: D31770157 Pulled By: QuentinDuval fbshipit-source-id: c99a22598c33186afefe03ff238c42652149206d
iseessel
force-pushed
the
zlapp-precision-recall
branch
from
9d49c40 to
64c7dcf
Compare
Summary: Pull Request resolved: facebookresearch#448 Test Plan: 1. Unit Tests 2. Objectnet eval with multiple meters. 3. Objectnet eval with multiple meters + tensorboard. 4. Checking logs are not affected. f302516569 f302378523 ``` INFO 2021-10-11 21:17:38,781 ssl_dataset.py: 224: Loading labels: manifold://ssl_framework/tree/datasets/objectnet/test_labels.npy INFO 2021-10-11 21:17:38,968 ssl_dataset.py: 182: Could not mmap without PathManager. Trying without mmap INFO 2021-10-11 21:59:32,429 trainer_main.py: 204: Meters synced INFO 2021-10-11 21:59:32,430 log_hooks.py: 525: Average test batch time (ms) for 581 batches: 4332 INFO 2021-10-11 21:59:32,430 log_hooks.py: 457: Rank: 0, name: test_accuracy_list_meter, value: {'top_1': {0: 34.3652}, 'top_5': {0: 56.9344}, 'top_10' : {0: 66.52850000000001}, 'top_50': {0: 90.24980000000001}, 'top_100': {0: 98.3687}} INFO 2021-10-11 21:59:32,430 log_hooks.py: 457: Rank: 0, name: test_precision_at_k_list_meter, value: {'top_1': {0: 34.3652}, 'top_5': {0: 11.7659}, 'top_10': {0: 6.9506}, 'top_50': {0: 1.9438}, 'top_100': {0: 1.0727}} INFO 2021-10-11 21:59:32,430 io.py: 63: Saving data to file: manifold://ssl_framework/tree/iseessel/checkpoints/f302378523 /eval_in1k_byol_reference_true_eval_resnet_8gpu_test_only.yaml_11_10_21/metrics.json INFO 2021-10-11 21:59:33,119 io.py: 89: Saved data to file: manifold://ssl_framework/tree/iseessel/checkpoints/f302378523 /eval_in1k_byol_reference_true_eval_resnet_8gpu_test_only.yaml_11_10_21/metrics.json INFO 2021-10-11 21:59:34,340 train.py: 131: All Done! INFO 2021-10-11 21:59:34,340 logger.py: 73: Shutting down loggers... INFO 2021-10-11 21:59:34,341 distributed_launcher.py: 168: All Done! INFO 2021-10-11 21:59:34,341 logger.py: 73: Shutting down loggers... ``` ``` INFO 2021-10-15 15:20:54,205 log_hooks.py: 566: Average test batch time (ms) for 2 batches: 1005 INFO 2021-10-15 15:20:54,205 log_hooks.py: 498: Rank: 0, name: test_accuracy_list_meter, value: {'top_1': {'res5': 0.0}, 'top_5': {'res5': 0.0}} INFO 2021-10-15 15:20:54,206 log_hooks.py: 498: Rank: 0, name: test_precision_at_k_list_meter, value: {'top_1': {'res5': 0.0}} INFO 2021-10-15 15:20:54,206 io.py: 63: Saving data to file: ./metrics.json INFO 2021-10-15 15:20:54,206 io.py: 89: Saved data to file: ./metrics.json ``` Reviewed By: prigoyal Differential Revision: D31548914 Pulled By: iseessel fbshipit-source-id: 1bcc0cbed219ea250dc1aeaaf9df208f6acbe92d
iseessel
force-pushed
the
zlapp-precision-recall
branch
from
64c7dcf to
d817c7e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.