-
Notifications
You must be signed in to change notification settings - Fork 13
Add --ignore-empty-record-stream flag
#177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,6 +13,7 @@ | |
|
|
||
| import flow.record.adapter | ||
| from flow.record import RecordWriter, iter_timestamped_records, record_stream | ||
| from flow.record.exceptions import RecordAdapterNotFound | ||
| from flow.record.selector import make_selector | ||
| from flow.record.stream import RecordFieldRewriter | ||
| from flow.record.utils import LOGGING_TRACE_LEVEL, catch_sigpipe | ||
|
|
@@ -143,6 +144,11 @@ def main(argv: list[str] | None = None) -> int: | |
| action="store_true", | ||
| help="Show count of processed records", | ||
| ) | ||
| output.add_argument( | ||
| "--ignore-empty-record-stream", | ||
| action="store_true", | ||
| help="Exit successfully if no records can be read", | ||
| ) | ||
|
|
||
| advanced = parser.add_argument_group("advanced") | ||
| advanced.add_argument( | ||
|
|
@@ -315,13 +321,17 @@ def main(argv: list[str] | None = None) -> int: | |
| record_writer.write(rec) | ||
|
|
||
| except Exception as e: | ||
| print_error(e) | ||
| if args.ignore_empty_record_stream and count == 0 and isinstance(e, RecordAdapterNotFound): | ||
| log.warning("No records were read from the input stream") | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If someone would use a selector that yields no records this warning would not be technically correct. As records have ben read from input stream but it yielded no records due to the selector. I was thinking about adding extra return data to
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. processed records are now tracked via an AppContext, see #184 |
||
| ret = 0 | ||
| else: | ||
| print_error(e) | ||
|
|
||
| # Prevent throwing an exception twice when deconstructing the record writer. | ||
| if hasattr(record_writer, "exception") and record_writer.exception is e: | ||
| record_writer.exception = None | ||
| # Prevent throwing an exception twice when deconstructing the record writer. | ||
| if hasattr(record_writer, "exception") and record_writer.exception is e: | ||
| record_writer.exception = None | ||
|
|
||
| ret = 1 | ||
| ret = 1 | ||
|
|
||
| finally: | ||
| if record_writer: | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why also
RecordAdapterNotFoundin the condition?