Fix the sized rotating logger size bug for the blf writer#1360
Fix the sized rotating logger size bug for the blf writer#1360j-c-cook wants to merge 9 commits intohardbyte:developfrom
Conversation
Codecov Report
@@ Coverage Diff @@
## develop #1360 +/- ##
========================================
Coverage 66.06% 66.06%
========================================
Files 86 86
Lines 8973 8976 +3
========================================
+ Hits 5928 5930 +2
- Misses 3045 3046 +1 |
|
|
||
| options = {"append": results.append} | ||
| if results.file_size: | ||
| options["max_container_size"] = results.file_size # bytes |
There was a problem hiding this comment.
I'm now passing in max_container_size as an option.
| append: bool = False, | ||
| channel: int = 1, | ||
| compression_level: int = -1, | ||
| **options: Any, |
There was a problem hiding this comment.
Added **options to BLFWriter API.
There was a problem hiding this comment.
Assuming that compression_level is specific to the BLFWriter, perhaps it should be moved into the options dict as well?
There was a problem hiding this comment.
Arguments specific to a Writer could be made more accessible if able to be set by **options . Currently, the only way to set compression_level would be to bypass the Logger (or RollingLogger) object and call the BLFWriter object itself.
When going about these changes, I'll want to make sure that the API is either unaffected by this improvement, or that a deprecation warning is added (similar to what I did here for SizedRotatingLogger).
There was a problem hiding this comment.
It is specific to the BlfWriter (and soon Mf4Writer) but i think it's more readable if it is not hidden away in **options
There was a problem hiding this comment.
I have confirmed that compression_level can be modified by passing via the argument in **kwargs. @zariiii9003's CLI arguments https://github.com/zariiii9003/python-can/tree/extra_args2 will provide the ability to modify variables such as compression_level. For example:
python -m can.logger -i socketcan -c vcan0 -b 250000 -f file.blf -s 100000 --compression-level=1 --max-container-size=200000
Once #1366 is merged, I will close this PR. The combination of #1367 and #1366 will resolve everything I had intended from this PR.
| #: Max log container size of uncompressed data | ||
| if "max_container_size" in options: | ||
| self.max_container_size = options["max_container_size"] | ||
| else: | ||
| self.max_container_size = 128 * 1024 # bytes |
There was a problem hiding this comment.
There is now an adjustable max buffer size.
|
@zariiii9003 Ready for review. |
| help="Maximum file size in bytes (or for the case of blf, maximum " | ||
| "buffer size before compression and flush to file). Rotate log " | ||
| "file when size threshold is reached.", |
| append: bool = False, | ||
| channel: int = 1, | ||
| compression_level: int = -1, | ||
| **options: Any, |
There was a problem hiding this comment.
Assuming that compression_level is specific to the BLFWriter, perhaps it should be moved into the options dict as well?
|
@hardbyte i see you approved this but personally i think bringing implementation details of the BlfWriter ( What do you think about adding a |
I see two options here:
-soption for the blf rolling logger will not define the file size to be written, but rather the buffer size prior to compressionI plan to move forward shortly with a commit that will implement the solution (1).
closes #1359