4290 prefect pipeline#266
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #266 +/- ##
==========================================
- Coverage 91.96% 90.39% -1.57%
==========================================
Files 39 41 +2
Lines 4082 4176 +94
Branches 590 485 -105
==========================================
+ Hits 3754 3775 +21
- Misses 265 334 +69
- Partials 63 67 +4
☔ View full report in Codecov by Sentry. |
digorgonzola
force-pushed
the
4290-prefect-pipeline
branch
from
e99a1f1 to
f666c6c
Compare
…re into 4290-prefect-pipeline
mhidas
left a comment
mhidas
left a comment
There was a problem hiding this comment.
Just adding my 2c's worth while I look at this.
| @@ -0,0 +1,64 @@ | |||
| from urllib.parse import urlparse | |||
| import boto3 | |||
There was a problem hiding this comment.
Some functionality for handling objects on S3 is already implemented as an S3StorageBroker class that suports query, download, upload and delete operations. Could that be reused instead of duplicating the functionality here?
| """ Override HandlerBase""" | ||
|
|
||
| try: | ||
| self._file_checksum = self.etag |
There was a problem hiding this comment.
I can't see where this etag attribute is set?
3 participants
These modify aodncore to allow for running of pipelines with Prefect and S3 buckets. The changes should not affect existing pipeline operations.