GSLA - update handler to handle non gzipped files

[lbesnard]

Add logic to transform a received NetCDF file into a .nc.gz

historically, files were always sent as *.nc.gz. But as of April 2021, files might be pushed as *.nc
To be consistent with the existing dataset, and gogoduck, we transform this .nc into a .nz.gz

[codecov]

Codecov Report

Merging #264 (c2e1e13) into master (9d37dfe) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #264      +/-   ##
==========================================
+ Coverage   87.98%   88.04%   +0.06%     
==========================================
  Files          50       50              
  Lines        3221     3229       +8     
  Branches      536      537       +1     
==========================================
+ Hits         2834     2843       +9     
  Misses        243      243              
+ Partials      144      143       -1     

Impacted Files	Coverage Δ
aodndata/gsla/handler.py	94.00% <100.00%> (+1.60%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d37dfe...c2e1e13. Read the comment docs.

[lbesnard]

linked to https://github.com/aodn/chef-private/pull/3799

[mhidas]

I just made a few suggestions to simplify handling of input file and file collection using recently added aodncore functionality.

[mhidas]

Since you already know it's a single netCDF file being handled, you can do this more simply...

Suggested change

	netcdf_collection = self.file_collection.filter_by_attribute_id('file_type', FileType.NETCDF)
	netcdf_file = netcdf_collection[0]
	netcdf_file.publish_type = PipelineFilePublishType.NO_ACTION
	self.file_collection.set_publish_types(PipelineFilePublishType.NO_ACTION)

[mhidas]

There's a shortcut available here too... (file_basename property)

Suggested change

	gzip_path = os.path.join(self.temp_dir, os.path.basename(self.input_file + '.gz'))
	gzip_path = os.path.join(self.temp_dir, self.file_basename + '.gz')

[mhidas]

And again... (see aodn/python-aodncore#209)

Suggested change

	netcdf_file_gz = PipelineFile(gzip_path, file_update_callback=self._file_update_callback)
	netcdf_file_gz.publish_type = PipelineFilePublishType.HARVEST_UPLOAD
	self.file_collection.add(netcdf_file_gz)
	self.add_to_collection(gzip_path, publish_type=PipelineFilePublishType.HARVEST_UPLOAD)

[lbesnard]

thanks @mhidas I followed your suggestions

[mhidas]

👍 I haven't really looked at the unittests, but they all seem to be passing, so should be ok. Is this ready to merge then?

[lbesnard]

it is ! thanks. oh yeah I looked at them quite a lot!