Skip to content

Refactoring smart_open usage as an optional dependency#51

Merged
dixonwhitmire merged 3 commits into
mainfrom
optional-smart_open
Oct 3, 2022
Merged

Refactoring smart_open usage as an optional dependency#51
dixonwhitmire merged 3 commits into
mainfrom
optional-smart_open

Conversation

@dixonwhitmire
Copy link
Copy Markdown
Member

@dixonwhitmire dixonwhitmire commented Oct 3, 2022

This PR updates Csv To FHIR project to treat "smart_open" as an optional dependency with the following changes:

  • define a new "optimized-streaming" extra in setup.cfg
  • add new functions in support.py to support parsing a uri scheme and opening a file
  • within the new support.py functions deferred imports are used

Note that the GitHub Actions test jobs DO NOT load the smart_open dependency at this time.

One caveat with this approach is that we can't use mocks or fixtures within the tests to simulate when smart_open is included or excluded since deferred imports are used. However one could argue that this would require more of an "integration" test to fully ensure that data contracts are read from external systems correctly.

closes #50

Signed-off-by: Dixon Whitmire dixonwh@gmail.com

Signed-off-by: Dixon Whitmire <dixonwh@gmail.com>
Comment thread tests/test_support.py Fixed
Comment thread tests/test_support.py Fixed
Signed-off-by: Dixon Whitmire <dixonwh@gmail.com>
@hammadk373 hammadk373 mentioned this pull request Oct 3, 2022
@dixonwhitmire
Copy link
Copy Markdown
Member Author

@klwhaley - Additionally, I have tested the "README" demo commands with and without the "optimized-streaming" dependency.

Comment thread tests/test_support.py Outdated
[
("/opt/data/data.csv", "file"),
("C:/data/data.csv", "file"),
("http://somserver/file.dat", "http")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: did you mean to make this someserver? (missing e)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update. Thanks!

Signed-off-by: Dixon Whitmire <dixonwh@gmail.com>
@dixonwhitmire dixonwhitmire merged commit 03be50e into main Oct 3, 2022
@dixonwhitmire dixonwhitmire deleted the optional-smart_open branch October 3, 2022 21:45
@klwhaley klwhaley added this to the 1.2b1 milestone Oct 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor to make smart_open an Optional Component

4 participants