Skip to content

Citus distribution support and async/remote indexing#3683

Merged
lmsurpre merged 45 commits into
mainfrom
robin-citus-1
Jun 21, 2022
Merged

Citus distribution support and async/remote indexing#3683
lmsurpre merged 45 commits into
mainfrom
robin-citus-1

Conversation

@punktilious
Copy link
Copy Markdown
Collaborator

@punktilious punktilious commented Jun 2, 2022

No description provided.

punktilious added 30 commits March 7, 2022 11:29
@punktilious
issue #3437 experimental support for Citus database
c4a11be 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 schema build changes to support Citus distribution
7244dce 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 citus requires some custom ingestion logic
a7f2379 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 distribute add_any_resource function in Citus
4398119 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 merge latest main
d0eaa36 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
Merge remote-tracking branch 'origin/HEAD' into robin-citus-1
8a237ad
@punktilious
Merge remote-tracking branch 'origin/HEAD' into robin-citus-1
10df2e8
@punktilious
issue #3437 distributed schema using shard_key
68819ab 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index kafka consumer
0752bbb 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index support for profile, tags and security
6343ab9 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 remote index wait until server transaction commits
a8acdb8 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 add logical_resource_ident and ref param support
7125f67 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fix reference unit tests for new ref_values table
f705e5a 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use logical_resource_ident for absolute references
0c1702b 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 compilation issues from shard update
8c5cc75 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fix remote index processing for new logical_resource_iden…
ca0bfa0 
…t recs

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 tidied javadoc
246897f 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed issues using Citus
24bb883 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use identity cache for more ids
3e6699d 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 merge with main for R4B
19daa4c 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 do not apply permissions to tables not created
fcf2c1e 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 initial derby unit test for fhir-remote-index
f2687cc 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 more unit test coverage for fhir-remote-index
f44b4cb 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 delete parameters from new xx_ref_values
4de35a5 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 allow reindex to be forced after database migration
40e2454 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 use logical_resource_ident for read and vread with citus
c3ab574 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed postgres and db2 add_any_resource to use lr ident
fe03611 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 db-type arg now required when running schema tool
1644408 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed schema migration for Db2
7ca7633 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixed missing grant for Db2
89e14ad 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@lmsurpre lmsurpre added the new-module Flag for identifying PRs that introduce a new module (and therefor require manual work for release) label Jun 13, 2022
@lmsurpre
Copy link
Copy Markdown
Member

lmsurpre commented Jun 13, 2022

note to self: we need to add the fhir-remote-index module to the list of cli tools. possibly it should get its on linux container image too.

lmsurpre
lmsurpre reviewed Jun 13, 2022
View reviewed changes
Comment thread fhir-remote-index/src/main/java/com/ibm/fhir/remote/index/app/Main.java
@punktilious
issue #3697 documentation for fhir-remote-index
9d96843 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3698 document schema changes incl citus support
451e804 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
lmsurpre
lmsurpre reviewed Jun 13, 2022
View reviewed changes
Comment thread fhir-remote-index/README.md Outdated
@punktilious
issue #3437 addressed review comments. Support whole system token val…
7a9221a 
…ues. Remove ref_version_id

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 fixing drop column for ref_version_id
dec8d3a 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 tab to space and fixed comments
dd08e87 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
lmsurpre
lmsurpre reviewed Jun 15, 2022
View reviewed changes
Comment thread fhir-database-utils/src/main/java/com/ibm/fhir/database/utils/common/DropView.java Outdated
@punktilious
issue #3437 fixed fhir-bucket and corrected DropView comment
f37233b 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 corrected DropView comment
5f7e25d 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 javadoc review comments
150e3c3 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
punktilious and others added 4 commits June 17, 2022 10:28
@punktilious
issue #3437 skip parameter storage for remote index with derby and db2
5aa9325 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 add instanceIdentifier to remote index messages and other…
c446a15 
… review fixes

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3713 use distributed add_any_resource function for citus (#3718)
583903a 
* issue #3713 use distributed add_any_resource function for citus

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>

* issue #3437 fix usage of logicalResourceId and resourceId for remote index

Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious
issue #3437 inject instanceIdentifier when sending remote index payload
6c8d712 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@lmsurpre lmsurpre self-requested a review June 21, 2022 13:02
lmsurpre
lmsurpre approved these changes Jun 21, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@lmsurpre lmsurpre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of good stuff in here. I reviewed all the files and ran locally for bit with remote indexing. I didn't run any of the Citus stuff.

@punktilious
issue #3437 more schema design documentation
59e3c24 
Signed-off-by: Robin Arnold <robin.arnold@ibm.com>
@punktilious punktilious added ci-skip Skips the CI Build and removed ci-skip Skips the CI Build labels Jun 21, 2022
@lmsurpre lmsurpre merged commit b6043f3 into main Jun 21, 2022
@lmsurpre lmsurpre deleted the robin-citus-1 branch June 21, 2022 14:02
@lmsurpre lmsurpre added the reindex Resolution of issue will require a $reindex during upgrade label Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lmsurpre lmsurpre lmsurpre approved these changes

Assignees

No one assigned

Labels

new-module Flag for identifying PRs that introduce a new module (and therefor require manual work for release) reindex Resolution of issue will require a $reindex during upgrade

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@punktilious @lmsurpre