feat: replace ReceivingService/AggregationService with Quartz-backed …#181
Open
mmalkhatib wants to merge 3 commits into
Open
feat: replace ReceivingService/AggregationService with Quartz-backed …#181mmalkhatib wants to merge 3 commits into
mmalkhatib wants to merge 3 commits into
Conversation
…jobs via SW-Scheduler Replace the previous polling services with IScheduledJob<TParam> implementations (ReceivingJob, AggregationJob) driven by SW-Scheduler — a typed Quartz.NET wrapper. Quartz tables are added via EF Core migrations on all three DB providers (PgSql, MySql, MsSql). SchedulerSeedService re-registers active subscriptions idempotently on startup; SubscriptionSchedulerService bridges the Schedule domain entity to IScheduleRepository. Clustering is enabled so only one node fires each trigger in a multi-node deployment. Also adds SW.Bitween.IntegrationTests with Testcontainers-based tests for entity persistence, bus connectivity, receiving jobs, and aggregation jobs. Docs: docs/scheduler.md (new), docs/architecture.md (scheduler section added). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 20562020 | Triggered | Generic Password | 5f87616 | SW.Bitween.PgSql/BitweenDbContextFactory.cs | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
|
Warning Review limit reached
More reviews will be available in 42 minutes and 32 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughReplaces the polling-based ChangesQuartz-based scheduling migration
Sequence DiagramsequenceDiagram
participant Startup
participant SchedulerSeedService
participant SubscriptionSchedulerService
participant IScheduleRepository
participant Quartz
rect rgba(100, 149, 237, 0.5)
note over Startup,Quartz: Application startup
Startup->>SchedulerSeedService: start as hosted service
SchedulerSeedService->>SubscriptionSchedulerService: ScheduleAll(sub) per active Receiving/Aggregation sub
SubscriptionSchedulerService->>IScheduleRepository: ScheduleIfNotExists(cronKey, ReceivingJobParams)
IScheduleRepository->>Quartz: register cron trigger in qrtz_* tables
end
rect rgba(144, 238, 144, 0.5)
note over Quartz,Quartz: Scheduled execution
Quartz->>ReceivingJob: Execute(ReceivingJobParams)
ReceivingJob->>ReceivingJob: RunReceiver (native or serverless branch)
ReceivingJob->>XchangeService: SubmitSubscriptionXchange per file
ReceivingJob->>BitweenDbContext: update health/schedules, SaveChangesAsync
end
rect rgba(255, 165, 0, 0.5)
note over UpdateHandler,Quartz: Subscription update sync
UpdateHandler->>SubscriptionSchedulerService: Sync(entity, oldSchedules)
SubscriptionSchedulerService->>IScheduleRepository: TryUnschedule removed schedules
SubscriptionSchedulerService->>IScheduleRepository: ScheduleIfNotExists new schedules
end
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
🧹 Nitpick comments (1)
SW.Bitween.Api/Services/SchedulerSeedService.cs (1)
31-45: ⚡ Quick winHonor cancellation during startup seeding loop.
The loop should stop promptly when shutdown is requested; otherwise startup seeding can delay host termination.
Suggested patch
foreach (var sub in subscriptions) { + if (stoppingToken.IsCancellationRequested) + break; + try { await subScheduler.ScheduleAll(sub);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@SW.Bitween.Api/Services/SchedulerSeedService.cs` around lines 31 - 45, The foreach loop in SchedulerSeedService that iterates through subscriptions and calls ScheduleAll does not check for cancellation requests, which can delay host shutdown. Modify the loop to check for cancellation by calling ThrowIfCancellationRequested on the CancellationToken parameter (ensure the method accepts a CancellationToken parameter if it does not already) and add this check at the beginning of each iteration before calling await subScheduler.ScheduleAll(sub), allowing the seeding process to be interrupted gracefully when shutdown is requested.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/architecture.md`:
- Around line 89-98: The fenced code block at line 89 in docs/architecture.md is
missing a language identifier on the opening fence, which triggers the MD040
markdown lint rule. Add a language identifier such as `text` to the opening
fence (change ``` to ```text) to specify the code block type and satisfy the
linting requirement.
In `@docs/scheduler.md`:
- Around line 108-111: The fenced code blocks in the scheduler.md file are
missing language identifiers, which violates markdownlint rule MD040. Add the
language identifier `text` to all fenced code blocks that lack one.
Specifically, add `text` to the opening fence for the code block at lines
108-111 that contains the receiver and aggregator subscription ID patterns, and
also add `text` to the opening fence for the code block at lines 141-147 that
contains the startup and SchedulerSeedService tree structure. Change each
opening ``` to ```text.
In `@SW.Bitween.Api/Resources/Subscriptions/Update.cs`:
- Around line 58-63: The scheduler operations (_subScheduler.Sync in Update.cs
at lines 58-63, _subScheduler.RunNow in AggregateNow.cs at lines 29-32, and
_subScheduler.RunNow in ReceiveNow.cs at lines 29-32) execute after database
commit without durability guarantees, causing transient failures to return
errors despite successful database changes. For each of the three affected
locations, refactor the scheduler call to route failures into a durable
reconciliation mechanism (such as a retry queue, outbox pattern, or background
repair job) instead of directly propagating the failure to the request handler.
This ensures that scheduler state can be reconciled asynchronously even if the
immediate scheduler operation fails, maintaining consistency between database
state and runtime scheduling.
In `@SW.Bitween.Api/Services/AggregationJob.cs`:
- Around line 30-37: The eligibility query in AggregationJob.cs uses incorrect
join correlations that risk matching on wrong keys and producing duplicate
xchange.Id candidates. In the xchangeQuery LINQ expression, the joins on lines
32-33 correlate using result.Id and agg.Id directly, but should instead use
proper foreign key relationships (such as result.XchangeId and agg.XchangeId) to
correctly associate XchangeResult and XchangeAggregation records with their
parent Xchange. Additionally, since multiple XchangeResult rows can exist for a
single Xchange, add a Distinct() call on xchange.Id within the query (before or
after the where clause) to eliminate duplicate candidates, or alternatively
refactor the joins to use Any() existence checks instead of left outer joins to
avoid duplication entirely. This ensures accurate aggregation mapping without
missed or duplicated entries.
In `@SW.Bitween.Api/Services/ReceivingJob.cs`:
- Around line 57-74: The receiver finalization is only executed on the success
path, causing resource leaks if exceptions occur during file operations. Wrap
the receiver initialization and the entire file processing loop (from
receiver.Initialize() to the foreach block) in a try block, and place the
receiver.Finalize() call in a finally block to guarantee it always executes.
Apply this same try/finally pattern to both the native adapter branch (starting
with nativeAdapterDiscovery.GetNativeReceiver) and the second branch mentioned
at lines 76-92 to ensure each receiver's finalization method is always called
regardless of success or failure.
In `@SW.Bitween.Api/Services/SubscriptionSchedulerService.cs`:
- Around line 21-26: The Sync method currently unschedules all old schedules
before attempting to schedule new ones. If any Schedule call fails, the
subscription loses trigger coverage even though the domain changes were already
committed. Refactor the logic in the Sync method to reverse the order: first
call Schedule for all new schedules in sub.Schedules, then afterward call
TryUnschedule only for the schedules that existed in oldSchedules but are no
longer in sub.Schedules. This way, existing trigger coverage is preserved if any
new schedule operations fail. Alternatively or additionally, consider wrapping
both the unschedule and schedule operations in a transactional boundary to
ensure that if any operation fails, no triggers are lost.
In `@SW.Bitween.IntegrationTests/Fixtures/BitweenFixture.cs`:
- Around line 46-49: The BitweenFixture class creates NpgsqlDataSource and a
host without storing them or disposing them, causing resource leaks. Store both
the host (created before line 46) and the dataSource variable (from
dataSourceBuilder.Build() at line 46-49) as private fields in the BitweenFixture
class, then ensure both are properly disposed in the fixture's Dispose method or
cleanup logic. Apply the same fix to the similar code at lines 119-124 where
another data source or host is created without being stored and disposed.
In `@SW.Bitween.IntegrationTests/Tests/AggregationTests.cs`:
- Around line 67-69: The test currently uses FirstOrDefaultAsync() to retrieve
the aggregation Xchange and then only asserts it is not null, which allows
duplicate rows to pass since FirstOrDefaultAsync returns the first match even if
multiple exist. Replace the FirstOrDefaultAsync query with CountAsync() to get
the total count of matching Xchange rows where SubscriptionId equals aggSub.Id,
then assert that the count equals exactly 1 using Assert.Equal. This will
enforce that exactly one aggregation Xchange row exists and catch any
duplicate-row regressions.
In `@SW.Bitween.MsSql/BitweenDbContextFactory.cs`:
- Around line 11-13: The connStr assignment in BitweenDbContextFactory.cs
contains a hardcoded privileged sa password in the fallback connection string,
which is a security credential leak. Remove the fallback default connection
string that includes credentials entirely, and instead throw an
InvalidOperationException or similar exception if the
"ConnectionStrings__BitweenDb" environment variable is not set. This ensures
credentials are never embedded in source code and must be provided via
environment configuration in any deployment environment.
In `@SW.Bitween.MsSql/Migrations/20260614184136_Quartz.Designer.cs`:
- Around line 1477-1487: The HasData call in the migration contains a hardcoded
API key "7facc758283844b49cc4ffd26a75b1de" which poses a security risk by
exposing credentials in version control. Remove the hardcoded Key value from the
seed data in the HasData method call, and instead implement environment-based
seeding through a separate configuration file or seeding strategy that is
excluded from version control, ensuring that credentials are loaded from secure
configuration sources only during runtime rather than being embedded in the
migration code itself.
In `@SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs`:
- Around line 224-229: The MySQL migration contains FK constraint names with
invalid `~` character suffixes (e.g.,
FK_QRTZ_blob_triggers_QRTZ_triggers_sched_name_trigger_name_tri~) that are
non-standard in SQL and likely invalid in MySQL, and these explicit names are
not reflected in the model snapshot, causing a schema desync. Choose one
approach: (1) Add explicit HasConstraintName() calls in the model snapshot for
all four QRTZ foreign keys (blob_triggers, cron_triggers, simple_triggers, and
simprop_triggers) using valid constraint names without the `~` suffix and
respecting MySQL's 64-character identifier limit, or (2) Remove the explicit
name: parameters from all four ForeignKey definitions in the migration file and
let EF Core auto-generate consistent constraint names. Either approach will
ensure the migration and snapshot are synchronized with valid constraint names.
In `@SW.Bitween.MySql/Migrations/20260614184126_Quartz.Designer.cs`:
- Around line 1477-1484: The HasData() call in the migration file
(SW.Bitween.MySql/Migrations/20260614184126_Quartz.Designer.cs) contains a
hardcoded API key "7facc758283844b49cc4ffd26a75b1de" for PartnerApiCredentials
seeding, which creates a security vulnerability by making a predictable
credential available wherever migrations are applied. Remove this deterministic
API key from the HasData() configuration and either relocate API credential
provisioning to a separate secure provisioning flow outside of migrations, or
seed a disabled/non-authenticating placeholder credential instead. After
updating the model seed configuration to remove or disable the hardcoded key,
regenerate this migration file and its snapshot from the updated seed data so
the changes are properly reflected in the migration artifacts.
In `@SW.Bitween.PgSql/Migrations/20260614184104_Quartz.Designer.cs`:
- Around line 1685-1692: The HasData call in the migration contains a hardcoded
API key value ("7facc758283844b49cc4ffd26a75b1de") which is a security risk.
Remove the Key property from the hardcoded seed data in the migration file, and
instead implement secure credential provisioning at runtime using
environment-specific secrets or a secure key management system. After making
these changes, regenerate the migration files and snapshots to ensure they
reflect the updated configuration without the hardcoded credential.
In `@SW.Bitween.Web/Startup.cs`:
- Around line 141-165: The scheduler registration methods (AddPgSqlScheduler,
AddSqlServerScheduler, and AddMySqlScheduler) are being called with the
connectionString before the Authentication=Active Directory Default
configuration is appended to it at line 244. To fix this, move the connection
string authentication configuration logic to execute before the scheduler
registration block, ensuring both the Quartz scheduler and Entity Framework
receive the same properly-configured connection string with authentication
details included.
---
Nitpick comments:
In `@SW.Bitween.Api/Services/SchedulerSeedService.cs`:
- Around line 31-45: The foreach loop in SchedulerSeedService that iterates
through subscriptions and calls ScheduleAll does not check for cancellation
requests, which can delay host shutdown. Modify the loop to check for
cancellation by calling ThrowIfCancellationRequested on the CancellationToken
parameter (ensure the method accepts a CancellationToken parameter if it does
not already) and add this check at the beginning of each iteration before
calling await subScheduler.ScheduleAll(sub), allowing the seeding process to be
interrupted gracefully when shutdown is requested.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1d756b2c-0dfc-44c8-a2ae-f5b922240fbe
📒 Files selected for processing (41)
SW.Bitween.Api/Extensions/ScheduleToCronExtension.csSW.Bitween.Api/Resources/Subscriptions/AggregateNow.csSW.Bitween.Api/Resources/Subscriptions/ReceiveNow.csSW.Bitween.Api/Resources/Subscriptions/Update.csSW.Bitween.Api/SW.Bitween.Api.csprojSW.Bitween.Api/Services/AggregationJob.csSW.Bitween.Api/Services/AggregationService.csSW.Bitween.Api/Services/ReceivingJob.csSW.Bitween.Api/Services/ReceivingService.csSW.Bitween.Api/Services/SchedulerSeedService.csSW.Bitween.Api/Services/SubscriptionSchedulerService.csSW.Bitween.IntegrationTests/Adapters/NativeTestReceiver.csSW.Bitween.IntegrationTests/Fixtures/BitweenFixture.csSW.Bitween.IntegrationTests/SW.Bitween.IntegrationTests.csprojSW.Bitween.IntegrationTests/Tests/AggregationTests.csSW.Bitween.IntegrationTests/Tests/BusTests.csSW.Bitween.IntegrationTests/Tests/EntityTests.csSW.Bitween.IntegrationTests/Tests/ReceivingTests.csSW.Bitween.MsSql/BitweenDbContext.csSW.Bitween.MsSql/BitweenDbContextFactory.csSW.Bitween.MsSql/Migrations/20260614184136_Quartz.Designer.csSW.Bitween.MsSql/Migrations/20260614184136_Quartz.csSW.Bitween.MsSql/Migrations/BitweenDbContextModelSnapshot.csSW.Bitween.MsSql/SW.Bitween.MsSql.csprojSW.Bitween.MySql/BitweenDbContext.csSW.Bitween.MySql/BitweenDbContextFactory.csSW.Bitween.MySql/Migrations/20260614184126_Quartz.Designer.csSW.Bitween.MySql/Migrations/20260614184126_Quartz.csSW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.csSW.Bitween.MySql/SW.Bitween.MySql.csprojSW.Bitween.PgSql/BitweenDbContext.csSW.Bitween.PgSql/BitweenDbContextFactory.csSW.Bitween.PgSql/Migrations/20260614184104_Quartz.Designer.csSW.Bitween.PgSql/Migrations/20260614184104_Quartz.csSW.Bitween.PgSql/Migrations/BitweenDbContextModelSnapshot.csSW.Bitween.PgSql/SW.Bitween.PgSql.csprojSW.Bitween.Web/SW.Bitween.Web.csprojSW.Bitween.Web/Startup.csSW.Bitween.slndocs/architecture.mddocs/scheduler.md
💤 Files with no reviewable changes (2)
- SW.Bitween.Api/Services/AggregationService.cs
- SW.Bitween.Api/Services/ReceivingService.cs
Comment on lines
+89
to
+98
| ``` | ||
| SchedulerSeedService (startup) | ||
| └── SubscriptionSchedulerService.ScheduleAll(sub) | ||
| └── IScheduleRepository.ScheduleIfNotExists<Job, Param>(param, cron, key) | ||
| └── Quartz persistent job store (qrtz_* tables, same DB as Bitween) | ||
|
|
||
| Quartz fires at scheduled time | ||
| └── ReceivingJob.Execute(ReceivingJobParams) ← polls receiver adapter, creates Xchanges | ||
| └── AggregationJob.Execute(AggregationJobParams) ← batches successful Xchanges into one | ||
| ``` |
There was a problem hiding this comment.
Add a language identifier to the fenced code block.
Line 89 opens a fenced block without a language token, which triggers MD040. Use something like ```text for this diagram block.
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 89-89: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/architecture.md` around lines 89 - 98, The fenced code block at line 89
in docs/architecture.md is missing a language identifier on the opening fence,
which triggers the MD040 markdown lint rule. Add a language identifier such as
`text` to the opening fence (change ``` to ```text) to specify the code block
type and satisfy the linting requirement.
Source: Linters/SAST tools
Comment on lines
+108
to
+111
| ``` | ||
| receiver-{subscriptionId}-{recurrence}-{on.Ticks}-{backwards ? 1 : 0} | ||
| aggregator-{subscriptionId}-{recurrence}-{on.Ticks}-{backwards ? 1 : 0} | ||
| ``` |
There was a problem hiding this comment.
Add language identifiers to fenced code blocks.
These fences are missing language tags, which triggers markdownlint MD040.
Suggested patch
-```
+```text
receiver-{subscriptionId}-{recurrence}-{on.Ticks}-{backwards ? 1 : 0}
aggregator-{subscriptionId}-{recurrence}-{on.Ticks}-{backwards ? 1 : 0}- +text
startup
└── SchedulerSeedService.ExecuteAsync()
├── query all active Receiving + Aggregation subscriptions with at least one Schedule
</details>
Also applies to: 141-147
<details>
<summary>🧰 Tools</summary>
<details>
<summary>🪛 markdownlint-cli2 (0.22.1)</summary>
[warning] 108-108: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
</details>
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @docs/scheduler.md around lines 108 - 111, The fenced code blocks in the
scheduler.md file are missing language identifiers, which violates markdownlint
rule MD040. Add the language identifier text to all fenced code blocks that
lack one. Specifically, add text to the opening fence for the code block at
lines 108-111 that contains the receiver and aggregator subscription ID
patterns, and also add text to the opening fence for the code block at lines
141-147 that contains the startup and SchedulerSeedService tree structure.
Change each opening totext.
</details>
<!-- fingerprinting:phantom:poseidon:hawk -->
<!-- cr-comment:v1:2c0566807a1294aa279f7270 -->
_Source: Linters/SAST tools_
<!-- This is an auto-generated comment by CodeRabbit -->
Comment on lines
58
to
+63
| await _dbContext.SaveChangesAsync(); | ||
| await _BitweenCache.BroadcastRevoke(); | ||
|
|
||
| // Sync Quartz: unschedule removed entries, schedule new/kept ones. | ||
| await _subScheduler.Sync(entity, oldSchedules); | ||
|
|
There was a problem hiding this comment.
Post-commit scheduler calls are not durable across command handlers.
All three handlers commit DB state first and then invoke scheduler operations without retry/compensation, so transient scheduler failures can return errors after a successful commit and leave runtime scheduling stale.
SW.Bitween.Api/Resources/Subscriptions/Update.cs#L58-L63: route_subScheduler.Sync(...)failures into a durable reconciliation path (retry queue/outbox/background repair), not immediate request failure only.SW.Bitween.Api/Resources/Subscriptions/AggregateNow.cs#L29-L32: make_subScheduler.RunNow(entity)resilient post-commit (retry/compensation) to avoid failed-response-after-commit behavior.SW.Bitween.Api/Resources/Subscriptions/ReceiveNow.cs#L29-L32: apply the same resilient post-commit scheduling strategy as above for symmetry and consistent semantics.
📍 Affects 3 files
SW.Bitween.Api/Resources/Subscriptions/Update.cs#L58-L63(this comment)SW.Bitween.Api/Resources/Subscriptions/AggregateNow.cs#L29-L32SW.Bitween.Api/Resources/Subscriptions/ReceiveNow.cs#L29-L32
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.Api/Resources/Subscriptions/Update.cs` around lines 58 - 63, The
scheduler operations (_subScheduler.Sync in Update.cs at lines 58-63,
_subScheduler.RunNow in AggregateNow.cs at lines 29-32, and _subScheduler.RunNow
in ReceiveNow.cs at lines 29-32) execute after database commit without
durability guarantees, causing transient failures to return errors despite
successful database changes. For each of the three affected locations, refactor
the scheduler call to route failures into a durable reconciliation mechanism
(such as a retry queue, outbox pattern, or background repair job) instead of
directly propagating the failure to the request handler. This ensures that
scheduler state can be reconciled asynchronously even if the immediate scheduler
operation fails, maintaining consistency between database state and runtime
scheduling.
Comment on lines
+30
to
+37
| var xchangeQuery = | ||
| from xchange in dbContext.Set<Xchange>() | ||
| join result in dbContext.Set<XchangeResult>() on xchange.Id equals result.Id | ||
| join agg in dbContext.Set<XchangeAggregation>() on xchange.Id equals agg.Id into xa | ||
| from agg in xa.DefaultIfEmpty() | ||
| where result.Success == true && agg == null && | ||
| xchange.SubscriptionId == aggSub.AggregationForId && !aggSub.Inactive | ||
| select xchange.Id; |
There was a problem hiding this comment.
Fix eligibility query to correlate by foreign keys and avoid duplicate candidates.
Line 32 and Line 33 correlate using result.Id / agg.Id, which risks matching on the wrong key path. Also, joining directly to XchangeResult can duplicate xchange.Id when multiple successful rows exist. This can cause missed or duplicated aggregation mappings. Use FK-based existence checks (Any) or Distinct on xchange.Id before Take(10000).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.Api/Services/AggregationJob.cs` around lines 30 - 37, The
eligibility query in AggregationJob.cs uses incorrect join correlations that
risk matching on wrong keys and producing duplicate xchange.Id candidates. In
the xchangeQuery LINQ expression, the joins on lines 32-33 correlate using
result.Id and agg.Id directly, but should instead use proper foreign key
relationships (such as result.XchangeId and agg.XchangeId) to correctly
associate XchangeResult and XchangeAggregation records with their parent
Xchange. Additionally, since multiple XchangeResult rows can exist for a single
Xchange, add a Distinct() call on xchange.Id within the query (before or after
the where clause) to eliminate duplicate candidates, or alternatively refactor
the joins to use Any() existence checks instead of left outer joins to avoid
duplication entirely. This ensures accurate aggregation mapping without missed
or duplicated entries.
Comment on lines
+57
to
+74
| if (serverlessId.StartsWith(NativeAdapterDiscoveryService.NativePrefix, StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| var receiver = nativeAdapterDiscovery.GetNativeReceiver(serverlessId, startupParameters); | ||
| await receiver.Initialize(); | ||
| var fileList = (await receiver.ListFiles()).ToList(); | ||
|
|
||
| logger.LogInformation("Subscription '{SubId}' found {Count} items for retrieval.", subId, fileList.Count); | ||
|
|
||
| foreach (var file in fileList) | ||
| { | ||
| var xchangeFile = await receiver.GetFile(file); | ||
| logger.LogInformation("Submitting received file for subscriber: '{SubId}'.", subId); | ||
| await xchangeService.SubmitSubscriptionXchange(subId, xchangeFile); | ||
| await receiver.DeleteFile(file); | ||
| } | ||
|
|
||
| await receiver.Finalize(); | ||
| } |
There was a problem hiding this comment.
Guarantee receiver finalization with try/finally in both execution branches.
Line 73 and Line 91 run only on the success path. If any file operation throws, finalization is skipped and remote/session resources can be left open. Wrap each branch in try/finally and always call the corresponding finalize method.
Also applies to: 76-92
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.Api/Services/ReceivingJob.cs` around lines 57 - 74, The receiver
finalization is only executed on the success path, causing resource leaks if
exceptions occur during file operations. Wrap the receiver initialization and
the entire file processing loop (from receiver.Initialize() to the foreach
block) in a try block, and place the receiver.Finalize() call in a finally block
to guarantee it always executes. Apply this same try/finally pattern to both the
native adapter branch (starting with nativeAdapterDiscovery.GetNativeReceiver)
and the second branch mentioned at lines 76-92 to ensure each receiver's
finalization method is always called regardless of success or failure.
Comment on lines
+1477
to
+1487
| b1.WithOwner() | ||
| .HasForeignKey("PartnerId"); | ||
|
|
||
| b1.HasData( | ||
| new | ||
| { | ||
| PartnerId = 1, | ||
| Id = 1, | ||
| Key = "7facc758283844b49cc4ffd26a75b1de", | ||
| Name = "default" | ||
| }); |
There was a problem hiding this comment.
Hardcoded API key in seed data is a security hygiene concern.
Line 1482 contains a hardcoded default API key "7facc758283844b49cc4ffd26a75b1de" that is visible in the migration snapshot and version control. Even though this is test/seed data intended for local development, storing credentials in code and checked-in migrations increases exposure risk.
Recommendation: Consider moving seed data (especially credentials) to a separate configuration file or environment-based seeding that is excluded from version control, or document that this key is non-production only.
🧰 Tools
🪛 Betterleaks (1.3.1)
[high] 1485-1485: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.MsSql/Migrations/20260614184136_Quartz.Designer.cs` around lines
1477 - 1487, The HasData call in the migration contains a hardcoded API key
"7facc758283844b49cc4ffd26a75b1de" which poses a security risk by exposing
credentials in version control. Remove the hardcoded Key value from the seed
data in the HasData method call, and instead implement environment-based seeding
through a separate configuration file or seeding strategy that is excluded from
version control, ensuring that credentials are loaded from secure configuration
sources only during runtime rather than being embedded in the migration code
itself.
Comment on lines
+224
to
+229
| table.ForeignKey( | ||
| name: "FK_QRTZ_blob_triggers_QRTZ_triggers_sched_name_trigger_name_tri~", | ||
| columns: x => new { x.sched_name, x.trigger_name, x.trigger_group }, | ||
| principalTable: "QRTZ_triggers", | ||
| principalColumns: new[] { "sched_name", "trigger_name", "trigger_group" }, | ||
| onDelete: ReferentialAction.Cascade); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify MySQL constraint name truncation consistency across migration files
# Extract all FK constraint names from the MySQL migration
echo "=== MySQL Migration Constraint Names ==="
rg -n "name:\s*\"FK_" SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs | head -20
# Extract all FK constraint names from the MsSql migration for comparison
echo "=== MsSql Migration Constraint Names (for comparison) ==="
rg -n "name:\s*\"FK_" SW.Bitween.MsSql/Migrations/20260614184136_Quartz.cs | head -20
# Count unique constraint names to check for collisions
echo "=== Checking for potential name collisions in MySQL migration ==="
rg "name:\s*\"FK_" SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs | wc -lRepository: simplify9/Bitween-api
Length of output: 1291
🏁 Script executed:
# Read the exact lines from the migration file to see actual constraint names
sed -n '200,330p' SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs | grep -A 5 "ForeignKey"Repository: simplify9/Bitween-api
Length of output: 2298
🏁 Script executed:
# Search for model snapshots that might contain these constraint names
find . -name "*ModelSnapshot*" -o -name "*snapshot*" | grep -i mysql | head -5Repository: simplify9/Bitween-api
Length of output: 128
🏁 Script executed:
# Check the model snapshot for FK constraint names
rg -n "FK_QRTZ_" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | grep -i constraintRepository: simplify9/Bitween-api
Length of output: 47
🏁 Script executed:
# Get more context around FK constraint definitions in the snapshot
rg -B 2 -A 2 "FK_QRTZ_blob_triggers_QRTZ_triggers" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | head -20Repository: simplify9/Bitween-api
Length of output: 47
🏁 Script executed:
# View the structure of the model snapshot file
head -100 ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.csRepository: simplify9/Bitween-api
Length of output: 3769
🏁 Script executed:
# Search for any QRTZ references in the snapshot
rg "QRTZ_" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | head -10Repository: simplify9/Bitween-api
Length of output: 744
🏁 Script executed:
# Search for FK constraint definitions in snapshot
rg "HasForeignKey|ForeignKey.*sched_name" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | head -20Repository: simplify9/Bitween-api
Length of output: 1335
🏁 Script executed:
# Look for constraint name definitions in snapshot
rg "HasConstraintName|HasName.*FK_" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | head -20Repository: simplify9/Bitween-api
Length of output: 208
🏁 Script executed:
# Search for QRTZ FK constraint names with more context
rg -B 2 -A 2 "SchedulerName.*TriggerName.*TriggerGroup" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.csRepository: simplify9/Bitween-api
Length of output: 2536
🏁 Script executed:
# Get more context around QRTZ FK definitions in snapshot
rg -B 5 -A 5 "FK_QRTZ_blob_triggers|blob_triggers.*HasForeignKey" ./SW.Bitween.MySql/Migrations/BitweenDbContextModelSnapshot.cs | head -40Repository: simplify9/Bitween-api
Length of output: 47
🏁 Script executed:
# Check if other migrations also have truncated FK names with ~
find . -path "*/Migrations/*Quartz.cs" -type f | xargs grep -l "FK_QRTZ" | head -10Repository: simplify9/Bitween-api
Length of output: 175
🏁 Script executed:
# Compare MsSql and MySql migrations side by side for constraint name patterns
echo "=== MsSql constraint name count ===" && \
rg "name:\s*\"FK_" ./SW.Bitween.MsSql/Migrations/20260614184136_Quartz.cs | wc -l && \
echo "=== MySql constraint name count ===" && \
rg "name:\s*\"FK_" ./SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs | wc -lRepository: simplify9/Bitween-api
Length of output: 141
🏁 Script executed:
# Look for any database execution or errors related to these constraint names
rg "tri~|trigger_name_" . --type-list | head -10Repository: simplify9/Bitween-api
Length of output: 455
🏁 Script executed:
# Check git history for this migration file to see when it was created and any related commits
git log --oneline -n 5 -- SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs 2>/dev/null | head -10Repository: simplify9/Bitween-api
Length of output: 164
🏁 Script executed:
# Check if there are any comments in the migration about the truncated names
grep -i "truncat\|constraint.*name\|identif" SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs | head -10Repository: simplify9/Bitween-api
Length of output: 47
🏁 Script executed:
# Verify MySQL constraint naming rules - search for any documentation or tests
rg "constraint.*name|identifier.*length" . --type md | head -10Repository: simplify9/Bitween-api
Length of output: 47
Fix MySQL foreign key constraint names—snapshot and migration are out of sync, and constraint names appear invalid.
The MySQL migration file explicitly specifies FK constraint names with ~ character suffixes (e.g., "FK_QRTZ_blob_triggers_QRTZ_triggers_sched_name_trigger_name_tri~"), but the model snapshot does not include corresponding HasConstraintName() calls for these QRTZ FKs. This creates two problems:
-
Invalid Constraint Names: The
~character is non-standard in SQL identifiers and likely invalid in MySQL. MySQL identifier rules typically allow alphanumeric characters, underscores, and dollar signs only. -
Schema Desync: When the migration runs, EF Core will auto-generate constraint names based on the snapshot's
HasForeignKey()definitions (without explicit names), which may differ from the manually truncated names in the migration file. This causes the actual database schema to diverge from the migration's intent.
Required action: Either:
- Add explicit
HasConstraintName()calls to the model snapshot for all QRTZ FK constraints (lines matching blob_triggers, cron_triggers, simple_triggers, simprop_triggers), using valid constraint names (remove the~suffix and properly truncate to MySQL's 64-character identifier limit), OR - Remove explicit
name:parameters from the migration and let EF Core auto-generate consistent names.
Apply to all four affected FKs: blob_triggers (224–229), cron_triggers (251–256), simple_triggers (277–282), simprop_triggers (316–321).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.MySql/Migrations/20260614184126_Quartz.cs` around lines 224 - 229,
The MySQL migration contains FK constraint names with invalid `~` character
suffixes (e.g.,
FK_QRTZ_blob_triggers_QRTZ_triggers_sched_name_trigger_name_tri~) that are
non-standard in SQL and likely invalid in MySQL, and these explicit names are
not reflected in the model snapshot, causing a schema desync. Choose one
approach: (1) Add explicit HasConstraintName() calls in the model snapshot for
all four QRTZ foreign keys (blob_triggers, cron_triggers, simple_triggers, and
simprop_triggers) using valid constraint names without the `~` suffix and
respecting MySQL's 64-character identifier limit, or (2) Remove the explicit
name: parameters from all four ForeignKey definitions in the migration file and
let EF Core auto-generate consistent constraint names. Either approach will
ensure the migration and snapshot are synchronized with valid constraint names.
Comment on lines
+1477
to
+1484
| b1.HasData( | ||
| new | ||
| { | ||
| PartnerId = 1, | ||
| Id = 1, | ||
| Key = "7facc758283844b49cc4ffd26a75b1de", | ||
| Name = "default" | ||
| }); |
There was a problem hiding this comment.
Avoid deterministic seeded API keys in migration artifacts.
Lines 1477-1484 seed a fixed API key into PartnerApiCredentials. That makes a predictable credential available wherever migrations are applied unless manually rotated, which weakens auth guarantees.
Please move this to a secure provisioning flow (or seed a disabled/non-auth credential), then regenerate this migration/snapshot from the updated model seed configuration.
🧰 Tools
🪛 Betterleaks (1.3.1)
[high] 1482-1482: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.MySql/Migrations/20260614184126_Quartz.Designer.cs` around lines
1477 - 1484, The HasData() call in the migration file
(SW.Bitween.MySql/Migrations/20260614184126_Quartz.Designer.cs) contains a
hardcoded API key "7facc758283844b49cc4ffd26a75b1de" for PartnerApiCredentials
seeding, which creates a security vulnerability by making a predictable
credential available wherever migrations are applied. Remove this deterministic
API key from the HasData() configuration and either relocate API credential
provisioning to a separate secure provisioning flow outside of migrations, or
seed a disabled/non-authenticating placeholder credential instead. After
updating the model seed configuration to remove or disable the hardcoded key,
regenerate this migration file and its snapshot from the updated seed data so
the changes are properly reflected in the migration artifacts.
Source: Linters/SAST tools
Comment on lines
+1685
to
+1692
| b1.HasData( | ||
| new | ||
| { | ||
| PartnerId = 1, | ||
| Id = 1, | ||
| Key = "7facc758283844b49cc4ffd26a75b1de", | ||
| Name = "default" | ||
| }); |
There was a problem hiding this comment.
Remove the hardcoded API credential seed value.
Seeding a fixed API key in migration/model artifacts creates a predictable credential shared across deployments. Move to secure runtime provisioning (or generate per-environment secret) and regenerate migrations/snapshots.
🧰 Tools
🪛 Betterleaks (1.3.1)
[high] 1690-1690: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.PgSql/Migrations/20260614184104_Quartz.Designer.cs` around lines
1685 - 1692, The HasData call in the migration contains a hardcoded API key
value ("7facc758283844b49cc4ffd26a75b1de") which is a security risk. Remove the
Key property from the hardcoded seed data in the migration file, and instead
implement secure credential provisioning at runtime using environment-specific
secrets or a secure key management system. After making these changes,
regenerate the migration files and snapshots to ensure they reflect the updated
configuration without the hardcoded credential.
Source: Linters/SAST tools
Comment on lines
+141
to
+165
| // Register the persistent Quartz scheduler using the same DB as Bitween. | ||
| // Clustering is enabled so only one node fires each job in a multi-node deployment. | ||
| if (string.Equals(bitweenOptions.DatabaseType, RelationalDbType.PgSql.ToString(), StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| services.AddPgSqlScheduler( | ||
| connectionString: connectionString, | ||
| schema: PgSql.BitweenDbContext.Schema, | ||
| configure: o => o.EnableClustering = true, | ||
| assemblies: typeof(BitweenDbContext).Assembly); | ||
| } | ||
| else if (string.Equals(bitweenOptions.DatabaseType, RelationalDbType.MsSql.ToString(), StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| services.AddSqlServerScheduler( | ||
| connectionString: connectionString, | ||
| configure: o => o.EnableClustering = true, | ||
| assemblies: typeof(BitweenDbContext).Assembly); | ||
| } | ||
| else | ||
| { | ||
| // MySql (default) | ||
| services.AddMySqlScheduler( | ||
| connectionString: connectionString, | ||
| configure: o => o.EnableClustering = true, | ||
| assemblies: typeof(BitweenDbContext).Assembly); | ||
| } |
There was a problem hiding this comment.
Align scheduler and EF SQL connection auth configuration.
Line 153 passes connectionString into Quartz before Line 244 appends Authentication=Active Directory Default. In SQL managed-identity mode, EF uses the updated string but the scheduler uses the stale one, which can break job persistence/auth while normal EF traffic works.
Suggested fix
- // Register the persistent Quartz scheduler using the same DB as Bitween.
+ // Normalize connection string before both scheduler and DbContext registration.
+ if (string.Equals(bitweenOptions.DatabaseType, RelationalDbType.MsSql.ToString(), StringComparison.OrdinalIgnoreCase)
+ && bitweenOptions.UseAzureManagedIdentity
+ && !connectionString.Contains("Authentication=", StringComparison.OrdinalIgnoreCase))
+ {
+ connectionString += ";Authentication=Active Directory Default";
+ }
+
+ // Register the persistent Quartz scheduler using the same DB as Bitween.
// Clustering is enabled so only one node fires each job in a multi-node deployment.
if (string.Equals(bitweenOptions.DatabaseType, RelationalDbType.PgSql.ToString(), StringComparison.OrdinalIgnoreCase))
{
services.AddPgSqlScheduler(
connectionString: connectionString,
@@
else if (string.Equals(bitweenOptions.DatabaseType, RelationalDbType.MsSql.ToString(), StringComparison.OrdinalIgnoreCase))
{
services.AddSqlServerScheduler(
connectionString: connectionString,
configure: o => o.EnableClustering = true,
assemblies: typeof(BitweenDbContext).Assembly);
}
@@
- if (bitweenOptions.UseAzureManagedIdentity &&
- !connectionString.Contains("Authentication=", StringComparison.OrdinalIgnoreCase))
- {
- connectionString += ";Authentication=Active Directory Default";
- }
-
services.AddDbContext<BitweenDbContext, MsSql.BitweenDbContext>(c =>
{Also applies to: 241-245
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@SW.Bitween.Web/Startup.cs` around lines 141 - 165, The scheduler registration
methods (AddPgSqlScheduler, AddSqlServerScheduler, and AddMySqlScheduler) are
being called with the connectionString before the Authentication=Active
Directory Default configuration is appended to it at line 244. To fix this, move
the connection string authentication configuration logic to execute before the
scheduler registration block, ensuring both the Quartz scheduler and Entity
Framework receive the same properly-configured connection string with
authentication details included.
Swap all local ProjectReferences to SW-SimplyScheduler with the published
NuGet packages (SimplyWorks.Scheduler.* v8.1.1). Each project now references
the correct package for its role:
SW.Bitween.Api → SimplyWorks.Scheduler.Sdk (IScheduledJob, IScheduleRepository)
SW.Bitween.PgSql → SimplyWorks.Scheduler.PgSql
SW.Bitween.MySql → SimplyWorks.Scheduler.MySql
SW.Bitween.MsSql → SimplyWorks.Scheduler.SqlServer
SW.Bitween.Web → SimplyWorks.Scheduler.EfCore (provider packages come
transitively through the DB provider projects)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the package layout table with the actual NuGet package names and versions (SimplyWorks.Scheduler.* v8.1.1) following the project reference → NuGet migration. Clarifies which project references which package and why. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…jobs via SW-Scheduler
Replace the previous polling services with IScheduledJob implementations (ReceivingJob, AggregationJob) driven by SW-Scheduler — a typed Quartz.NET wrapper. Quartz tables are added via EF Core migrations on all three DB providers (PgSql, MySql, MsSql). SchedulerSeedService re-registers active subscriptions idempotently on startup; SubscriptionSchedulerService bridges the Schedule domain entity to IScheduleRepository. Clustering is enabled so only one node fires each trigger in a multi-node deployment.
Also adds SW.Bitween.IntegrationTests with Testcontainers-based tests for entity persistence, bus connectivity, receiving jobs, and aggregation jobs.
Docs: docs/scheduler.md (new), docs/architecture.md (scheduler section added).
Summary by CodeRabbit
Release Notes
New Features
Tests