Skip to content

OUT-3815 | OUT-3819: collapse qb_product_sync to one row per product, drop price columns #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SandipBajracharya merged 2 commits into from
Jun 5, 2026
Merged

OUT-3815 | OUT-3819: collapse qb_product_sync to one row per product, drop price columns #259

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f567cbe
feat(OUT-3815): collapse qb_product_sync to one row per product, drop…
SandipBajracharya Jun 4, 2026
8fcbfc5
fix(OUT-3815): return live mapping from handleProductMap on conflict
SandipBajracharya Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@ next-env.d.ts

# local decision notes (not published)
/docs
/supabase/snippets
224 changes: 107 additions & 117 deletions src/app/api/quickbooks/product/product.service.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,10 +208,7 @@ export class ProductService extends BaseService {
* On intial save, save all flatten products. If mapped, we include and if not, those are excluded
* On every save after that, we update the record on the basis of productId
*/
async handleProductMap(
body: ProductMappingSchemaType,
returningFields?: (keyof typeof QBProductSync)[],
) {
async handleProductMap(body: ProductMappingSchemaType) {
const { mappingItems, changedItemReference } = body
const settingService = new SettingService(this.user)
const setting = await settingService.getOneByPortalId([
Expand All @@ -220,49 +217,51 @@ export class ProductService extends BaseService {

return await this.db.transaction(async (tx) => {
this.setTransaction(tx)

if (!setting?.initialProductSettingMap) {
const formattedPayload = mappingItems.map((item) => {
return {
...item,
portalId: this.user.workspaceId,
}
})
const query = this.db.insert(QBProductSync).values(formattedPayload)
const products = returningFields?.length
? await query.returning(
buildReturningFields(QBProductSync, returningFields),
)
: await query.returning()
this.unsetTransaction()
return products
}

if (changedItemReference.length > 0) {
await Promise.all(
changedItemReference?.map(async (item) => {
const payload = {
try {
if (!setting?.initialProductSettingMap) {
const formattedPayload = mappingItems.map((item) => {
return {
...item,
portalId: this.user.workspaceId,
productId: item.id,
name: item.isExcluded ? null : item.qbItem?.name,
description: item.isExcluded ? null : item.description,
qbItemId: item.isExcluded ? null : item.qbItem?.id,
qbSyncToken: item.isExcluded ? null : item.qbItem?.syncToken,
copilotName: item.name,
unitPrice: null,
isExcluded: item.isExcluded,
}
const conditions = and(
eq(QBProductSync.portalId, this.user.workspaceId),
eq(QBProductSync.productId, item.id),
) as WhereClause
await this.updateOrCreateQBProduct(payload, conditions)
}),
)
}
})
// Skip products already saved so a repeated save doesn't error.
await this.db
.insert(QBProductSync)
.values(formattedPayload)
.onConflictDoNothing({
target: [QBProductSync.portalId, QBProductSync.productId],
where: isNull(QBProductSync.deletedAt),
})
return await this.getAll()
}

this.unsetTransaction()
return await this.getAll()
if (changedItemReference.length > 0) {
await Promise.all(
changedItemReference?.map(async (item) => {
const payload = {
portalId: this.user.workspaceId,
productId: item.id,
name: item.isExcluded ? null : item.qbItem?.name,
description: item.isExcluded ? null : item.description,
qbItemId: item.isExcluded ? null : item.qbItem?.id,
qbSyncToken: item.isExcluded ? null : item.qbItem?.syncToken,
copilotName: item.name,
isExcluded: item.isExcluded,
}
const conditions = and(
eq(QBProductSync.portalId, this.user.workspaceId),
eq(QBProductSync.productId, item.id),
) as WhereClause
await this.updateOrCreateQBProduct(payload, conditions)
}),
)
}

return await this.getAll()
} finally {
this.unsetTransaction()
}
})
}

Expand Down Expand Up @@ -362,15 +361,7 @@ export class ProductService extends BaseService {
const mappedProducts = await this.getAllByProductId(
productResource.id,
mappedConditions,
[
'id',
'qbItemId',
'qbSyncToken',
'name',
'description',
'unitPrice',
'copilotName',
],
['id', 'qbItemId', 'qbSyncToken', 'name', 'description', 'copilotName'],
)

if (!mappedProducts || !mappedProducts.length) {
Expand Down Expand Up @@ -451,9 +442,6 @@ export class ProductService extends BaseService {
Name: qbItemName,
sparse: true,
...(productDescription && { Description: productDescription }),
...(product.unitPrice
? { UnitPrice: parseFloat(product.unitPrice) / 100 }
: {}),
IncomeAccountRef: {
value: z.string().parse(incomeAccountRef),
},
Expand Down Expand Up @@ -504,72 +492,74 @@ export class ProductService extends BaseService {

await this.db.transaction(async (tx) => {
this.setTransaction(tx)
try {
const mappedProduct = await this.getOne(
// 01. if this product is already mapped to a QB item, do nothing.
and(
eq(QBProductSync.portalId, this.user.workspaceId),
eq(QBProductSync.productId, productResource.id),
) as WhereClause,
['id'],
)

const mappedProduct = await this.getOne(
// 01. if this product is already mapped to a QB item, do nothing.
and(
eq(QBProductSync.portalId, this.user.workspaceId),
eq(QBProductSync.productId, productResource.id),
) as WhereClause,
['id'],
)

addSyncBreadcrumb('Product mapping check', {
alreadyMapped: !!mappedProduct,
})
if (mappedProduct) {
console.info('Product already mapped to a QB item; skipping')
return
}
addSyncBreadcrumb('Product mapping check', {
alreadyMapped: !!mappedProduct,
})
if (mappedProduct) {
console.info('Product already mapped to a QB item; skipping')
return
}

const qbItemName = truncateForQB(
replaceSpecialCharsForQB(productResource.name),
)
const productDescription = convert(productResource.description)
const qbItemName = truncateForQB(
replaceSpecialCharsForQB(productResource.name),
)
const productDescription = convert(productResource.description)

// check if item with name exists in QBO
let qbItem = await intuitApi.getAnItem(qbItemName, undefined, true)

if (!qbItem) {
const tokenService = new TokenService(this.user)
const incomeAccountRef =
await tokenService.checkAndUpdateAccountStatus(
AccountTypeObj.Income,
qbTokenInfo.intuitRealmId,
intuitApi,
qbTokenInfo.incomeAccountRef,
)
// create item in QB. No price at product.created time — invoice lines
// carry their own UnitPrice.
qbItem = await this.createItemInQB(
{
productName: z.string().parse(qbItemName),
incomeAccRefVal: z.string().parse(incomeAccountRef),
productDescription,
},
intuitApi,
)
}

// check if item with name exists in QBO
let qbItem = await intuitApi.getAnItem(qbItemName, undefined, true)
// map product to the QB item
await this.createQBProduct({
portalId: this.user.workspaceId,
productId: productResource.id,
qbItemId: qbItem.Id,
qbSyncToken: qbItem.SyncToken,
name: qbItemName,
copilotName: productResource.name,
description: productDescription,
})

if (!qbItem) {
const tokenService = new TokenService(this.user)
const incomeAccountRef = await tokenService.checkAndUpdateAccountStatus(
AccountTypeObj.Income,
qbTokenInfo.intuitRealmId,
intuitApi,
qbTokenInfo.incomeAccountRef,
)
// create item in QB. No price at product.created time — invoice lines
// carry their own UnitPrice.
qbItem = await this.createItemInQB(
{
productName: z.string().parse(qbItemName),
incomeAccRefVal: z.string().parse(incomeAccountRef),
productDescription,
},
intuitApi,
console.info(
'WebhookService#webhookProductCreated | Product created in QB',
)
await this.logSync(productResource.id, qbItem.Id, EventType.CREATED, {
productName: productResource.name,
qbItemName: qbItem.Name,
})
} finally {
this.unsetTransaction()
}

// map product to the QB item
await this.createQBProduct({
portalId: this.user.workspaceId,
productId: productResource.id,
qbItemId: qbItem.Id,
qbSyncToken: qbItem.SyncToken,
name: qbItemName,
copilotName: productResource.name,
description: productDescription,
})

console.info(
'WebhookService#webhookProductCreated | Product created in QB',
)
await this.logSync(productResource.id, qbItem.Id, EventType.CREATED, {
productName: productResource.name,
qbItemName: qbItem.Name,
})

this.unsetTransaction()
})
}

Expand Down Expand Up @@ -708,7 +698,7 @@ export class ProductService extends BaseService {
async unmapProducts(qbItemId: string): Promise<void> {
await this.db
.update(QBProductSync)
.set({ qbItemId: null, qbSyncToken: null, name: null, unitPrice: null })
.set({ qbItemId: null, qbSyncToken: null, name: null })
.where(
and(
eq(QBProductSync.qbItemId, qbItemId),
Expand Down
4 changes: 4 additions & 0 deletions src/db/migrations/20260603102427_collapse_qb_product_sync_one_row_drop_price_columns.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
CREATE UNIQUE INDEX "uq_qb_product_sync_product_active" ON "qb_product_sync" USING btree ("portal_id","product_id") WHERE "qb_product_sync"."deleted_at" is null;--> statement-breakpoint

@greptile-apps greptile-apps Bot Jun 4, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Non-concurrent unique index creation takes an ACCESS EXCLUSIVE lock

CREATE UNIQUE INDEX (without CONCURRENTLY) acquires an ACCESS EXCLUSIVE lock on qb_product_sync for the entire duration of the index build, blocking all reads and writes. On a busy production table this will cause visible downtime. CREATE UNIQUE INDEX CONCURRENTLY builds the index without the table-level lock, but it cannot run inside a transaction block — if drizzle-kit migrate wraps statements in a transaction, the CONCURRENTLY keyword must be handled via a raw migration outside that transaction.

@SandipBajracharya SandipBajracharya Jun 4, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're managing all migrations via drizzle-kit. Besides, the records in the table is quiet low (2k) so this might not have severe impact.

ALTER TABLE "qb_product_sync" DROP COLUMN "price_id";--> statement-breakpoint
ALTER TABLE "qb_product_sync" DROP COLUMN "unit_price";--> statement-breakpoint
ALTER TABLE "qb_product_sync" DROP COLUMN "copilot_unit_price";
Loading
Loading