Problem
Expensify Classic's domain company-card CSV upload lets you map a Unique ID column, which acts as a dedup key: re-uploading the same (or an overlapping) file skips rows you've already imported instead of duplicating them. New Expensify's company card CSV import (workspace → Company cards → add feed via CSV, with the "Advanced Fields" toggle) has no such field, so re-imports create duplicate transactions. This is the single biggest cause of duplicate-transaction issues on CSV feeds, and it's currently the main gap versus Classic for customers we'd otherwise move to New Expensify.
Root cause / investigation
The backend dedup that makes Unique ID useful already exists and is already on the New Expensify path — nothing new needs to be built there:
- NewDot's
ImportCSVCompanyCards funnels into the same backend chain as Classic's domain CSV upload: ImportCSVCompanyCards → UploadCompanyCardFile → BWM scraper → Auth ScrapeCard → CreateImportedMoneyRequest.
- The PHP layer and Auth's
ImportCSVCompanyCards.cpp are pass-throughs and don't touch externalID.
- Dedup happens at the shared chokepoint
Transaction::isExternalDupe, called from ScrapeCard.cpp at insert time and re-checked in CreateImportedMoneyRequest.cpp. It's the identical dedup Classic uses: per-cardID + externalID match, compound merchant/date/amount flags, "skip on match, don't update," and the "ignore very short externalIDs" rule.
The reason it isn't effective today is purely a frontend issue: in src/libs/actions/CompanyCards.ts, buildOptimisticCompanyCardCSVTransactions() appends a synthetic externalID column and fills every row with a fresh rand64() value on every import, and EXTERNAL_ID is never exposed as a user-mappable column in the mapping screen (CompanyCardsImportedPage.tsx). So re-imports always get brand-new random IDs and isExternalDupe can never match — duplicates get created, exactly like Classic when no Unique ID column is mapped.
Proposed solution (frontend-only)
- Expose a
Unique ID role in the advanced column roles of the company-cards mapping screen (CompanyCardsImportedPage.tsx), pointing at the existing CONST.CSV_IMPORT_COLUMNS.EXTERNAL_ID slot.
- In
CompanyCards.ts, when a user maps a Unique ID column, use that column's value as externalID instead of generating rand64(). Keep the rand64() fallback when the column is unmapped so existing behavior is unchanged for people who don't use it.
- Add the
Unique ID label to the companyCards.addNewCard.csvColumns block in src/languages/en.ts and src/languages/es.ts.
Once stable IDs flow through, isExternalDupe starts matching and re-uploads become idempotent, matching Classic.
Out of scope
Letting people add advanced fields to a feed after the initial upload (a separate, larger change to how a feed's mapping is stored and re-edited) should be tracked as its own issue.
cc @tgolen
Problem
Expensify Classic's domain company-card CSV upload lets you map a
Unique IDcolumn, which acts as a dedup key: re-uploading the same (or an overlapping) file skips rows you've already imported instead of duplicating them. New Expensify's company card CSV import (workspace → Company cards → add feed via CSV, with the "Advanced Fields" toggle) has no such field, so re-imports create duplicate transactions. This is the single biggest cause of duplicate-transaction issues on CSV feeds, and it's currently the main gap versus Classic for customers we'd otherwise move to New Expensify.Root cause / investigation
The backend dedup that makes
Unique IDuseful already exists and is already on the New Expensify path — nothing new needs to be built there:ImportCSVCompanyCardsfunnels into the same backend chain as Classic's domain CSV upload:ImportCSVCompanyCards→UploadCompanyCardFile→ BWM scraper → AuthScrapeCard→CreateImportedMoneyRequest.ImportCSVCompanyCards.cppare pass-throughs and don't touchexternalID.Transaction::isExternalDupe, called fromScrapeCard.cppat insert time and re-checked inCreateImportedMoneyRequest.cpp. It's the identical dedup Classic uses: per-cardID+externalIDmatch, compound merchant/date/amount flags, "skip on match, don't update," and the "ignore very shortexternalIDs" rule.The reason it isn't effective today is purely a frontend issue: in
src/libs/actions/CompanyCards.ts,buildOptimisticCompanyCardCSVTransactions()appends a syntheticexternalIDcolumn and fills every row with a freshrand64()value on every import, andEXTERNAL_IDis never exposed as a user-mappable column in the mapping screen (CompanyCardsImportedPage.tsx). So re-imports always get brand-new random IDs andisExternalDupecan never match — duplicates get created, exactly like Classic when noUnique IDcolumn is mapped.Proposed solution (frontend-only)
Unique IDrole in the advanced column roles of the company-cards mapping screen (CompanyCardsImportedPage.tsx), pointing at the existingCONST.CSV_IMPORT_COLUMNS.EXTERNAL_IDslot.CompanyCards.ts, when a user maps aUnique IDcolumn, use that column's value asexternalIDinstead of generatingrand64(). Keep therand64()fallback when the column is unmapped so existing behavior is unchanged for people who don't use it.Unique IDlabel to thecompanyCards.addNewCard.csvColumnsblock insrc/languages/en.tsandsrc/languages/es.ts.Once stable IDs flow through,
isExternalDupestarts matching and re-uploads become idempotent, matching Classic.Out of scope
Letting people add advanced fields to a feed after the initial upload (a separate, larger change to how a feed's mapping is stored and re-edited) should be tracked as its own issue.
cc @tgolen