Import Consumer Processing - YouNegotiate Docs

Real-World Example

ABC Collections can import consumers in two ways: an employee manually uploads a CSV in the portal, or the company drops a CSV into its SFTP folder. Once the file is accepted, both paths use the same background processing pipeline. That shared pipeline stores the file, creates import history, splits the work into chunks, processes rows safely, records failed rows, finalizes counts, and triggers the correct consumer communications.

Visual Flow

Shared Processing Rules

Stage	What Happens
File storage	The CSV is copied into `import_consumers` so queue workers can read it outside the web request or SFTP connection.
Import history	A `file_upload_histories` record tracks company, sub-account, import type, filename, status, totals, failed count, and whether the source was SFTP.
Chunk creation	The coordinator skips blank rows and creates one `consumer_import_chunks` record per 500 data rows.
Chunk processing	Each chunk validates rows, rejects opted-out imported contact values, performs create/update/delete work, batches database writes, and records failed rows.
Finalization	The final job aggregates chunk counts, marks the import completed or failed, records duration, sends deferred notifications, and runs billing or subscription sync work.

Manual Import Path

Manual import starts from Import File. The creditor selects a mapped Header Profile, chooses Add, Add + CFPB, Update, or Delete, uploads a CSV, and the app validates the file headers before storing the file. Manual imports calculate total records before the coordinator creates chunks.

SFTP Import Path

SFTP import starts from SFTP Connections. The scheduled command scans active mapped SFTP header profiles twice daily. When it finds CSV files in the expected folders, it copies each file into import_consumers, creates SFTP import history, and dispatches the same coordinator job used by manual uploads. After dispatching the import, the SFTP follow-up job moves the remote source file into proceed. If a failed-records CSV is available, it is copied back into the remote failed folder.

Four Import Types

Type	Processing Result
Add	Creates active consumer accounts, reusing an existing consumer profile when the same PII identity already exists.
Add + CFPB	Creates active consumer accounts and makes the completed file available in CFPB Validation Letter workflows.
Update	Finds existing accounts by account number and updates only the selected fields. Pay-term changes can queue updated-offer communications.
Delete	Deactivates matching accounts, cancels scheduled or failed transactions, and queues removed-account communications.

Workflow Boundaries

Import processing owns file validation, consumer matching, account creation or updates, and eligible offer assignment. A valid account is made available in the Consumer Portal before outbound notification work begins. Communications then applies contact permissions, consent, opt-outs, and delivery rules. Negotiation workflows begin only after the consumer responds to an available account or offer.

Account And Offer Delivery

After a row is processed successfully:

The platform creates or reuses the consumer identity associated with the account.
The account is matched using the consumer identity keys, including last name, date of birth, and last four digits of SSN where required.
The highest eligible settlement offer and payment plan offer are assigned.
The account and offers become available in the Consumer Portal.
Notification eligibility is evaluated separately.

The consumer can then review the account, accept an available offer, submit a counteroffer, dispute the account, or submit a no-pay response. Failed rows must not create consumer-visible accounts or offers.

Communication Behavior

The import pipeline does not blindly send messages while rows are still processing. It stores communication work and lets follow-up jobs send or schedule messages after the import work is safe.

Add rows fail before account creation when the uploaded email or mobile is in the creditor company’s opt-out registry. Update rows fail when a selected email or mobile replacement is opted out.
New accounts can trigger new-account or existing-PII-account communication.
Pay-term updates can trigger updated-offer communication.
Deleted accounts can trigger creditor-removed-account communication.
Secure CFPB EcoLetters trigger CFPB delivery communication when the creditor sends those letters.
Email/SMS still depends on feature flags, opt-out state, contact permissions, valid email/phone, and available communication templates.

For provider delivery, open tracking, SMS opt-out replies, and webhook callbacks, see Communication Delivery & Webhooks.

How It Should Work

It should process manual and SFTP files through the same safe queue pipeline.
It should make import progress visible through import history.
It should keep failed row details attached to the import.
It should enforce the same opted-out email and mobile row rejection for manual and SFTP imports.
It should make SFTP imports auditable by marking the history as SFTP.
It should finalize only after all chunks finish.

How It Should Not Work

It should not process all rows in a single browser request.
It should not mix files or failed rows between creditors.
It should not mark an import completed while chunks are still pending or processing.
It should not send communications for rows that failed validation.

​Real-World Example

​Visual Flow

​Shared Processing Rules

​Manual Import Path

​SFTP Import Path

​Four Import Types

​Workflow Boundaries

​Account And Offer Delivery

​Communication Behavior

​How It Should Work

​How It Should Not Work