Real-World Example
ABC Collections can import consumers in two ways: an employee manually uploads a CSV in the portal, or the company drops a CSV into its SFTP folder. Once the file is accepted, both paths use the same background processing pipeline. That shared pipeline stores the file, creates import history, splits the work into chunks, processes rows safely, records failed rows, finalizes counts, and triggers the correct consumer communications.Visual Flow
Shared Processing Rules
Manual Import Path
Manual import starts from Import File. The creditor selects a mapped Header Profile, chooses Add, Add + CFPB, Update, or Delete, uploads a CSV, and the app validates the file headers before storing the file. A manual upload needs at least one nonblank account row below the headers. If a report export contains only headers or blank rows, add an account row and upload it again. The portal shows the error before creating an Import History entry. Manual imports calculate total records before the coordinator creates chunks.SFTP Import Path
SFTP import starts from SFTP Connections. The scheduled command scans active mapped SFTP header profiles twice daily. When it finds CSV files in the expected folders, it copies each file intoimport_consumers, creates SFTP import history, and dispatches the same coordinator job used by manual uploads.
SFTP uses the same configured size limit as manual imports. It checks the remote file size before opening the CSV. Files over the limit or with an unavailable size stay in the source folder and receive a warning in the import logs. Other files can still import. Split an oversized file into smaller CSVs before retrying.
After processing finishes, the SFTP follow-up job moves the remote source file into proceed and copies any failed-records CSV into the remote failed folder.
If payment or required setup changes during an import
YouNegotiate checks required setup before applying each batch of rows. It also checks membership payment for manual Add and Add + CFPB files and for SFTP imports. If a check fails, the import keeps the rows it already saved and lists the remaining rows in the failed CSV with the reason. For example, say ABC Collections uploads 100 new accounts. After YouNegotiate saves 50, the membership payment is returned. The first 50 accounts stay in place, and the other 50 appear in the failed CSV. Once you resolve the payment, upload just those 50 rows. You can download the CSV from Import History or retrieve it from the SFTPfailed folder.
Four Import Types
Workflow Boundaries
Import processing owns file validation, consumer matching, account creation or updates, and eligible offer assignment. A valid account is made available in the Consumer Portal before outbound notification work begins. Communications then applies contact permissions, consent, opt-outs, and delivery rules. Negotiation workflows begin only after the consumer responds to an available account or offer.Account And Offer Delivery
After a row is processed successfully:- The platform creates or reuses the consumer identity associated with the account.
- The account is matched using the consumer identity keys, including last name, date of birth, and last four digits of SSN where required.
- The highest eligible settlement offer and payment plan offer are assigned.
- The account and offers become available in the Consumer Portal.
- Notification eligibility is evaluated separately.
Communication Behavior
The import pipeline does not blindly send messages while rows are still processing. It stores communication work and lets follow-up jobs send or schedule messages after the import work is safe.- For Add and Add + CFPB, a valid account reaches New Offers even when the matching ecoAddress has opted out of an uploaded contact. The row counts as imported and creates no immediate or scheduled consumer email or SMS. The opt-out applies only to the matching ecoAddress.
- A contact restriction from a qualifying SendGrid or Twilio failure is checked when sending. The account still imports, and another eligible contact can still receive a notification.
- Update rows fail when a selected email or mobile replacement is opted out.
- New accounts can trigger new-account or existing-PII-account communication.
- Pay-term updates can trigger updated-offer communication.
- Deleted accounts can trigger creditor-removed-account communication.
- Secure CFPB EcoLetters trigger CFPB delivery communication when the creditor sends those letters.
- Email/SMS still depends on feature flags, opt-out state, contact permissions, valid email/phone, and available communication templates.
How It Should Work
- It should process manual and SFTP files through the same safe queue pipeline.
- It should make import progress visible through import history.
- It should keep failed row details attached to the import.
- Manual and SFTP imports should use the same contact opt-out rules for account delivery and notifications.
- It should make SFTP imports auditable by marking the history as SFTP.
- It should finalize only after all chunks finish.
How It Should Not Work
- It should not process all rows in a single browser request.
- It should not mix files or failed rows between creditors.
- It should not mark an import completed while chunks are still pending or processing.
- It should not send communications for rows that failed validation.

