Data Export Upgrade

Action required: Between May 16th and June 4th, you will need to update any processes that depend on the files delivered to your Data Export bucket to switch from using the old file names to the new file names. This is necessary in order to ensure that you continue to receive the most up-to-date data.

MANTL will be making several critical upgrades to our Data Export infrastructure that enable Data Export to handle larger volumes of data. This upgrade will include two changes that affect existing data ingestion processes for customers:

New File Names - As part of this upgrade we will be bringing the report names in Data Export into alignment with MANTL’s existing report documentation, wherein reports are divided into Simplified and Raw Reports. The table below provides a map of the old filenames to the new filenames.
Consolidation of Applications Report - The Raw Application reports are currently spread across separate files for consumer/business deposit create account, consumer deposit add secondary owner, business update person, and business add secondary owner. All of these reports contain the same columns and they will be consolidated into one raw_reports__applications file going forward. Different application types in this report will be distinguishable by the `type` field. See the table below for a detailed mapping of old-to-new report names.

We will provide the upgraded reports in parallel over the course of the next month to ensure a sufficient amount of time to update downstream data ingestion processes. Our timeline for changes to data export is as follows:

On Thursday, May 16, 2024, we will
- Begin providing the enhanced reports to your Data Export Report bucket alongside all existing reports.
- Stop updating the “old” reports in your UAT buckets.
On Thursday, June 4, 2024 we will stop updating the old report files. The old report files will remain available in the bucket for the time being, but beginning June 4th, they will cease to be updated on a regular basis and the new-file-name report files will be the source of up-to-date information for each report.

MANTL Data Export - File Name Changes
Old Source	New Source
acquisition.csv	raw_reports__acquisitions.csv
acquisition_enhanced.csv	simplified_reports__acquisitions.csv
agreements_enhanced.csv	simplified_reports__agreements.csv
applications_business_deposit_add_authorized_signer.csv	raw_reports__applications.csv
applications_business_deposit_create_account.csv	raw_reports__applications.csv
applications_business_update_persons.csv	raw_reports__applications.csv
applications_consumer_deposit_add_secondary_owner.csv	raw_reports__applications.csv
applications_consumer_deposit_create_account.csv	raw_reports__applications.csv
attestations_enhanced.csv	simplified_reports__attestations.csv
behavior.csv	raw_reports__behavior.csv
behavior_enhanced.csv	simplified_reports__behavior.csv
business_applications_enhanced.csv	business_applications_enhanced.csv
consumer_applications_enhanced.csv	simplified_reports__consumer_applications.csv
documents.csv	raw_reports__documents.csv
documents_enhanced.csv	simplified_reports__documents.csv
email.csv	raw_reports__email.csv
email_enhanced.csv	simplified_reports__email.csv
funding.csv	raw_reports__funding.csv
funding_transactions_enhanced.csv	simplified_reports__funding.csv
kyc_evaluations.csv	raw_reports__decisioning.csv
kyc_evaluations_enhanced.csv	simplified_reports__decisioning.csv
product_services_enhanced.csv	simplified_reports__product_services.csv
product_services_report.csv	raw_reports__product_services.csv
user_login_report.csv	user_login_report.csv
user_permission_report.csv	user_permission_report.csv

APP-1606

Update to the business self-serve "Review & Sign" page to display an Edit button for the primary signer for all ownership types. This will enable the primary signer to edit person form details. Non-primary signers will see a View button on the Review & Sign page that opens a read-only person form.

Upcoming:

DENG-3482

We are changing the name of a column in Data Export and Console Reports from 'Time to Submit Application (Minutes)' to 'Time to Submit Application in Minutes' in order to remove special characters from column titles that can interfere with some data processing systems and conform with all other reported columns. Specifically, this column will be removed from the Simplified Business Applications Report and the Simplified Consumer Applications Report.

This update will be in UAT 6/11 and Production 6/12

Upcoming:

DENG-3483

Update to delete the Device ID column from Reports. This column, provided by Heap does not capture any data and all records are NULL. We will be removing this empty column from all reports where it was included--specifically the Simplified Acquisitions Report and Simplified Behavior Report.

This update will be in UAT 6/11 and Production 6/12

Upcoming:

DENG-3198

In order to enable users of MANTL's Reports to see when a row in a report was last synced with the MANTL database, we are adding a column to every report called 'MANTL Last Synced At'. This column will show the timestamp of the time at which the row was added to the report or at which the row was last checked for any updates in the MANTL system.

In Data Export and Console Reports, this column will be provided as a string value in the downloaded CSV, formatted in EST time. For example: '2024-01-01 12:01:01'
In Direct SQL, the new column will be included in all reports as a BigQuery TIMESTAMP data type. It will not be added to Raw Views in Direct SQL, which are realtime.

This update will be in UAT 6/11 and Production 6/12

Upcoming:

DENG-3515

Update to Direct SQL Reports to match existing fields already available in Console Reports:
Simplified Decisioning Report:

Added Qualifle Account Action Info (qualifile_account_action_info) and Primary Address Country (primary_address_country)

This update will be in UAT 6/11 and Production 6/12

Change Log

Data Export Upgrade

Change Log

Related articles