| Ticket | Description | ||||||||||||||||||||||||||||||||||||||
|
Upcoming DirectSQL Update: DENG-3003 |
On March 31st, MANTL will be sunsetting the following BigQuery tables in your DirectSQL projects. These have been migrated to BigQuery views, which have identical contents. The transition to views will increase the data freshness and enable greater flexibility in managing the tables’ contents. These views are now available for use in your DirectSQL projects. With this change, we will also be updating the naming convention to align all nomenclature. A complete list of name changes can be found below. Action required: Please modify your data pipelines to use the new views/naming scheme by March 31, 2024, to ensure you do not see any disruption in your data. NOTE: On Thursday, February 1, we added two new datasets to Direct SQL for our customers who use Apache Spark to automate data workflows: '_materialization_pii' and '_materialization_non_pii'. These datasets are intended to be used for materialization of temporary data, and your existing pii and non-pii service accounts will be granted the permissions necessary to perform that task in their corresponding dataset. Spark users will need to add the following Spark configs to read from BigQuery views: viewsEnabled = true materializationDataset = "_materialization_pii" OR materializationDataset = "_materialization_non_pii" Please let us know if you have any questions or concerns about this change.
Naming Updates
|