Databricks Hints Filter

The foglamp-filter-databricks-hints plugin enables you to provide SQL hints for the Databricks Hints plugin, optimizing how data is processed and stored in Databricks.

Overview

When you add this filter, a configuration page appears as shown below:

Key Features

• Enable/Disable the Filter: Easily toggle the filter on or off as needed.

Configuration Options

The Databricks Hints configuration requires specifying an asset to which the filter will be applied:

• Asset Name:

  The name of the asset to which the SQL hint will be applied. This should match the asset name used in your data pipeline.

• Table Name:

  The name of the table to be created in Databricks for storing data related to the specified asset. Ensure the table name follows your organization’s naming conventions and does not conflict with existing tables.

Usage Notes

• Make sure the asset and table names are unique and descriptive.

• Review your Databricks account permissions to ensure the filter can create and write to the specified table.

• Refer to your organization’s data governance policies when naming assets and tables.

For more details, refer to the official documentation or contact your system administrator.