Google Cloud BigQuery BI Engine monitoring integration

Monitor your Google Cloud BigQuery BI Engine resources to track in-flight and total request volumes and monitor query latency for accelerated analytics. The BigQuery BI Engine monitor helps you maintain fast, reliable performance for your business intelligence (BI) dashboards and reports.

Benefits

Monitoring BigQuery BI Engine helps ensure that your service remains reliable, secure, and high-performing. Key benefits include the following:

Identify query latency spikes before they impact dashboard and report responsiveness.
Track in-flight and total request volumes to fine-tune BI Engine capacity reservations.
Maintain SLA compliance and reduce downtime with the help of automated alerts and IT automation.
Consolidate BI Engine metrics with those of other GCP services for a single-pane-of-glass view.
Ensure smooth monitoring across multiple BI Engine models with bulk configuration and agent upgrades.

Use cases

Here are some use cases where BigQuery BI Engine monitoring is helpful:

Monitor model request latencies to detect bottlenecks affecting BI dashboards in real time.
Capture in-flight request trends to quickly identify capacity constraints during peak reporting periods.
Visualize request volumes and peak usage periods to plan capacity and optimize scaling.
Set thresholds for request latency to ensure contractual obligations are met.
Track request count growth to strengthen capacity planning for BI workloads.
Integrate BI Engine monitoring with monitoring of other GCP services (like BigQuery, Looker Studio, and Data Studio) for holistic visibility.

Setup and configuration

Adding BigQuery BI Engine while configuring a new Google Cloud monitor

If you have not configured a Google Cloud monitor yet, add one by following the steps below:
1. Go to Cloud > GCP > Add GCP Monitor or Admin > Cloud Monitoring > Google Cloud Platform (GCP).
2. Provide a unique display name for identification purposes.
3. Upload the JSON file that contains the private key of the service account to authenticate Site24x7 and perform resource discovery.
4. Select BigQuery BI Engine from the Select the Resources for Monitoring list.
5. Select existing Notification Profiles, User Alert Groups, Tags, and IT Automation Templates or add new ones. You can also integrate alarms with your preferred third-party service.
6. Click Start GCP Monitoring.
Adding BigQuery BI Engine to an existing Google Cloud monitor

If you already have a Google Cloud monitor configured for the service account, you can add BigQuery BI Engine by following the steps below:
1. Go to Cloud > GCP and select your GCP monitor.
2. Click the hamburger icon next to Service View and select Edit, which brings you to the Edit GCP Monitor page.
3. On the Edit GCP Monitor page, select BigQuery BI Engine from the Select the Resources for Monitoring list and click Save.
4. After successful configuration, go to Cloud > GCP > BigQuery BI Engine. Now you can view the discovered BigQuery BI Engine resources.

It will take approximately five minutes to discover new GCP resources.

Licensing

Each BigQuery BI Engine monitor consumes one basic monitor license.

Polling frequency

The BigQuery BI Engine monitor collects minute-wise metric data, and the statuses of your resources are reported every five minutes.

Supported metrics

Metric name	Description	Statistic	Unit
Model Inflight Requests	The number of BI Engine model requests currently being processed	Average	Count
Model Request Count	The total number of requests processed by the BI Engine model	Total	Count
Model Request Latencies	The latency distribution of requests processed by the BI Engine model	Average	Milliseconds

Threshold configuration

Global configuration
1. In the web client, go to the Admin section on the left navigation pane.
2. Select Configuration Profiles from the left pane and select Threshold and Availability from the drop-down menu.
3. Click Add Threshold Profile in the top-right corner.
4. For the Monitor Type, select BigQuery BI Engine.
5. Specify the threshold values for the required metrics, then click Save.
Monitor-level configuration
1. In the web client, go to Cloud > GCP > BigQuery BI Engine.
2. Select a resource you would like to set a threshold for, then click the hamburger icon.
3. Select Edit, which directs you to the Edit BigQuery BI Engine Monitor page.
4. Specify the threshold values for the required metrics, then click Save.

IT Automation

IT Automation tools automatically resolve performance degradation issues. These tools react to events proactively rather than waiting for manual intervention. The IT Automation tools automate repetitive tasks and automatically remediate threshold breaches. The alarm engine continually evaluates system events for which thresholds are set and executes the mapped automation when there is a breach.

Learn how to configure IT Automation for a monitor.

Configuration Rules

Editing multiple monitors to associate them with different monitor groups or add a different tag can be a tedious process. With Configuration Rules, you can automate the configuration settings of your monitoring resources. Create custom rules to track configuration changes continuously and achieve the ideal configuration settings.

Learn how to add Configuration Rules.

Summary

The Summary tab will give you the performance data organized by time for the metrics listed above. To view the summary:

Go to Cloud > GCP > BigQuery BI Engine.
Select a resource.
Click the Summary tab.

Configuration Details

The Configuration Details tab provides details on the configurations of application instances. To get the configuration details:

Go to Cloud > GCP > BigQuery BI Engine.
Select a resource.
Click the Configuration Details tab.

Reports

Gain in-depth data about the various parameters of your monitored resources and accentuate your service performance using our insightful reports.

To view reports for a BigQuery BI Engine resource:

Go to the Reports section on the left navigation pane.
Select BigQuery BI Engine from the menu on the left.
You can find the Availability Summary Report, Performance Report, and Inventory Report for one selected monitor, or you can get the Summary Report, Availability Summary Report, Health Trend Report, and Performance Report for all the BigQuery BI Engine monitors.

You can also get reports from the Summary tab of the BigQuery BI Engine monitor:

Click the Summary tab.
Get the Availability Summary Report of the monitor by clicking Availability.
You can also find the Performance Report of the monitor by clicking any chart title.