Tag: BigQuery

Resources about BigQuery collected from the weekly GCP Weekly newsletter — a hand-curated collection of Google Cloud Platform news, technical articles, tutorials, videos and release notes. Each entry below links back to the original GCP Weekly issue and retains its publish date and title, giving you a focused, searchable knowledge base for this topic.

BigQuery Gemini Terraform July 6, 2026

Gemini Enterprise: Building a Periodic BigQuery Search Agent with Terraform - In the era of Generative AI, Large Language Models are only as smart as the enterprise data you give them.

BigQuery FinOps July 6, 2026

BigQuery Fluid Scaling: the numbers are in! - Google Cloud's new BigQuery Fluid Scaling dramatically improves reservation billing by eliminating the previous 60-second minimum, shifting to precise per-second consumption. This change significantly reduces "cooldown waste," making reservations much more cost-effective and suitable for bursty workloads, thus simplifying cost analysis and reinforcing BigQuery's serverless capabilities.

Agents BigQuery July 6, 2026

Conversational analytics in BigQuery brings trusted agentic reasoning to everyone - Use Conversational Analytics in BigQuery to query data, run analyses, and generate reports using natural language on data in BigQuery and Lakehouse.

BigQuery Networking July 6, 2026

Scaling Network Analysis for Fraud Prevention with BigQuery Graph - Data analysis at scale to identify hidden fraud and achieve transaction savings.

AI BigQuery July 6, 2026

Synthesize the big picture and analyze trends with BigQuery's AI.AGG function - Learn to use BigQuery’s new AI.AGG function to analyze unstructured data such as logs and documents at scale.

BigQuery Official Blog Python June 29, 2026

Boost BigQuery with Python: Managed Python UDFs now generally available - Managed Python User-Defined Functions (UDFs) in BigQuery let you define and execute Python scalar functions directly within your BigQuery SQL queries.

BigQuery Dataform June 29, 2026

How to solve Environment Promotion and Isolation in Dataform - This article details how to overcome challenges in Dataform CI/CD by moving beyond "hope-based" deployments and continuous compilation. It proposes leveraging the Dataform API for programmatic compilation and version pinning. Two strategies are presented: Git Tag Promotion for simplicity, and using an External State Store for enhanced audit trails and metadata, both enabling robust environment isolation.

Agents BigQuery Data Science June 29, 2026

Architectural Trade-offs in Building Data Foundations for Agentic AI - This article examines the crucial architectural trade-offs when designing data foundations for agentic AI, distinguishing between operational and analytical datastores. It highlights that operational databases, like AlloyDB, are best for real-time, low-latency user interactions requiring immediate consistency, while analytical data warehouses, such as BigQuery, excel at high-throughput historical data scanning for strategic insights. The appropriate choice depends on whether the AI agent supports a user experience loop or an asynchronous analytical process, with critical implications for performance and data consistency.

BigQuery Dataform June 29, 2026

Dataform Incremental Load Optimizations - Four practical techniques that dramatically reduced BigQuery costs and execution time for production Dataform incremental pipelines.

BigQuery June 29, 2026

Cut Your BigQuery Bill in Half Without Rewriting a Single Query - On-demand vs. slots, partitioning, clustering, and the one preview button that stops $60 mistakes before they run.

BigQuery Paywall June 29, 2026

Managing BigQuery with Google ADK, MCP, Cloud Run, Streamlit, and OIDC Authentication - Building a production-ready conversational analytics platform with Google ADK, BigQuery MCP, Streamlit, Cloud Run, and OIDC authentication.

BigQuery Cloud Firestore Python June 29, 2026

Bridging the OLAP/OLTP Divide: Container-First Data Syncing with Python and Kestra - This article explores a container-first strategy for efficiently synchronizing data between OLAP (BigQuery) and OLTP (Firestore) databases. It details building a stateless bridge with Python and Kestra in Kubernetes, where change detection is offloaded to the data warehouse using `row_hash` for optimal performance. This method offers a simpler and scalable alternative to traditional reverse ETL tools for keeping live applications updated.

AI BigQuery Data Analytics June 29, 2026

From What to Why: Unlocking Key Driver Analysis in BigQuery with AI.KEY_DRIVERS - BigQuery has launched AI.KEY_DRIVERS in Public Preview.

AI BigQuery Data Analytics June 29, 2026

The Semantic Brain & Operational Muscle: Solving the Enterprise AI Context Deficit - With Google Cloud Knowledge Catalog and Gemini Enterprise.

Agents BigQuery Data Agent Kit June 22, 2026

What’s new in data agents: Supercharging your AI workflows - Learn about new data agents and tools for business analysts, data scientists, and database admins to integrate with the Agentic Data Cloud.

Agents BigQuery GCP Experience GKE June 22, 2026

Architecting a trusted agentic platform with graph technologies: A Yahoo case study - Using Google graph technologies, Yahoo's Seller Agent platform is a model for other industries striving to build a trusted agentic system of action.

Apache Iceberg BigQuery dbt June 22, 2026

Building a Serverless, Multi-Source Data Ingestion Framework on GCP (Snowflake & Databricks to BigQuery) - This article details a serverless, multi-source data ingestion framework developed on Google Cloud Platform, designed to consolidate data from Snowflake and Databricks into BigQuery. Leveraging GCP Workflows for orchestration and Dataproc Serverless for compute, this solution significantly reduces costs and operational overhead by eliminating persistent clusters.

AI BigQuery FinOps June 22, 2026

How AI made a rejected BigQuery cost optimisation worth shipping (loveholidays AI diary #8) - Loveholidays successfully implemented a BigQuery cost optimization strategy that was previously deemed too costly to pursue, by leveraging AI coding assistants. This AI-powered solution automates the intelligent switching between BigQuery's logical and physical storage billing models, leading to significant projected savings. The initiative demonstrates how a robust AI agent environment can enable the delivery of projects that were once considered unfeasible.

BigQuery June 22, 2026

The Hidden Powerhouse: Demystifying the BigQuery Storage Write API - The BigQuery Storage Write API is a powerful, gRPC-based engine that enables highly optimized and efficient streaming of data into BigQuery, often used implicitly by various Google Cloud services. This API significantly improves data ingestion performance over previous methods by leveraging Protocol Buffers and bidirectional streaming, offering flexible write modes to suit real-time, transactional, or buffered data requirements.

Airflow BigQuery Cloud Storage June 22, 2026

Backfilling 30 Million Files from GCS to BigQuery - This article details the complex journey of backfilling 30 million files from Google Cloud Storage into BigQuery, highlighting challenges often overlooked in data engineering. It describes the iterative process of building a robust pipeline, overcoming issues like API rate limits, data corruption, and system interruptions through strategies like idempotency and separating file discovery from ingestion.

BigQuery June 22, 2026

Optimizing BigQuery performance for transactional use cases: Fine-grained DML - This article demonstrates how Google Cloud's BigQuery leverages its Fine-grained DML feature to optimize transactional use cases, such as real-time updates and deletions. Through comparative testing, it shows significant improvements in execution speed and reduced compute costs compared to traditional DML methods, by separating logical and physical data changes. This enhancement is crucial for efficiently handling high-volume, dynamic data operations within cloud-native data platforms.

Big Data BigQuery LLM Machine Learning June 15, 2026

You Probably Don’t Need a Vector Database - If Your Data Already Lives in BigQuery - How I built an end-to-end RAG proof-of-concept in pure SQL: no new infrastructure, no data movement, and pay-per-query cost.

BigQuery June 15, 2026

Optimizing Data Vault in BigQuery: Why Physical Clustering Matters More Than Hash-Based Keys - Traditional Data Vault designs, heavily reliant on hash-based business keys, create performance inefficiencies in BigQuery because their random distribution prevents effective physical data pruning. BigQuery's clustering mechanism requires structured data locality to efficiently skip storage blocks, a capability undermined by high-entropy hash keys.

BigQuery FinOps June 15, 2026

A practitioners view on BigQuery Workload Management - This guide offers a practitioner's perspective on optimizing BigQuery workload management for performance and cost efficiency. It covers critical techniques across storage, query, compute, billing, and caching, preparing data platforms for the demands of AI agentic workflows. Mastering these strategies is key to ensuring BigQuery deployments scale effectively and cost-efficiently.

BigQuery Data Agent Kit June 15, 2026

Data Agent Kit - I Explored GCS, Visualized Data, and Built a Pipeline Without Leaving My Editor - A hands-on look at Google’s new Data Agent Kit, from a data engineer’s chair.

BigQuery Data Analytics Official Blog June 15, 2026

Introducing the Open Knowledge Format - Learn how the Open Knowledge Format helps secure data sharing and improves collaboration across teams with standardized documentation.

Airflow BigQuery Paywall June 8, 2026

Designing a Medallion Architecture Pipeline for Spotify Listening Analytics - From Spotify API ingestion to BigQuery transformations and genre-level aggregation with Airflow orchestration.

BigQuery FinOps June 8, 2026

The $4,000 COALESCE: When a BigQuery Date Filter Quietly Does Nothing - How one CTE turned every query against a view into a 1.21 TB scan — and the rewrite that cut it to 46 GB.

AI BigQuery Data Science June 8, 2026

The “Store of Tomorrow” Demands a Knowledge Catalog - Scaling Agentic AI in Retail — The FairPrice Group Blueprint.

BigQuery Data Analytics Databases Official Blog June 8, 2026

Modeling a digital twin of a food supply chain using BigQuery Graph - BigQuery Graph allows you to build a digital twin of your entire supply chain by turning your physical world—items, recipes, and locations—into a searchable map of nodes and edges, providing a new level of clarity.

Showing 91–120 of 2427 resources

 

Latest Issues




Contact

Zdenko Hrček
Třebanická 183
Prague, Czech Republic
Phone: +420 777 283 075
Email: [email protected]