Tag: BigQuery
Resources about BigQuery collected from the weekly GCP Weekly newsletter — a hand-curated collection of Google Cloud Platform news, technical articles, tutorials, videos and release notes. Each entry below links back to the original GCP Weekly issue and retains its publish date and title, giving you a focused, searchable knowledge base for this topic.
API BigQuery March 24, 2025Leveraging Google Cloud Functions to Create a Powerful Fivetran Connector to integrate custom APIs - Fivetran, a cloud-based ETL service, simplifies data ingestion from various sources, including custom APIs, to your data warehouse. By leveraging Google Cloud Functions as connectors, you can automate data extraction and transformation from your custom API, eliminating manual coding and maintenance.
BigQuery Billing March 24, 2025BigQuery Cost Optimization with slot management - This article covers essential cost-saving techniques, including optimizing slot usage, query performance, and proactive monitoring.
BigQuery Data Analytics Official Blog March 24, 2025Announcing BigQuery repositories: Git-based collaboration in BigQuery Studio - BigQuery repositories, a new feature in BigQuery Studio, enables data teams to collaborate on code stored in Git repositories. It provides a comprehensive set of features to integrate Git workflows directly into the BigQuery environment, including setting up new repositories, connecting to remote Git hosts, editing code, and performing Git operations.
AI BigQuery Data Analytics GCP Experience Official Blog March 24, 2025The AI lens: How Arpeely uses multimodality and BigQuery to revolutionize AdTech - Arpeely uses Google Cloud AI to revolutionize AdTech by leveraging multimodality and BigQuery. Their AI algorithms analyze massive datasets to identify high-value customers, using multimodal AI to analyze web page screenshots and extract visual information. BigQuery helps them crunch numbers, analyze user behavior, and generate insights from over 25 petabytes of compressed data.
BigQuery March 17, 2025A Non-Obvious Partition Pruning Technique in BigQuery - By creating an index table and using partition pruning, it's possible to significantly reduce the amount of data scanned and slots used when retrieving specific transactions by their IDs.
BigQuery Python March 17, 2025Introducing partial ordering mode for BigQuery DataFrames (bigframes) - BigQuery DataFrames introduces a new feature called partial ordering mode, which prevents the creation of a deterministic ordering and sequential index. This optimization allows for faster and more efficient data processing by pushing down row and column filters, especially on large clustered and partitioned tables.
BigQuery Serverless Spark March 17, 2025Spark Stored Procedures in BigQuery: A Practical Guide That Works (Part 1) - A step-by-step practical guide to setting up serverless Spark stored procedures in BigQuery that actually works.
BigQuery March 17, 2025BigQuery data preparation: A Step-by-Step Guide - Learn how to use BigQuery data preparation, an AI-powered visual interface to transform raw data into analysis-ready tables.
BigQuery Generative AI March 17, 2025LangGraph meets Vertex AI: Debug and Evaluate your BigQuery data agent - Tutorial on debugging with LangGraph and Evaluating with Vertex AI Gen AI Model Eval Service.
BigQuery Dataflow March 17, 2025Slowly Changing Dimensions in GCP: A Comprehensive Guide - This article explores how to efficiently implement Slowly Changing Dimensions using BigQuery and Dataflow. We will cover different types of SCDs, including Type 0 (Retain Original Data), Type 1 (Overwrite with Latest Data), Type 2 (Maintain Full History), Type 3 (Store Previous Data in Additional Columns), Type 4 (History in a Separate Table), and Type 6 (Hybrid Approach). We will also discuss how to implement SCD using Dataflow for real-time processing.
BigQuery Databases GCP Experience Official Blog Vertex AI March 17, 202510 months to innovation: Definity's leap to data agility with BigQuery and Vertex AI - Definity Insurance migrated its data platform to Google Cloud's BigQuery and Vertex AI in just ten months, achieving exceptional user satisfaction and significant cost savings. The migration improved performance, increased agility and innovation, and unlocked new analytical possibilities.
AI BigQuery Generative AI March 10, 2025Getting data into BigQuery for building (Gen)AI demos - How can you show the value (Gen)AI has by bridging the gap between theory and practice? Let’s start by getting data into BigQuery.
BigQuery Dataflow Datastream GIS March 10, 2025Migrating POINT Data Type from MySQL to BigQuery using Dataflow - This blog post discusses the challenges of migrating POINT data type from MySQL to BigQuery and presents a solution using Apache Beam and Google Cloud Dataflow.
BigQuery Datastream March 3, 2025Journey to Create a CDC Solution with Google Services: Lessons Learned and Challenges Overcome - Building a Change Data Capture (CDC) solution to move real-time data from MySQL into Google BigQuery using Google services.
BigQuery dbt March 3, 2025Appending Nested Columns in dbt Incremental Updates on BigQuery - Workaround for managing nested columns in bq with dbt incremental updates.
BigQuery Datastream Oracle March 3, 20253 Pro Tips For Datastream - Datastream is a managed service that allows you to replicate data with default out of the box configuration. But you can go beyond!!
BigQuery Data Science Paywall March 3, 2025Four missing Functions in BigQuery - And how to work around them.
BigQuery GCP Experience Official Blog March 3, 2025Transforming data: How Vodafone Italy modernized its data architecture in the cloud - Vodafone Italy has modernized its data architecture by migrating to Google Cloud, leveraging BigQuery and a "clone and shift" approach in collaboration with Google Cloud and Amdocs.
BigQuery Data Analytics GCP Experience Official Blog March 3, 2025African super app Yassir delivers on data with BigQuery migration - Yassir, an African super app, migrated its data infrastructure to Google Cloud's BigQuery to unify data and AI, improving access, scalability, and performance. The migration allowed Yassir to consolidate data processing, deploy new AI models with Vertex AI, and implement centralized data governance controls.
BigQuery Data Science dbt Feb. 24, 2025End-to-end Implementation of an ETL Pipeline from an API with dbt & BigQuery - Building a Weather Data ETL Pipeline with OpenWeatherMap API and dbt.
BigQuery Feb. 24, 2025Batch processing at Scale: How PostgreSQL outperformed BigQuery for us - The strategic decision to migrate from BigQuery to PostgreSQL when doing regular updates on a small amount of rows in the table.
BigQuery Data Analytics Gemini Official Blog Feb. 24, 2025How to use gen AI for better data schema handling, data quality, and data generation - Gemini in BigQuery helps with data engineering tasks by automating schema mapping and transformation, improving data quality by identifying subtle inconsistencies, and generating synthetic and structured data from diverse sources. It uses large language models (LLMs) to perform tasks such as data schema handling, data quality automation, and generation of synthetic and structured data.
BigQuery Feb. 17, 2025SQL is all you need! - Supercharge BigQuery with BigFunctions.
AI BigQuery Machine Learning Feb. 17, 2025Data Harmony: Leveraging Embeddings, Vector Search, and LLMs for MDM - Harness the power of Embeddings and Gemini for Fuzzy Matching in MDM systems.
BigQuery Gemini Feb. 17, 2025Build an Agentic Workflow for your BigQuery data using LangGraph and Gemini - This blog post explores how LangGraph can be used to build agentic workflows for data stored in BigQuery.
BigQuery Dataform Feb. 17, 2025Hands-on Guide to External Tables in BigQuery using Dataform - External tables in BigQuery allow you to query data directly from cloud storage without the need for data migration. By using Dataform to create and manage external tables, you can simplify data management and leverage BigQuery's powerful query engine.
BigQuery Feb. 10, 2025How I Slashed BigQuery Query Time by 69%: A Performance Optimization Journey - Analyzing BigQuery execution graph to identify the Input stage as the bottleneck and discovering significant data skewness.
BigQuery Cloud Marketplace Official Blog Partners Feb. 10, 2025BigQuery datasets now available on Google Cloud Marketplace - Google Cloud Marketplace now offers BigQuery datasets, enabling organizations to access high-quality data for analytics, AI, and business applications. Customers can easily find, buy, and consume datasets from a centralized catalog, simplifying data procurement and reducing administrative burden.
BigQuery Feb. 3, 2025BigQuery “NOT IN” vs “NOT EXISTS” vs “EXCEPT DISTINCT” understand the differences in 3 minutes! - This article explains the difference between operators for excluding data in results like NOT IN, NOT EXISTS, EXCEPT DISTINCT.
BigQuery Dataform Paywall Feb. 3, 2025Export BigQuery Saved Queries programmatically - Hands on tutorial how to export/backup using the Dataform API your BigQuery saved queries.
Showing 421–450 of 2427 resources
Useful Links
Contact
Třebanická 183
Prague, Czech Republic
Phone: +420 777 283 075
Email: [email protected]