Tag: BigQuery

BigQuery Dec. 10, 2018

Snowplow for Google Cloud Platform is here - Snowplow Analytics supports loading data into BigQuery.

BigQuery Google Cloud Dataflow Google Cloud IoT Dec. 3, 2018

A solution for implementing industrial predictive maintenance: Part III - A full predictive maintenance reference solution from Google Cloud Platform products, including Cloud IoT Core and Cloud IoT Edge, big data and data processing tools like BigQuery and Cloud Dataflow, and machine learning platforms like Cloud ML Engine.

BigQuery Python Dec. 3, 2018

Loading JSON formatted strings into Google BigQuery (Python3) - Loading JSON into Google BigQuery using Python.

BigQuery Dec. 3, 2018

Import JSON into BigQuery with Google Cloud Functions - Using Google Cloud Functions to Import JSON into BigQuery.

BigQuery GCP Experience Dec. 3, 2018

Investing in Pandora’s Core Differentiators - Pandora's (music streaming service) experience with migration from on premise Hadoop to BigQuery.

BigQuery Nov. 26, 2018

Plotting Ethereum Address Growth Chart in BigQuery - Solution to plot ethereum address growth chart in BigQuery.

BigQuery Machine Learning TensorFlow Nov. 26, 2018

GCP Machine Learning Face-off: BigQuery ML -vs- Custom Estimators in TensorFlow - Article compares two different Machine Learning solutions on Google Cloud.

BigQuery Nov. 26, 2018

Using BigQuery as a data warehouse in B2B startup - Video recorded at DevFest Ukraine 2018.

BigQuery Machine Learning Nov. 26, 2018

Machine learning on machines: building a model to evaluate CPU performance - Training machine learning model to predict machine performance based on hardware and software specifications.

BigQuery Nov. 19, 2018

How to Query Balances for all Ethereum Addresses in BigQuery - Beta availability of transaction traces in GCP's public Ethereum dataset in BigQuery.

BigQuery Nov. 19, 2018

Data pipelines for everyone with CRMint - Automate BigQuery SQL queries with this elegant GCP pipeline application.

BigQuery Official Blog Nov. 5, 2018

Modern data warehousing with BigQuery: a Q&A with Engineering Director Jordan Tigani - Engineering Director Jordan Tigani to talk about the evolution of data warehousing, the current technology landscape, and how BigQuery fits in.

BigQuery Google Cloud Dataflow Oct. 29, 2018

How to transfer BigQuery tables between locations with Cloud Dataflow - Article explains process (with code sample) of copying data in BigQuery from one region to another

BigQuery Google Cloud Dataflow Oct. 29, 2018

Analyzing the Game of Baseball on GCP - Series of articles describing baseball data analysis using producs on Google Cloud Platform

BigQuery Business Oct. 22, 2018

Google's BigQuery Brings GIS Into The Petascale Era - How BigQuery is making difference for big data analysis.

BigQuery GCP Experience Official Blog Oct. 15, 2018

How Traveloka built a Data Provisioning API on a BigQuery-based microservice architecture - Building a Data Provisioning API on a BigQuery-based microservice architecture.

BigQuery Official Blog Oct. 15, 2018

BigQuery arrives in the London region, with more regions to come - Rollout of new regions for BigQuery starting with London, with more regions to follow in the coming months.

BigQuery Machine Learning Official Blog Oct. 15, 2018

What’s happening in BigQuery: a new ingest format, data type updates, ML, and query scheduling - New features in beta, including query scheduling, new BigQuery ML models and functions, and geospatial types and queries.

BigQuery Oct. 15, 2018

Benchmarking Google BigQuery at Scale - Benchmarking against BigQuery and migrating Data Warehouse to BigQuery on Google Cloud.

Beginner BigQuery Oct. 15, 2018

How BigQuery GIS scales up your geospatial projects - Overview of BigQuery GIS

BigQuery Oct. 8, 2018

BigQuery ML + Tableau - Using BigQuery ML and Tableau for dynamic model predictions.

BigQuery Oct. 8, 2018

Identifying Spammy ERC20 Transfers - How to identify Spammy ERC20 Transfers using BigQuery dataset.

BigQuery Cloud Composer Official Blog Oct. 8, 2018

How to transfer BigQuery tables between locations with Cloud Composer - How to create and run an Apache Airflow workflow in Cloud Composer.

BigQuery Official Blog Oct. 8, 2018

BigQuery and surrogate keys: a practical approach - Importance of surrogate keys and ways to define them.

BigQuery Official Blog Python Sept. 24, 2018

Ibis and BigQuery: scalable analytics with the comfort of Python - How Ibis works with BigQuery to provide an elegant and flexible Python interface for composing SQL queries.

Apache Beam BigQuery Google Cloud Dataflow Sept. 24, 2018

Micro-batching with Apache Beam and BigQuery - Explore option for overcoming BigQuery limit whilst still being able to import your data in a timely fashion.

BigQuery Google Kubernetes Engine Sept. 24, 2018

Data Lake API on Microservice Architecture using BigQuery - How Traveloka created solution to deliver data for business (Data Lake API) using GCP.

BigQuery Cloud Composer Sept. 17, 2018

Ethereum in BigQuery: how we built this dataset - Overview of how architecture behind BigQuery Ethereum Public Dataset is designed.

BigQuery Sept. 17, 2018

BigQuery: level up your queries with these advanced tricks - Exploring some of BigQuery features using public dataset for Metropolitan Museum of Art of New York.

BigQuery Google Cloud Functions Google Cloud Storage Tutorial Sept. 17, 2018

Loading Data Into BigQuery With Cloud Functions - Setting up Cloud Functions to upload file to BigQuery when file is uploaded to Cloud Storage.

BigQuery Sept. 17, 2018

Introduction to BigQuery Geographic Information Systems (GIS) - BigQuery GIS allows you to analyze and visualize geospatial data in BigQuery by using geography data types and standard SQL geography functions.

BigQuery GCP Experience Sept. 17, 2018

Moving to BigQuery — Data at our Fingertips - Description of migrating to BigQuery for hike.in (chat app).

Advanced BigQuery Data Studio Security Sept. 10, 2018

Share Data with Confidence: Cell-level Access Controls in BigQuery and Data Studio - Cell-level Access Controls in BigQuery and Data Studio.

BigQuery Google Cloud Storage Sept. 10, 2018

Nerdy McJumpyface — analyzing the jump shot - Flow of that raw data from Google Cloud Storage into a more manageable one-entry-per-shot state in BigQuery.

BigQuery Machine Learning Sept. 10, 2018

Prototyping a ML Model for Employee Turnover with Google BigQuery ML - Using Google BigQuery ML and Kaggle dataset for HR turnover data.

BigQuery Official Blog Sept. 3, 2018

Using BigQuery ML and BigQuery GIS together to predict NYC taxi trip cost - Building a machine learning model using BigQuery ML and use BigQuery’s support for spatial functions.

BigQuery Sept. 3, 2018

Ethereum in BigQuery: a Public Dataset for smart contract analytics - Ethereum dataset (transaction history) is available in BigQuery.

BigQuery Sept. 3, 2018

Date Partitioning and Table Clustering in Google BigQuery (and Looker PDTs) - Example on Date Partitioning and Table Clustering in Google BigQuery.

BigQuery Sept. 3, 2018

Using Google Datalab and BigQuery for Image Classification comparison - Using Google Datalab and BigQuery for image classification project.

BigQuery Sept. 1, 2018

Programmatic Patent Searches Using Google’s BigQuery & Public Patent Data - Patent Searches Using Google’s BigQuery & Public Patent Data.

BigQuery Aug. 27, 2018

Building Datawarehouse on Google Cloud Bigquery - Building Datawarehouse on Google Cloud Bigquery.

BigQuery Official Blog Aug. 20, 2018

What’s happening in BigQuery: integrated machine learning, maps, and more - New features that make BigQuery more performant and more cost effective.

BigQuery Aug. 20, 2018

How to do online prediction with BigQuery ML - How to pull out the necessary weights and scaling parameters from the training output tables and compute the prediction with BigQuery ML.

BigQuery Machine Learning Aug. 20, 2018

Predicting Taxi Fare Price With BigQuery ML - How BigQuery ML can be used to build liner regression model for taxi fare predictions with training 55M rows of data in 30 minutes.

BigQuery Google Cloud Dataflow Google Cloud Pub/Sub Python Aug. 13, 2018

Aggregated Audit Logging With Google Cloud and Python - Taking Apache2 server access logs from a web server, converting the log file line-by-line to JSON data, publishing that JSON data to a Google PubSub topic, transforming the data using Google DataFlow, and storing the resulting log file in Google BigQuery long-term storage.

BigQuery Aug. 13, 2018

Optimizing BigQuery: Cluster your tables - Clustering of tables in BigQuery explained in Wikipedia data set.

BigQuery Aug. 6, 2018

Configure BigQuery ODBC Driver On Linux - How to configure BigQuery ODBC driver on Linux.

BigQuery Aug. 6, 2018

Tips for using BigQuery in the enterprise - Tips for using BigQuery in the enterprise.

BigQuery Google Cloud Platform Python Aug. 6, 2018

Aggregated Audit Logging with Google Cloud and Python - Aggregating all of logs into Google BigQuery Audit Logs.

BigQuery Aug. 6, 2018

Performing large-scale mutations in BigQuery - Article explains why DML statement quotas are in place and goes through some approaches for accomplishing large scale mutations without going over DML quotas.

BigQuery Google Cloud Spanner Google Cloud SQL Official Blog July 30, 2018

On GCP, your database your way - New database features to get the most out of your databases for your business.

BigQuery Data Studio Official Blog July 30, 2018

Bridging the gap between data and insights - Number of updates for BigQuery , Data Studio Explorer, Cloud Composer and Dataproc that will make data analytics easier.

BigQuery Machine Learning July 30, 2018

Machine Learning in your Database? The Case For and Against BigQuery ML - Pros and Cons of BigQuery ML.

BigQuery GCP Experience July 30, 2018

How Buzzfeed democratised content analytics using BigQuery and Looker - How publisher Buzzfeed has rebuilt its analytics processes around BigQuery data warehouse and Looker dashboards for consumption by reporters and editors.

BigQuery Java July 23, 2018

Streaming data directly into BigQuery - Cost effective ways to streaming data directly into BigQuery.

Beginner BigQuery July 23, 2018

Google BigQuery — List all partitions in a partitioned table - Queries to list all partitions in a partitioned table in Google BigQuery.

BigQuery GCP Experience July 23, 2018

Streamlining predictive analytics in retail marketing - Online flash-sale retailer Zulily uses BigQuery and Tableau to help power its predictive analytics, which, in turn, boosts its marketing efforts and ability to manage incoming data.

BigQuery July 23, 2018

BigQuery Updates - BigQuery has (finally) new UI.

Apache Beam BigQuery Google Cloud Dataflow Official Blog July 16, 2018

Measuring patent claim breadth using Google Patents Public Datasets - Analysing Patent public dataset and building machine learning model using GCP products.

BigQuery July 9, 2018

Resolving Big Data Challenges: Why We Use BigQuery at Outfit7 - How gaming company Outfit7 is using BigQuery

BigQuery July 9, 2018

From BigQuery to Features - A look at under the hood of FIFA World Cup predictions.

BigQuery Data Studio July 2, 2018

The 2018 World Cup Visualized: All the Goals So Far - Visualization of goals from World Cup with BigQuery and Data Studio.

BigQuery July 2, 2018

BigQuery in June: a new data type, new data import formats, and finer cost controls - Monthly summary of important news: new data type, data format, enhancing DML.

BigQuery Python July 2, 2018

Google Search Console Data into Google BigQuery - Adapting an existing Python script to pull bulk data from Google Search Console into Google BigQuery.

BigQuery Cloud Datalab Google Cloud Dataprep July 2, 2018

Building a Personal Genome Data Warehouse with Google Cloud, 23andMe and Family Tree DNA - Using Cloud Dataprep to clenup DNA raw data and then import to BigQuery and analyzing in Cloud Datalab.

BigQuery Google Cloud Pub/Sub Google Compute Engine June 25, 2018

Infrastructure as Code with Terraform for Google Cloud - Using terraform for creating pubsub, compute engine and big-query resources in Google cloud.

BigQuery June 25, 2018

Visualising on-street parking in Melbourne with open data - Using BigQuery to prepare open data for visualization.

BigQuery Business GCP Experience June 18, 2018

Why Migrating your Data Warehouse to Google Big Query can be the Next Big Thing for your Enterprise? - What BigQuery is and examples of companies which have migrated their data to BigQuery.

BigQuery June 18, 2018

Manual partition tables in BigQuery - Partitions strategies in BigQuery.

BigQuery Dataflow Machine Learning June 18, 2018

Making World Cup Sausage with Cloud Dataflow and BigQuery - Making World Cup predictions with Cloud Dataflow and BigQuery.

BigQuery June 18, 2018

BigQuery Public Datasets - Why BigQuery public datasets are awesome.

BigQuery Google Cloud Dataflow Google Cloud Pub/Sub June 11, 2018

Serverless and realtime Data Analytics for a retailer on GCP - GCP customer journey from scale issues to serverless and from once a day refreshed dashboards to realtime analytics.

BigQuery Firebase June 11, 2018

Making sense of Google Analytics for Firebase BQ data - Querying data from Firebase in BigQuery.

BigQuery Machine Learning June 11, 2018

Explaining the joke: “Half the time when companies say they need ‘AI’, what they really need is a SELECT clause with a GROUP BY” - Deeper look at the claim.

BigQuery June 4, 2018

How to enable Pandas to access BigQuery from a service account - Using Pandas to run a query against BigQuery.

BigQuery Google Cloud Dataflow Kubernetes June 4, 2018

Say goodbye to Mixpanel. Meet Banias! - Banias is serverless event analytics pipeline based on Kubernetes, Apache Beam and Google BigQuery.

BigQuery Google Cloud Dataflow Google Cloud Pub/Sub June 4, 2018

Realtime Streaming Data Pipeline using Google Cloud Platform and Bokeh - Build a real-time streaming data pipeline and a simple dashboard to visualize the streaming data.

BigQuery Google Compute Engine May 14, 2018

Automatically Storing Tweets in BigQuery using Google’s Compute Engine and Twitter’s Search API - Storing Tweets in BigQuery using GCE and Twitter’s Search API.

BigQuery May 14, 2018

3 Step Setup Guide To Activating A Google BigQuery Data Warehouse Destination - Simple and easy setup guide for activating Google BigQuery within your Openbridge account.

BigQuery NoSQL Official Blog Stackdriver May 14, 2018

Transform publicly available BigQuery data and Stackdriver logs into graph databases with Neo4j - GCP announced availability of Neo4j Enterprise VM solution and Test Drive on Google Cloud. Launcher.

AWS BigQuery May 7, 2018

FileZilla: 3 Simple Steps To Load Data to Google BigQuery, Amazon Redshift, Amazon Redshift Spectrum or Amazon Athena - Loading CSV files to Google BigQuery and AWS warehouses using FileZilla.

BigQuery May 7, 2018

The Missing Guide To Setting Up Google Cloud Service Accounts For Google BigQuery - Setting Up Google Cloud Service Accounts For Google BigQuery.

BigQuery Official Blog April 30, 2018

Accessing external (federated) data sources with BigQuery’s data access layer - Description of how to use data access layer in BigQuery without moving your data to a BigQuery dataset.

BigQuery Business April 30, 2018

Google’s BigQuery Gaining Steam As Cloud Warehouse Wars Heat Up - Overview of BigQuery in context of other cloud data warehouses.

BigQuery April 23, 2018

BigQuery arrives in the Tokyo region - Roll out for regional availability for BigQuery in Google Cloud’s Tokyo region.

BigQuery Dataflow GCP Experience Google Cloud Dataflow April 23, 2018

Traveloka’s journey to stream analytics on Google Cloud Platform - Traveloka recently migrated streaming data processing pipeline from a legacy architecture to a multi-cloud solution that includes the Google Cloud Platform (GCP) data analytics platform.

BigQuery April 23, 2018

BigQuery lazy data loading: DDL, DML, partitions, and half a trillion Wikipedia pageviews - Analyze a 4 terabyte dataset of Wikipedia pageviews with BigQuery.

Beginner BigQuery April 23, 2018

What BigQuery is and isn’t? - Explore basics of BigQuery.

BigQuery Data Studio April 23, 2018

Harnessing the power of Phish.AI, PhishTank, Google Big Query and Google Data Studio to analyze Phishing Trends - Using Big Query and Google Data Studio to analyze Phishing Trends from PhishTank which is community website collecting phishing data.

BigQuery Official Blog April 16, 2018

BigQuery lazy data loading: SQL data languages (DDL and DML), partitions, and half a trillion Wikipedia pageviews - Analyze a 4 terabyte dataset of Wikipedia pageviews with BigQuery.

BigQuery Data Studio April 16, 2018

BigQuery and Data Studio cost optimization - Cost optimization using Data Studio on BigQuery.

BigQuery April 16, 2018

BigQuery beyond SQL and JS: Running C and Rust code at scale - BigQuery running arbitrary C and Rust code.

BigQuery Cloud Datalab TensorFlow April 16, 2018

Predicting San Francisco Bikeshare availability with TensorFlow and LSTMs - Using Google Datalab, BigQuery, and TensorFlow to perform machine learning.

BigQuery April 16, 2018

Introducing column based partitioning in BigQuery - Explanation of column date partitioning in BigQuery.

BigQuery Dataflow April 9, 2018

Give meaning to 100 billion analytics events a day - Orchestrate Kafka, Dataflow and BigQuery together to ingest and transform a large stream of events.

BigQuery Google Cloud Dataflow Google Cloud Dataprep April 9, 2018

Oracle data to Google BigQuery using Google Cloud Dataflow and Dataprep - Load gigabytes or terabytes of data from Oracle into BigQuery using Google Cloud Dataflow and Dataprep relatively easy and very efficiently.

BigQuery Cloud Natural Language API Java April 9, 2018

Analyzing Twitter Location Data with Heron, Machine Learning, Google’s NLP, and BigQuery - Using different products from Google Cloud Platform to get insights from Twitter data.

BigQuery Dataflow Official Blog April 2, 2018

How Tokopedia modernized its data warehouse and analytics processes with BigQuery and Cloud Dataflow - Tokopedia is leading online marketplace in Indonesia, the article explores their modernization journey of data warehouse and analytics processes with BigQuery and Cloud Dataflow.

BigQuery Dataflow Google Cloud Spanner Google Kubernetes Engine Official Blog April 2, 2018

Architecting live NCAA predictions: from archives to insights - Article explores architecting NCAA real-time predictions, achieved through a few months of data ingestion, ETL, analysis, and modeling.

BigQuery Cloud Natural Language API April 2, 2018

Sentiment analysis with BigQuery and Google Cloud Natural Language - Making sense of community sentiment using Google Cloud’s Google Cloud Natural Language.

BigQuery Python April 2, 2018

Getting Started with Google BigQuery in Python — Part 1: Setting Up the Dev Environment - Setting Up the Dev Environment for using BigQuery in Python

BigQuery March 26, 2018

Deficit Reduction - Analysis of NCAA public dataset using BigQuery.

BigQuery GCP Experience March 19, 2018

The switch to self-service marketing analytics at zulily: best practices for using Tableau with BigQuery - Zulily is e-commerce company and shares experience of using Tableau with BigQuery.

BigQuery Machine Learning March 19, 2018

Deep Neural Network implemented in pure SQL over BigQuery - Fun project to test out the limits of SQL and BigQuery and to look at neural network training from a declarative data transformation perspective.

BigQuery GCP Experience Official Blog March 12, 2018

How Color uses the new Variant Transforms tool for breakthrough clinical data science with BigQuery - GCP Customer Story: Use of new Variant Transforms tool for breakthrough clinical data science with BigQuery.

BigQuery March 12, 2018

Leaf #0 — Top 13 Techniques for Google BigQuery Optimization - BigQuery best practices.

BigQuery GCP Experience March 12, 2018

TeamSnap: Supercharging digital marketing with Google BigQuery and Tableau - TeamSnap began using Google Analytics 360, which integrates with Google BigQuery, to provide a fully managed big data analysis service. TeamSnap analyzes the data using Tableau, which connects directly to Google BigQuery for fast analytics and helps the company share and collaborate on that information with self-service ease.

BigQuery March 12, 2018

Statistics for NCAA men’s and women's basketball back to 1894 - Fans of basketball can play with different stats about teams, games etc. in BigQuery.

BigQuery March 12, 2018

Loading Parquet Data from Google Cloud Storage to BigQeury - Parquet is an open source column-oriented data format that is widely used in the Apache Hadoop ecosystem.

BigQuery Google Cloud Datastore Feb. 26, 2018

Bigquery-to-datastore with homebrew - Homebrew library for bigquery-to-datastore utility.

BigQuery Firebase Feb. 19, 2018

From Firestore to BigQuery with Firebase Functions - Using Firebase functions to get data from Firestore into BigQuery.

BigQuery Feb. 19, 2018

Incremental MYSQL loads to BigQuery using Matillion - Using Matillion, an ETL to Incremental data sync from MYSQL to BigQuery.

BigQuery Official Blog Feb. 12, 2018

Bitcoin in BigQuery: blockchain analytics on public data - The Bitcoin blockchain data are now available for exploration with BigQuery. Learn about some interesting queries and analysis.

BigQuery Feb. 12, 2018

I/O of the Google BigQuery Execution - Learn about the internals of Google BigQuery, and how it executes and gives the super performance for the big data problems.

BigQuery Feb. 12, 2018

Creating and Using Partitioned Tables - Long awaited feature of BigQuery: partitioning by column (DATE or TIMESTAMP type) if currently in Beta.

BigQuery Official Blog Feb. 5, 2018

How to process weather satellite data in real-time in BigQuery - Example of processing weather satellite data in real-time in BigQuery. Data is from the GOES-16 geostationary satellite and is freely available on Google Cloud.

BigQuery Google Cloud Dataprep Feb. 5, 2018

Stream Twitter Data into BigQuery with Cloud Dataprep - Guide to use Cloud DataPrep, BigQuery to analyze data gathered from Twitter’s Streaming API.

BigQuery Tutorial Feb. 5, 2018

Learning to analyze huge BigQuery datasets using Python on Kaggle - Step by step tutorial to set Python's client library for BigQuery and use in notebook.

BigQuery Feb. 5, 2018

BigQuery performance tips: Searching for text 8x faster — “LOWER() is slower” - Example of how to speed up query and get same result.

BigQuery Google Cloud Functions Google Cloud Pub/Sub Google Cloud Storage Jan. 29, 2018

Carefully calculating Google Cloud Storage Buckets size with Cloud Functions & Pub/Sub - Find an easy way to analyze all of your Google Cloud Storage buckets sizes with just a few clicks using Pub/Sub, Cloud Functions and BigQuery.

BigQuery Data Studio Jan. 29, 2018

The best code is no code! Using Google Cloud’s new automated services - Digesting and analyzing data from Internet Service Provider.

BigQuery Jan. 22, 2018

Analyzing your BigQuery usage with Ocado Technology’s GCP Census - GCP Census is a tool that collects metadata about BigQuery tables and stores it back into BigQuery for analysis.

BigQuery Google Cloud Functions Google Cloud Storage Jan. 22, 2018

Scheduling BigQuery jobs using Cloud Storage and Cloud Functions - Trigger BigQuery jobs periodically with Cloud Storage and Cloud Functions.

BigQuery Google Cloud Dataprep Jan. 22, 2018

How to schedule a BigQuery ETL job with Dataprep - Running periodical job in BigQuery with Cloud Dataprep

BigQuery GCP Experience Google Cloud ML Jan. 15, 2018

Trash talk: How moving Apigee Sense to GCP reduced our “data litter” and decreased our costs - Apigee deployed several services to Google Cloud Platform (GCP). This article talks about benefits of moving Apigee Sense to GCP to use its advanced machine learning capabilities.

BigQuery Jan. 15, 2018

Using Google Cloud Platform to store and query 1.4 billion usernames and passwords - The article explains process of importing large dataset into BigQuery and how to get interesting insights from it.

BigQuery Google Cloud Pub/Sub Jan. 15, 2018

Writing a Pub/Sub Stream to BigQuery - Publishing tweets from Twitter's to Pub/Sub and then streaming to BigQuery.

BigQuery Google Cloud Dataprep TensorFlow Jan. 15, 2018

Analyzing Israeli weed market with Google Cloud DataPrep, BigQuery and TensorFlow - Getting, cleaning and analyzing data about marijuana sales with Google Cloud Platform products.

BigQuery Data Studio Tutorial Jan. 8, 2018

Analytics for RESTful interface - This article demonstrate an example project on how to track each request to our RESTful API, analyze and visualize the data in Googles DataStudio Reports.

BigQuery Python Jan. 8, 2018

pandas-gbq Version 0.3.0 - Quick note on pandas-gbq Version 0.3.0

BigQuery Docker Tutorial Jan. 1, 2018

Stop Worrying & Face Facts: You Can Do Data Pipelines Like A Boss With Speedy Google BigQuery Platform - Step by step tutorial to create service for BigQuery tasks.

BigQuery Dec. 25, 2017

Busting 12 myths about BigQuery - Explaining some myths about BigQuery from originated from enterprise customers.

BigQuery Dec. 18, 2017

One year with BigQuery - Thoughts about BigQuery after using it for one year.

BigQuery Dec. 18, 2017

What Is Google BigQuery — Eye Popping Features, Scary Fast, Friendly Pricing and Does The Kessel Run In Less Than Twelve Parsecs - List of basic BigQuery features.

BigQuery GCP Experience Google App Engine Google Cloud Dataflow Google Cloud Dataproc Dec. 18, 2017

How We Implemented a Fully Serverless Recommender System Using GCP - In depth description with code samples of implementing recommendation (serverless) system on Google Cloud Platform.

Advanced BigQuery Google App Engine Google Cloud ML Machine Learning TensorFlow Dec. 18, 2017

Real-time forecasts in the cloud: from market feed capture to ML predictions - how to train a machine learning model to predict FX rates using TensorFlow in Google Cloud, with application components running as App Engine services and communicating with each other by publishing messages via Cloud Pub/Sub and by writing data into BigQuery.

BigQuery Dec. 18, 2017

State of npm scripts - Analyzing npm script from Github using BigQuery.

BigQuery Dec. 11, 2017

How to get real-time, actionable insights from your Fastly logs with Looker and BigQuery - Fastly, whose edge cloud platform offers content delivery, streaming, security and load-balancing, recently integrated its platform with Looker, a business intelligence tool. Using Google BigQuery as its analytics engine, you can use Fastly plus Looker to do things like improve your operations, analyze the effectiveness of marketing programs — even identify attack trends.

BigQuery Google App Engine Dec. 11, 2017

Drawings in the Cloud: introducing the Quick, Draw! dataset - Read how Quick Draw was created, using Google Cloud Platform of course.

BigQuery Dec. 11, 2017

Building a Recursive BigQuery Mapper - The New York Times wrote a Go application that can do the job of replicating our data from Datastore into BigQuery orders of magnitude faster than alternative solutions. Using this system, they were able to achieve over 1.5 million streaming inserts per second.

BigQuery Dec. 4, 2017

Separation of storage and compute in BigQuery - In depth description of how BigQuery works.

BigQuery Dec. 4, 2017

How to take control of your BigQuery costs - Tips and tricks how to control costs when using BigQuery.

Apache Beam BigQuery Dec. 4, 2017

Japanese tokenizer for BigQuery in Apache Beam - Approach to analyze Japanese text on BigQuery.

BigQuery Nov. 27, 2017

BigQuery UI Alpha launched, but is it what we've been waiting for? - Reflection on new BigQuery UI (currently in private testing).

BigQuery Google Cloud Dataflow Google Cloud Storage Nov. 27, 2017

Scheduling tasks on Google cloud platform - Examining different possibilities to schedule batch jobs on Google Cloud Platform.

BigQuery Data Studio Nov. 20, 2017

Monitor and manage your costs with Cloud Platform billing export to BigQuery - Article explains how to export billing data from Google Cloud Platform account to BigQuery and visualize in Data Studio

BigQuery Google Cloud SQL Nov. 20, 2017

Importing data from PostgreSQL to Google BigQuery - Article demonstrates exporting PostgreSQL data to BigQuery.

BigQuery Google Cloud Dataflow Nov. 20, 2017

Using Apache Beam and Cloud Dataflow to integrate SAP HANA and BigQuery - Leveraging both SAP HANA and BigQuery for analytics needs, synced with Cloud Dataflow.

BigQuery Nov. 20, 2017

Announcing BigQuery Data Transfer Service general availability - Service which transfers data from SaaS platforms like AdWords, Youtube to BigQuery is Generally Available.

BigQuery Nov. 20, 2017

BigQuery with Jordan Tigani

BigQuery Google Cloud Dataflow Nov. 20, 2017

How-To: Loading Eloqua Activity Data in to Google BigQuery - Article and github repository provides example how to import data from Eloqua into BigQuery via Dataflow.

BigQuery Nov. 13, 2017

Visualizing BigQuery jobs with Stackdriver, Cloud Functions, Firebase and Pub/Sub - Simple command-line utility as well as web application to let you view the currently running queries as well as some of the history.

BigQuery Nov. 6, 2017

Streaming with Redis — BigQuery - Demonstrating BigQuery streaming capability with pulling data from Redis.

BigQuery Nov. 6, 2017

Patch and Update Table — BigQuery - Article shows example of updating BigQuery table with Python script.

BigQuery Nov. 6, 2017

Scheduling Bigquery jobs using google apps script - Using Google Apps scripting in Spreadsheets, it's possible to run queries on BigQuery in specific time.

BigQuery Nov. 6, 2017

Google Patents Public Datasets: connecting public, paid, and private patent data - In BigQuery dataset, there are 90 million patent publications.

BigQuery Data Studio Google Cloud Storage Oct. 30, 2017

How I Created a Better Google Analytics in 3 Hours - Creating user website tracking analytics using Google Cloud Platform.

BigQuery Oct. 30, 2017

Data Warehouse in Google Cloud - GDG DevFest Ukraine 2017

BigQuery Oct. 30, 2017

Who contributed the most to open source in 2017? - Analyzing Github data with BigQuery to get the answer

BigQuery Oct. 30, 2017

Creating custom interactive dashboards with Bokeh and BigQuery - Bokeh is a Python interactive visualization library that targets modern web browsers for presentation. In tutorial, data from BigQuery is displayed on web application deployed on Google Container Engine.

BigQuery Google Cloud Dataflow Oct. 30, 2017

Big Data Processing at Spotify: The Road to Scio (Part 2) - Description of Scala wrapper for Apache Beam Java SDK created in Spotify.

BigQuery Oct. 30, 2017

Get to know your trees: US Forest Service (FIA) dataset now available in BigQuery - Dataset from the US Forest Service (USFS) Forest Inventory and Analysis (FIA) program (ongoing forest measurement program that provides a nationwide inventory of the forest resources in the United States and its territories) is publicly available in BigQuery.

BigQuery Google Cloud Datastore Oct. 30, 2017

Airflow meets bigquery-to-datastore - Using Apache Airflow to export data from BigQuery do Datastore.

BigQuery Data Studio Firebase Google Cloud IoT IoT Oct. 23, 2017

Build a Weather Station using Google Cloud IoT Core and MongooseOS - Building end to end prototype of weather station using Google Cloud Platform.

BigQuery Oct. 23, 2017

Accelerate BigQuery solution development with intelligent log analysis - With Stackdriver it's possible to see BigQuery errors by creating a logs based metric for the error which we want to detect, then set up alerts.

BigQuery Google Cloud Dataflow Oct. 16, 2017

Separation of compute and state in Google BigQuery and Cloud Dataflow (and why it matters) - Article explain in depth why seperation of state and compute improves speed of big data processing.

BigQuery Oct. 16, 2017

Redshift versus Snowflake versus BigQuery / Part 1: Performance - Benchmarking speed and cost of 3 datawarehouses.

BigQuery Oct. 16, 2017

Export & Load Job with MongoDB — BigQuery Part - Step by step instructions to export data from MongoDB to BigQuery.

BigQuery Sept. 25, 2017

Streaming Google Analytics Data for BigQuery - Data from Google Analytics can be now streamed every 10 minutes to BigQuery

BigQuery Sept. 25, 2017

Getting Started Accessing the HTTP Archive with BigQuery - Starting with BigQuery by analyzing http archive (public dataset)

BigQuery Sept. 25, 2017

Twitter Analytics (Part 1) - Streaming data from Twitter into BigQuery

BigQuery Sept. 18, 2017

GCP Podcast - Have you ever wanted to know what powers BigQuery under the hood?

BigQuery Google Cloud Dataflow Google Cloud Datastore Sept. 18, 2017

Export BigQuery to Google Datastore with Apache Beam/Google Dataflow

BigQuery Sept. 18, 2017

BigQuery: Managed storage for analytics - @Scale conference 2017, Jordan Tigani, Software Engineer, Google

BigQuery Sept. 11, 2017

How to crunch your business data from Sheets in BigQuery - Example how BigQuery can query Google Sheets just like any other table.

BigQuery Google Cloud ML Machine Learning TensorFlow Sept. 4, 2017

How to predict your Google Cloud Platform monthly bill with Google CloudML & BigQuery - Creating billing prediction system with BigQuery, TensorFlow and Cloud ML

BigQuery Sept. 4, 2017

Codelab: Querying and Visualising Location Data in BigQuery using Google Maps API - In this codelab, you'll write and run some queries that demonstrate how to provide location based insights into very large public datasets using BigQuery. You'll also build a web page that loads a map using the Google Maps JavaScript API, and that runs and visualizes spatial queries against the same very large public datasets using the Google APIs Client Library for Javascript, and the BigQuery API.

BigQuery Aug. 28, 2017

From data to insights: Preview a new Google Cloud training course for SQL analytics - Preview of content of course: From Data to Insights with Google Cloud Platform

BigQuery Windows Aug. 28, 2017

Guide to using Google BigQuery on Microsoft PowerShell - run BigQuery commands & queries straight from Windows PowerShell

BigQuery Google Cloud Storage Aug. 20, 2017

How to analyze Fastly real-time streaming logs with BigQuery - Fastly offers cloud, content delivery and now logs from Fastly can be streamed to Google Cloud Storage or BigQuery

BigQuery Aug. 20, 2017

How was Game Of Thrones S07 E05? Tweet Analysis with Kafka, BigQuery and Tableau - Analyzing data from Twitter regarding Game of Thrones using Tableau and BigQuery

BigQuery Google Cloud Dataflow Google Cloud Pub/Sub Aug. 20, 2017

How we saved over $240K per year by replacing Mixpanel with BigQuery, Dataflow & Kubernetes - Description how to use Google Cloud Platform Products to replace Mixpanel (Analytics for web / mobile)

BigQuery Aug. 20, 2017

BigQuery: Required Reading List - Tino Tereshko from Google put list of articles regarding BigQuery

BigQuery Aug. 14, 2017

When art meets big data: Analyzing 200,000 items from The Met collection in BigQuery - Analyzing meta data about items from The Metropolitan Museum of Art

BigQuery Data Studio Aug. 7, 2017

The most famous reddit accounts (analyzed with BigQuery) - Analyzing Reddit content to get most influential Reddit users using BigQuery and displaying in Data Studio

BigQuery Google Cloud Dataflow Aug. 7, 2017

Life of a Cloud Dataflow service-based shuffle - Shuffle implementation (currently in beta) is in the Cloud Dataflow SDK for Java version 2.0. In this post, it's explained and demonstrated the practical impact of the new shuffle on data pipelines using the Opinion Analysis project as an example.

BigQuery Google Cloud Dataflow Google Cloud Pub/Sub Aug. 7, 2017

Traveloka’s journey to stream analytics on Google Cloud Platform - Traveloka recently migrated this pipeline from a legacy architecture to a multi-cloud solution that includes the Google Cloud Platform (GCP) data analytics platform.

BigQuery Cloud Natural Language API Aug. 7, 2017

Using Google BigQuery, Google Cloud Natural Language API and Looker to Work-Out Exactly How Much Mail Online Readers Hate Me - Analyzing users comments of article using Natural Language API and displaying results in Locker.

BigQuery July 31, 2017

Leading with commas — ugly or efficient? An investigation over 320 GB of SQL code - How to analyze SQL queries files from Github and see if they use trailing or leading comma with BigQuery

BigQuery July 24, 2017

Google Cloud Platform for Data Scientists: Using R with Google BigQuery, Part 2 (storing and retrieving data frames) - How to create an R data frame and stash it in BigQuery and then access that data frame later from the BigQuery web interface and RStudio.

BigQuery July 17, 2017

The 12 Components of Google BigQuery - Overview of 12 BigQuery components (properties)

BigQuery July 17, 2017

Inside Google BigQuery - Overview of BigQuery properties

BigQuery July 17, 2017

How to aggregate data for BigQuery using Apache Airflow - Creating Airflow (workflow management framework that lets you programmatically create, schedule and monitor workflows using simple Python syntax) pipelines for BigQuery

BigQuery July 10, 2017

Counting uniques faster in BigQuery with HyperLogLog++

Big Data BigQuery June 26, 2017

The Google Data WareCity - Interesting and unique aspects of BigQuery’s data sharing capability

Big Data BigQuery June 26, 2017

GCE BigQuery vs AWS Redshift vs AWS Athena - Basic comparison on data loading and simple queries between Google BigQuery and Amazon Redshift and its cousin Athena.

BigQuery June 12, 2017

Analyzing Django requirement files on Gi - Getting insights about how Django is used in open source projects by analyzing Github data with BigQuery

BigQuery June 12, 2017

Analyzing Google Cloud Billing Data with Big Query - Step by step example with sample queries regarding costs analysis for Google Cloud Platform products and services with BigQuery

BigQuery Google Cloud Pub/Sub June 12, 2017

Infinite backup for Cloud Pub/Sub - Storing streaming data into BigQuery via Pub/Sub

BigQuery June 12, 2017

BigQuery, Looker And Big Data Analytics At Petabyte-Scale

BigQuery June 12, 2017

Google BigQuery, Large Table Joins and How Nested, Repeated Values and the Capacitor Storage Format (and Looker) Saves the Day - Example from BigQuery public data sets about how to structure data (by nesting) for efficient querying

BigQuery Business June 12, 2017

Think Economics -- Not Features -- When Evaluating Big Data Value - Using BigQuery instead Hadoop should save $$$ in long term

BigQuery June 12, 2017

Google + OpenAQ Team Up to Make Global Air Quality Data Easier to Access - Globally-aggregated data of air quality has now been added to the Google Cloud BigQuery Public Datasets program. For example, OpenAQ-aggregated data can be accessed, visualized and explored through Google Data Studio, or directly using Google

BigQuery Data Studio June 4, 2017

Visualize GCP Billing using BigQuery and Data Studio - Creating detailed billing reports of Google Cloud Platform with Data Studio

BigQuery June 4, 2017

Life of a BigQuery streaming insert - In depth article about how streaming inserts work in BigQuery

BigQuery Google Cloud Dataflow June 4, 2017

BigQuery partitioning with Beam streams - using TableReference functions

BigQuery Data Studio May 29, 2017

Analyze your Amazon Web Services (AWS) Bills w/Google BigQuery & Data Studio - Step by step tutorial about how to export billing data from AWS to GCP, load to BigQuery and display in Data Studio

BigQuery May 29, 2017

How To Make Learning Google Cloud and BigQuery Easy - Pre-built Docker Image With Google Cloud SDK and BigQuery goodness

BigQuery May 29, 2017

BigQuery now has 10 GB of free storage, and 1 TB of free monthly queries (try it today) - In free BigQuery tier is now included 10GB of data storage

BigQuery May 22, 2017

New healthcare and population datasets now available in Google BigQuery - RxNorm provides normalized names for clinical drugs and links its names to many of the drug vocabularies commonly used in pharmacy management and drug interaction software.

BigQuery Google Cloud Storage May 22, 2017

How to do serverless pixel tracking with GCP - One way to understand user behavior is by using pixels, small 1x1 transparent images embedded into the web property. When loaded, the pixel calls a web server that records the request parameters passed in the URL that can be processed later.

BigQuery May 15, 2017

Interpreting 23andme Raw Genome Data with Google Genomics and BigQuery

BigQuery Google App Engine Google Cloud Dataflow Google Cloud Pub/Sub May 15, 2017

Designing ETL architecture for a cloud-native data warehouse on Google Cloud Platform - Example of ETL process on Google Cloud Platform utilizing Dataflow, BigQuery, App Engine

BigQuery May 8, 2017

Let’s talk BigQuery - Short chat about BigQuery

BigQuery May 8, 2017

What BI Development Looks like with BigQuery, Google Cloud APIs, Looker and Fluentd - Examples of using BigQuery with BI tools

BigQuery Google App Engine May 1, 2017

How to build a BI dashboard using Google Data Studio and BigQuery - Use case for visualization of data from BigQuery with Google Data Studio

BigQuery May 1, 2017

Top 5 tips for migrating your data warehouse to Google BigQuery - Main points from article bellow Framework for migrating your data warehouse to Google BigQuery

BigQuery May 1, 2017

Framework for migration your data warehouse to Google BigQuery - Everything you need to know when migrating to BigQuery... 20 page description

BigQuery April 24, 2017

BigQuery — one store to rule them all - Description of BigQuery experience.

BigQuery April 24, 2017

Microservice Usage Logging with Openresty and Google BigQuery - Capturing number of transferred bytes in Nginx with Lua and storing data in BigQuery

BigQuery April 24, 2017

Why You Should Check Out BigQuery for Big Data - It's fast, it's easy, it's cheap, it's growing

BigQuery April 24, 2017

DNA Sequencing of 1000 Cannabis Strains publicly available in Google BigQuery - Applying standard bioinformatics analyses to newly sequenced Cannabis strains via Google BigQuery

BigQuery April 24, 2017

Whispers from the other side of the globe with BigQuery - Analyzing radio data with BigQuery

BigQuery April 24, 2017

Analyzing Big Data in less time with Google BigQuery

BigQuery April 24, 2017

Investigating Global Temperature Trends with BigQuery and Tableau - Digging through public database of climate observations with BigQuery and displaying with Tableau

BigQuery April 17, 2017

Fun with BigQuery and R — Building a Google Analytics Alternate - Short tutorial about how to start with BigQuery and R

BigQuery April 17, 2017

How to do cross-platform analytics with Google BigQuery - Namely Google Analytics 360 and Firebase Analytics

BigQuery April 17, 2017

The top GitHub projects per country - What is the most popular Github project in every country? You can find out by analyzing Github data with BigQuery

BigQuery April 17, 2017

Email Logs in BigQuery

Big Data BigQuery April 10, 2017

BI Performance Benchmarks with Google BigQuery

BigQuery April 10, 2017

How to Use BigQuery for Large-Scale SEO (Or Whenever Excel Fails) - Step by step tutorial about how to start with BigQuery and use it to analyse Google Analytics data

BigQuery April 3, 2017

Exploring 1.5 Billion NYC Taxi/Uber rides with LL Notebook and BigQuery - Nice visualizations of New York City taxi rides with BigQuery and LiquidLandscape

BigQuery April 3, 2017

Using Google’s BigQuery to Better Understand the Python Ecosystem - Analyzing Github data about Python with BigQuery

BigQuery April 3, 2017

How Ocado moved its databases to the cloud - "Since adopting the Google Cloud Platform, we’ve reduced storage costs to a tenth, increased our storage capacity over twenty times and improved performance by hundreds of times compared to our previous approach of hosting data on-premise" that's conclusion from Ocado

BigQuery Google Compute Engine Machine Learning March 27, 2017

Analyzing data with BigQuery and Machine Learning APIs (Forecast the Dutch elections based on social media presence) - Getting data from Twitter, using Translate API to translate from Dutch to English so that Natural Language API can be used. All with Javascript

BigQuery March 27, 2017

Big Data, BigQuery and a story of bikes - Analyzing bike rides via BigQuery

BigQuery Google Cloud SQL March 13, 2017

Alooma Provides Real-Time Data Pipelines to Google Cloud SQL and BigQuery

Big Data BigQuery PubSub March 6, 2017

Combining Thomson Reuters data with Google BigQuery and Google Cloud Pub/Sub API - Proof of concept to analyze data with BigQuery ingested from Reuters API

BigQuery March 6, 2017

Visitor Acquisition, Retention, and Attrition, Using BigQuery and a Simple Data Model - In depth article which explains how BigQuery can be used to analyze data from Google Analytics namely visitor acquisition, retention, attrition Export of results to Google Sheets to use it's functionality for further analysis

BigQuery Feb. 27, 2017

The Rise and Fall of New York City (311 complaints) - Lots of interesting insights with New York open data

BigQuery Feb. 27, 2017

The top weekend programming languages — based on GitHub’s activity - Playing with BigQuery and Stackoverflow data

BigQuery Dataflow Kubernetes Feb. 27, 2017

Adding machine learning to a serverless data analysis pipeline - When you put together Pub/Sub, Kubernetes, Dataflow, BigQuery you get serverless data analysis pipeline

BigQuery Feb. 27, 2017

BigQuery, Looker and Big Data’s Rediscovery of Data Warehousing and Semantic Models … at Google Scale

BigQuery Google Cloud Functions Python

Turning GA360 BigQuery exports into partitioned tables using Cloud Functions - Article describes how to create a cloud function that will copy a BigQuery GA360 data export to another BigQuery table.

BigQuery Official Blog

Finding data insights faster with BigQuery and GCP Marketplace solutions - GCP Marketplace offers other options to get data into BigQuery.

 

Latest Issues




Contact

Zdenko Hrček
Třebanická 183
Prague, Czech Republic
Phone: +420 777 283 075
Email: zdenko@gcpweekly.com