Google Cloud Platform Newsletter

Check Archive for older issues

With coronavirus touching everybody's lives, it's encouraging to see how people and companies are supporting each other and how technology and cloud have a big impact in these times.

On GCP side note, we have several CI/CD pipeline examples as well as new AI Platform pipelines.

News

Introducing Cloud AI Platform Pipelines - AI Platform Pipelines provides enterprise-ready infrastructure for deploying and running structured ML workflows, and pipeline tools for building, debugging, and sharing pipelines and components.

Local SSDs + VMs = love at first (tera)byte - In Google Cloud Storage you can now attach 6TB and 9TB local SSDs to virtual machines (VMs) for higher throughput and IOPS per VM.

Modern analytics made easy with new Redshift, S3 migration tools - Data warehouse migrations to Google Cloud’s BigQuery are now easier with new tools to move your Redshift and S3 workloads.

Get the Flink Operator for Kubernetes in Anthos on Marketplace - The open source Apache Flink for Kubernetes operator is now available in the Google Cloud Marketplace to deploy in your own cluster.

Compute Engine instance creation made easy with machine images - Compute Engine machine images capture everything you need to create a new instance.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Finding a problem at the bottom of the Google stack - See a real-world example of how Google’s SRE practices can identify and help fix issues, even at the bottom of the hardware stack.

Parameterised Kubernetes deployments without Helm via GCP Cloud Build - Setting Cloud Build for Google Kubernetes Engine CI/CD.

CI/CD at Ai Incube - The explanation of the CI/CD system on GCP.

Managing Billing Permissions in Google Cloud - The article goes through basic steps to set up a billing account, permissions, etc.

Identity-Aware Proxy for On-Prem applications - Using Identity Aware Proxy to secure internal systems at home.

Improving Security with Impersonation - The article describes the impersonation of service accounts and how to set it up.

App Development, Serverless, Databases, DevOps

Automating Cloud Storage Data Classification: Setup Cloud Storage and Pub/Sub - Automation of data classification in Cloud Storage for security and organizational purposes using Data Loss Prevention API.

How to Create an Image Translation Web App in 25 Lines of Code - Simple OCR web application using Firebase and other GCP products.

How to use Terraform to schedule backups for your Google Cloud SQL database - Using Terraform to schedule daily backups of Cloud SQL database to Cloud Storage bucket.

Deploy GCP App Engine from Azure Release Pipelines - Interesting combination, deploying App Engine app from Azure DevOps pipelines.

A Healthy Dialogflow Part I: A Case for Google Cloud Platform’s NLP Service - Proof of concept for chatbot related to health implemented with Dialogflow.

Knative Eventing Delivery Methods - The article explains delivery methods in Knative.

Big Data, Analytics, ML&AI

CI/CD and Change Management for Pipelines — Part 1 - Process and steps to take in order to implement a CI/CD pipeline for CDAP (Cloud Data Fusion).

Use SRE principles to monitor pipelines with Cloud Monitoring dashboards - Try SRE principles and the four golden signals as the metrics to build a monitoring dashboard for your data pipelines.

How to query and calculate GA App + Web event data in BigQuery - In-depth of new Firebase Google Analytics way of measurement and how to use raw data in BigQuery.

Apache Spark and Jupyter Notebooks made easy with Dataproc component gateway - Make use of the new Dataproc optional components and component gateway features to easily use Jupyter Notebooks.

Setting Up Jupyter on Google Cloud - A scriptable list of command lines to deploy Jupyter in Google Cloud, securely and cost-effectively, with added exercises.

Processing 10TB of Wikipedia Page Views - Part 1 - Processing and uploading Wikipedia page views into BigQuery.

Sending data from BigQuery to Intercom using Google Cloud Functions - Using Stackdriver triggering to send notifications when BigQuery table is updated.

Yet another GeoJson to Ndjson converter - You might have already seen many ways to convert GeoJson files to something BigQuery can understand. Let’s invent one more wheel!

Data analysis with SQL and BigQuery on New york city bikes data. - Starting with New York biking open data analysis.

Building an end to end Machine Learning Pipeline in Bigquery - Building a simple end to end BigQuery Machine Learning pipeline using open-source framework Dataform.

Get started with PyTorch, Cloud TPUs, and Colab - Running Machine Learning With PyTorch on TPUs in Colab.

TensorFlow Extended (TFX): Using Apache Beam for large scale data processing - Using Apache Beam (Cloud Dataflow) for TensorFlow Extended pipelines.

Various

How to pass GCP Professional Data Engineer exam in 2 months - Sharing experience of preparing and passing GCP Professional Data Engineer certification.

My journey of GCP Professional Data Engineer (2020)! - A journey to GCP Professional Data Engineer Exam.

Slides, Videos, Audio

GCP Podcast - #211 Digital Services with xMatters.

Kubernetes Podcast - #94 gRPC, with Richard Belleville.

Releases

BigQuery Transfer - BigQuery Data Transfer Service now supports the Finland region. BigQuery Data Transfer Service now supports the Zürich region.

Cloud Build - The Create trigger page on the Cloud Console has been updated.

Cloud Composer - You can now control access to the Airflow web server, either allowing access from any IP address (default), or specifying which IP ranges have access.

Config Connector - ComputeHealthCheck's location field now supports supplying a region. Fixed an issue with deleting StorageBucketAccessControl when the ServiceAccount did not exist: https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/39. With the exception of role-bindings, moved all system components for namespaced mode into the cnrm-system, note: you must completely uninstall and reinstall to upgrade namespaced mode completely for this release. Added a version annotation to the Config Connector manifests.

Data Catalog - Support for custom entries is now in beta.

Dataproc - Added the following flags to gcloud dataproc clusters create and gcloud dataproc workflow-templates set-managed-cluster commands: --num-secondary-workers --num-secondary-worker-local-ssds --secondary-worker-boot-disk-size --secondary-worker-boot-disk-type --secondary-worker-accelerator. The following flags to gcloud dataproc clusters create and gcloud dataproc workflow-templates set-managed-cluster commands have been deprecated: --num-preemptible-workers --num-preemptible-worker-local-ssds --preemptible-worker-boot-disk-size --preemptible-worker-boot-disk-type --preemptible-worker-accelerator See the related change, above, for the new flags to use in place of these deprecated flags.

Datastore - Support for us-west3 (Salt Lake City) and asia-northeast3 (Seoul).

Dialogflow - On March 16, 2020, the Inline Editor will use Cloud Functions instead of Cloud Functions for Firebase. Event names are now limited to 50 characters.

Cloud Firestore - Support for us-west3 (Salt Lake City) and asia-northeast3 (Seoul).

Stackdriver Logging - Cloud Logging Agent for Windows version 1-11 is now available. Logs Viewer (Preview) now contains a histogram panel.

Managed Microsoft AD - VPC Service Controls integration is now in beta.

Stackdriver - Cloud Logging Agent for Windows version 1-11 is now available. Logs Viewer (Preview) now contains a histogram panel.

Cloud Vision API - OCR model upgrades The text_detection and document_text_detection models have been upgraded to newer versions.

VPC Service Controls - Beta stage support for: Managed Service for Microsoft Active Directory.

AI Platform - Runtime version 2.1 for AI Platform Prediction is now available. If you deploy a model version for online prediction that uses runtime version 2.1 with a GPU, AI Platform Prediction uses TensorFlow 2.0.0 (instead of TensorFlow 2.1.0) to serve predictions. Runtime version 2.1 for AI Platform Training is now available. Runtime version 2.1 includes scikit-learn 0.22 rather than 0.22.1. When you train with runtime version 2.1 or later, AI Platform Training uses the chief task name to represent the master VM in the TF_CONFIG environment variable.

Dialogflow Enterprise - On March 16, 2020, the Inline Editor will use Cloud Functions instead of Cloud Functions for Firebase. Event names are now limited to 50 characters.

Secret Manager - Secret Manager is generally available.