Tag: Streaming

Cloud Dataflow Data Analytics Official Blog Streaming June 10, 2024

Boost developer productivity with new pipeline validation capabilities in Dataflow - Dataflow pipeline validation is now generally available. It performs dozens of checks to ensure that your batch or streaming job is error-free and can run successfully.

BigQuery Cloud Dataflow Official Blog Streaming June 3, 2024

Accelerating CDC insights with Dataflow and BigQuery - This post covers how to use BigQuery’s new CDC capability in Dataflow along with the new Dataflow at-least-once streaming mode to simplify your CDC pipeline and reduce costs.

AWS Cloud Pub/Sub Data Analytics Official Blog Streaming June 3, 2024

Easily stream data from AWS Kinesis to Google Cloud with Pub/Sub import topics - Pub/Sub import topics enable streaming ingestion into BigQuery from external sources, with the first supported external source being Amazon Kinesis Data Streams. Import topics provide a simplified way to ingest data from Amazon Kinesis Data Streams directly into Pub/Sub, reducing the complexity of setting up data pipelines between clouds. Once the connection is established, Amazon Kinesis producers can be gradually migrated to Pub/Sub publishers. Data from Amazon Kinesis Data Streams can be routed to BigQuery using BigQuery subscriptions, and Pub/Sub autoscales to adapt to changes in the Amazon Kinesis data stream.

Cloud Dataflow Data Analytics Official Blog Streaming May 27, 2024

More flexibility for your Dataflow jobs with new controls for latency versus cost - Dataflow Streaming Engine users can now choose between lower peak latency or lower streaming costs for their workloads by adjusting the autoscaling utilization hint value. The autoscaling hint value can be set to a higher or lower value using a Dataflow service option. Dataflow’s autoscaling UI provides insights on when it’s worth adjusting the autoscaling behavior and additional dashboards and metrics to monitor the impact of changes.

Data Analytics Official Blog Streaming May 27, 2024

Google Data Cloud innovations for continuous real-time intelligence - Google Cloud offers innovations for continuous real-time intelligence, enabling organizations to harness real-time analytics and make informed decisions. With Dataflow, BigQuery, and Apache Kafka for BigQuery, enterprises can leverage streaming infrastructure for visibility, predictions, and activation. Customers like Spotify, Puma, Compass, and Tyson Foods have achieved significant business impact using Google Cloud's data, AI, and real-time solutions.

Cloud Dataflow Official Blog Streaming May 20, 2024

No work items left unturned: How Dataflow mitigates stragglers

 

Latest Issues




Contact

Zdenko Hrček
Třebanická 183
Prague, Czech Republic
Phone: +420 777 283 075
Email: [email protected]