Tag: LLM

Resources about LLM collected from the weekly GCP Weekly newsletter — a hand-curated collection of Google Cloud Platform news, technical articles, tutorials, videos and release notes. Each entry below links back to the original GCP Weekly issue and retains its publish date and title, giving you a focused, searchable knowledge base for this topic.

Cloud Storage GKE LLM Aug. 10, 2026

How to build an elastic, scalable LLM Inference Platform on GKE using Fluid Compute - This guide provides a comprehensive walkthrough on architecting a fault-tolerant, cost-optimized Large Language Model (LLM) serving platform on Google Cloud using diverse GPU consumption types. It demonstrates how to leverage Google Kubernetes Engine (GKE), multi-region GCS buckets with zonal rapid caching, and custom compute classes to balance high availability with significant cost savings.

AI DevOps LLM Aug. 10, 2026

Scaling real-time AI agents with session-aware load balancing - Real-time AI agents break traditional request-response load balancing paradigms because they rely on long-lived, stateful bidirectional streams that obscure true server capacity. To solve this, developers must implement application-level session tracking directly within the runtime to accurately measure the committed concurrent workload of active conversations.

AI API Gateway LLM Aug. 10, 2026

Model routing with Google Cloud API Gateway - Google Cloud API Gateway now offers a model routing feature in Public Preview, allowing developers to dynamically route traffic to models like Gemini, Claude, or OpenAI OSS-GPT without hardcoding endpoints or managing open-source proxies. Developers can easily configure these routing rules directly within their OpenAPI 3.x specifications by mapping virtual model names to specific backend targets on a shared host.

ADK LLM Paywall Aug. 3, 2026

Evaluating Google ADK Agents: From Execution Traces to Regression Tests (Part 4) - A Practical Guide to Eval Sets, Eval Configs, Trajectory Metrics, and Rubric-Based Scoring.

Generative AI LLM Aug. 3, 2026

Announcing Day 0 support for Kimi K3 on Google Cloud - This article explains how to efficiently deploy and serve Moonshot AI’s massive 2.8-trillion-parameter Kimi K3 model using Google Cloud's advanced AI infrastructure. It outlines flexible deployment pathways—ranging from quick setups via Model Garden to distributed serving on Google Kubernetes Engine (GKE) utilizing SGLang and high-performance hardware accelerators.

LLM TPU Aug. 3, 2026

Inside the optimization of Mistral 3 large inference on Ironwood - This technical article details how engineers optimized the inference performance of the Mistral 3 Mixture-of-Experts (MoE) model on Google’s Ironwood TPUs. By implementing advanced strategies like hybrid sharding, SparseCore tree reductions, and asynchronous scheduling, the team achieved up to a 48% boost in throughput and a 30% reduction in latency without sacrificing benchmark accuracy.

AI LLM Aug. 3, 2026

Announcing support for the Inkling model family on Google Cloud - Google Cloud now supports Thinking Machine Lab's Inkling multimodal model family—including the highly efficient Inkling-Small variant—through Model Garden on Agent Platform.

Agents Data Science LLM Machine Learning July 27, 2026

Dynamic Tool Orchestration & Writing - Beyond Fixed Catalogs: How Adaptive AI Agents can write Tools on the Fly, Manage Risk, and Degrade Gracefully.

Infrastructure Kubernetes LLM July 27, 2026

A Deep Dive into High-Efficiency Agentic Serving on GKE with vLLM and llm-d - This article details a high-efficiency architecture for serving LLM agents on Google Kubernetes Engine (GKE) using vLLM and llm-d. It tackles the "Memory Tax" problem in multi-turn agentic workflows by disaggregating prefill and decode operations, and intelligently offloading idle KV caches from expensive GPU memory to cheaper host RAM. This approach significantly maximizes GPU utilization, eliminates prefill stalls, and enables linear scalability of agentic services on GKE.

AI GKE LLM Official Blog July 27, 2026

Minimize idle accelerators: Native RL job interleaving with co-operative time-slicing in llm-d - Co-operative time-slicing in llm-d treats discrete RL steps as schedulable entities so you can interleave RL jobs onto shared physical hardware.

LLM Machine Learning Vertex AI July 20, 2026

Tuning enterprise embeddings in Google Vertex AI: The mathematics of LoRA, Weighted Contrastive Loss, and GCP Pipelines - This deep-dive explores the engineering and mathematics behind fine-tuning embeddings on Google Cloud's Vertex AI to overcome the limitations of general pre-trained models for enterprise data. It details how proprietary jargon is handled by mathematically warping the embedding space using advanced techniques like Weighted Multiple Negatives Ranking Loss (wMNRL) and Parameter-Efficient Fine-Tuning (LoRA). This process, managed through Vertex AI Pipelines, significantly improves retrieval accuracy for specialized corporate use cases.

AI Generative AI LLM July 20, 2026

Building scalable AI agents with modular prompt transpilation - To resolve the scaling bottlenecks and runtime errors caused by monolithic system prompts, engineering teams should treat prompts as build artifacts by modularizing instructions into reusable templates. By running these modular "skill files" through a transpiler, developers can enforce static validation, catch missing dependencies at build time, and integrate prompt generation directly into their CI/CD pipelines. This deterministic approach prevents code drift and ultimately establishes a safe framework where agents can propose updates to their own logic via standard pull requests.

AI Generative AI LLM July 20, 2026

Guide to AI Tokenomics: Eleven Principles for Token Efficient Software Engineering - Optimize your software engineering with our Guide to AI Tokenomics. Explore 11 actionable principles to improve coding assistant performance, reduce latency, and control costs effectively.

AI CI LLM Threat Intelligence July 20, 2026

Demystifying AI Exploits: A Blueprint for AI-Assisted Vulnerability Management - Details the architectural risks of integrating LLM agents into CI/CD pipelines for automated vulnerability discovery.

LLM Quadrant July 20, 2026

Google named a Leader in the 2026 IDC MarketScape for Worldwide Foundation Model Software - Google has been named a Leader in the IDC MarketScape: Worldwide Foundation Model Software 2026 Vendor Assessment.

AI LLM Partners July 20, 2026

Claude at scale on Google Cloud: Frontier AI, built for enterprise production - To achieve frontier AI at scale, Claude brings the reasoning and Google Cloud brings the managed infrastructure, global reach, and compliance posture that enterprises already run on.

ADK Data Science LLM Paywall July 13, 2026

Deploying Google ADK Agents: From Local Script to a Managed Cloud Runtime - A Step-by-Step Guide to Taking a Local Multi-Agent System to a Managed Cloud Endpoint.

ADK LLM OpenTelemetry July 13, 2026

Mastering Observability in Google ADK: Implementing OpenTelemetry for GCP Metrics, Traces, and Logging - This article demonstrates how to achieve robust observability for complex LLM applications built with Google's Agent Development Kit. It details implementing OpenTelemetry to export comprehensive metrics, traces, and logs directly into Google Cloud Platform. This integration provides crucial visibility into agent reasoning, performance bottlenecks, and debugging capabilities, transforming black-box AI into an observable system.

LLM July 13, 2026

Evolving LLM fine-tuning hyperparameters with AlphaEvolve on Google Cloud

Cloud Run Generative AI LLM Security July 13, 2026

Safely run AI-generated code in Cloud Run sandboxes - Today, at WeAreDevelopers World Congress, we are announcing Google Cloud Run sandboxes in public preview. Cloud Run sandboxes are a native, secure, and ultra-fast runtime environment built specifically for executing untrusted code and agent workloads, starting in milliseconds.

Cloud Marketplace Gemini Generative AI LLM July 13, 2026

A developer's guide to publishing agents in Gemini Enterprise and Google Cloud Marketplace - For agent builders and developers looking to publish and commercialize high-impact, third-party agents through Google Cloud Marketplace and deploy them to the Gemini Enterprise app, this guide provides a step-by-step path to a fully integrated solution.

LLM July 6, 2026

Decoding the deep geometry of enterprise embeddings: The definitive guide to Google’s Vertex AI, Gecko, and Vector Physics - This comprehensive guide offers an engineering deep-dive into Google Cloud's Vertex AI text embeddings, particularly gemini-embedding-001, detailing their mathematics and backend architecture. It explains how these advanced models power enterprise AI applications like RAG and semantic search, covering innovative features such as multilingual support, Matryoshka Representation Learning, and crucial production considerations like chunking and drift detection.

Generative AI LLM July 6, 2026

Get started with the Claude apps gateway for Google Cloud - The Claude apps gateway is a self-hosted service that sits directly between your local Claude Code clients and Gemini Enterprise Agent Platform. This post breaks down why you should run it and what secure deployment looks like on Google Cloud.

GKE Google Cloud Managed Lustre LLM July 6, 2026

Scaling LLM Inference: Multi-Node KV Cache Offloading with GKE & Managed Lustre - Scale enterprise LLM inference using GKE and Google Cloud Managed Lustre. Learn to offload KV caches for better long-context support and reduced GPU-hour costs.

Gemini LLM July 6, 2026

Bringing speed and strong cost performance to the market with Gemini Omni Flash and Nano Banana 2 Lite - We’re adding two new models to Gemini Enterprise Agent Platform. Nano Banana 2 Lite (Gemini 3.1 Flash-Lite Image) is available for everyone, and Gemini Omni Flash is available in public preview. Read more to get started.

ADK LLM Paywall June 29, 2026

Google ADK Explained: Building Multi-Agent Systems With Google’s Agent Development Kit - A Practical Introduction to Google’s Code-First Agent Framework.

GKE LLM TPU June 29, 2026

Accelerate TPU model loading while saving RAM on GKE - Large language models on TPUs often face significant "cold start" bottlenecks due to slow loading times and excessive memory usage from double-buffering. The new Run:ai Model Streamer, now integrated with Google Cloud Storage, directly streams model tensors from storage, bypassing local disk and eliminating the double-buffering trap. This dramatically accelerates model loading and reduces peak memory consumption, enabling faster and more efficient scaling of vLLM inference pipelines on Google Kubernetes Engine.

Kubernetes LLM June 22, 2026

Deploying NVIDIA Nemotron-3 Ultra 550B, with B200 GPUs, vLLM on Google Kubernetes Engine — Football edition - This guide outlines the process of deploying the NVIDIA Nemotron-3 Ultra 550B Large Language Model on Google Kubernetes Engine, utilizing B200 GPUs and vLLM for high-performance inference. It details setting up a secure and private network architecture, complete with a custom Virtual Private Cloud and an internal load balancer. The deployment is then demonstrated through an interactive, football-themed CLI application that leverages the model's capabilities for dynamic simulation and commentary.

Cloud Run Docker LLM Terraform June 22, 2026

Serverless GPU Inference: Deploy Any Hugging Face Model on Google Cloud Run

Antigravity Go LLM Typescript June 22, 2026

How I learned Go in a Day with Antigravity 2.0 and How You Can Do the Same - Learn how to migrate a TypeScript CLI to Go in just one day using Antigravity 2.0. This guide covers architectural goals, AI-driven TDD, and building fast, secure software.

Showing 31–60 of 302 resources

 

Latest Issues




Contact

Zdenko Hrček
Třebanická 183
Prague, Czech Republic
Phone: +420 777 283 075
Email: [email protected]