The secret to cost-efficient AI inference // TRAIN BRAIN

The secret to cost-efficient AI inference

See the detailed reference architecture → https://goo.gle/4bKh5aR
Learn how to use JAX, Google Kubernetes Engine (GKE) and NVIDIA Triton Inference Server as a winning combination for low cost AI inference.
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
Speaker: Don McCasland
Products Mentioned: AI Infrastructure

Google Cloud Tech

Helping you build what's next with secure infrastructure, developer tools, APIs, data analytics and machine learning....

Build AI Agents on GKE in the GKE Turns 10 Hackathon

Visualizing BigQuery geospatial data in Colab

How to deploy a container image to Cloud Run

The Agent Factory - Episode 4: Remember me: Memory in Agents

Container-Optimized Compute Platform for GKE Autopilot

The Future of Automation: How AI Agents are Revolutionizing Data and Robotics

Enable Looker Studio report generation with one click!

GKE Inference Reference Architecture, Your Blueprint for Production-Ready Inference

This Month in GKE: July Edition

The Agent Factory - Episode 3: Building Custom Tools for Agents

The future of AI architecture: Using agents to supervise other agents

What is Private Google Access?

Defining AI agents: From definitions to stopping conditions

The AI Agent Bake-Off Ep.1 Trailer / Can They Build an AI Agent in 3 Hours?

The AI Agent Bake-Off Ep.1 | Can they build an AI Agent in 3 hours?

Roll & Score: Evolve arcade with AI

How to use Agentspace search on Google Distributed Cloud

AI Agents: From concept to code

Let’s build a GenAI app on Cloud Run

Save time and follow data best practices with Looker Blocks!

Unlock AI Potential: Introducing the Google Cloud & NVIDIA Community

Self Serving Allowlisting of Privileged Apps on GKE Autopilot

The Agent Factory - Episode 2: Multi-Agent Systems, Concepts & Patterns

Develop with Looker and Looker Studio

Is your Infrastructure Ready for GPUs and AI Agents?

Build AI agents with Cloud Run and Firebase Genkit

The Agent Factory - Episode 1: Agents, their frameworks and when to use them

Agentic Business Intelligence done right

Ways to enhance your developers’ productivity with Gemini

Platform engineering: building a developer platform

Develop with Looker Studio Community Connectors and Visualizations

How to Turn Images Into Animated Stories Using Gemini and Veo 2

Secure Your AI Lifecycle with AI Protection in Security Command Center

Build AI Agents in the Agent Development Kit Hackathon

This Month in GKE: April Edition

Ground Gemini with Google Search with the Gen AI SDK

Effortless Image Editing + Animation with Gemini 2.0 Flash and Veo 2

Scaling your tuned models with Cloud Run

Optimizing AI infrastructure for modern retail

Watch devs analyze Formula E races with Google Cloud

Sentiment analysis with LLMs/BQML over your data

How to optimize costs and fuel innovation by moving VMware to Google Cloud

How Google Cloud makes AI scale at unbelievable speed!

AI Crash Course: From Still to Motion: Create Visuals & Videos with Imagen 3 and Veo 2

Evaluating and Debugging Non-Deterministic AI Agents

What is Gemini in BigQuery?

How to host DeepSeek with Cloud Run GPUs in 3 steps

Memory in AI agents

Supercharge your AI/ML: Designing effective GCP storage

Build an app by just talking about it with Gemini!

Watch Gemini judge the Quick Build game show!

Why use GKE to deploy models?

Platform engineering with Cloud Run

Conversational vs non-conversational AI agents

Build a better AI model by fine tuning Gemini

Fine tuning Gemini with Google AI Studio

How Gemini helps predict wildfire behavior

How a cloud architect predicts wildfire behavior with Google Cloud

How a startup founder uses Google Cloud to predict wildfire behavior

How a software engineer predicts wildfire behavior with Google Cloud

Cloud Load Balancer, the secret to uptime for AI inference

The secret to AI training at Google scale

Develop with AI Hypercomputer

Cost-efficient AI inference

Cluster toolkit: The secret to making AI setup easy

Hypercomputer Clusters: The secret to AI training at Google scale

The secret to cost-efficient AI inference

How Improving Aviation uses Gemini and Google Cloud to predict and combat wildfires

Understanding and unifying Looker and Looker Studio

Agentic AI: Workflows vs. agents

Fine-tuning use cases

Connect your language model with external systems and APIs

Function calling with the Gemini API

Looker Reports: an end-to-end demo

Gemini 2.0 and the Gen AI SDK: A powerful duo

Kubernetes jobs for batch workload

Austin & Seattle- rsvp for our in-person events!

Level up your Google Colab notebooks

How to use widgets and forms in Google Colab

Build compelling data stories with Looker reports