This video demonstrates how to connect your AI agent, built with the Agent Development Kit (ADK), to a powerful, GPU accelerated Large Language Model (LLM) hosted on Google Cloud Run. Following up on our previous episode where we deployed Gemma, this installment shows how to decouple your LLM "brain" from your agent for independent scaling. We will guide you through the `agent.py` code, the use of LiteLlm for unified model interfaces, and the deployment of the lightweight ADK agent service. Learn how environment variables facilitate seamless communication between these services, bringing your AI agent to life.
Chapters:
0:00 - Introduction: Connecting agent to LLM
0:53 - Building the agent: `agent.py` and LiteLlm
1:06 - Configuring the agent model parameter
1:35 - Deploying the ADK agent service
1:58 - Agent-LLM communication via environment variables
2:16 - Testing the AI agent in the web UI
2:52 - Conclusion
Resources:
Codelab → http://goo.gle/475sUpV
GitHub repository → http://goo.gle/3KJVc1Y
Google Cloud Run GPU → http://goo.gle/48sn3NV
ADK documentation → http://goo.gle/3LauFL8
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#GoogleCloud #LLM #CloudRun #ADK
Speakers: Amit Maraj
Products Mentioned: Cloud GPUs, Cloud Run