# ChatVertexAI > **Class** in `langchain_google_vertexai` 📖 [View in docs](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI) Google Cloud Vertex AI chat model integration. ## Signature ```python ChatVertexAI( self, *, model_name: str | None = None, **kwargs: Any = {}, ) ``` ## Description **Setup:** You must either: - Have credentials configured for your environment (gcloud, workload identity, etc...) - Store the path to a service account JSON file as the `GOOGLE_APPLICATION_CREDENTIALS` environment variable This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth. **More information:** - [Credential types](https://cloud.google.com/docs/authentication/application-default-credentials#GAC) - `google.auth` [API reference](https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth) Key init args — completion params: model: str Name of ChatVertexAI model to use. e.g. `'gemini-2.5-flash'`, `'gemini-2.5-pro'`, etc. temperature: Optional[float] Sampling temperature. seed: Optional[int] Sampling integer to use. max_tokens: Optional[int] Max number of tokens to generate. stop: Optional[List[str]] Default stop sequences. safety_settings: Optional[Dict[vertexai.generative_models.HarmCategory, vertexai.generative_models.HarmBlockThreshold]] The default safety settings to use for all generations. Key init args — client params: max_retries: int Max number of retries. wait_exponential_kwargs: Optional[dict[str, float]] Optional dictionary with parameters for wait_exponential: - multiplier: Initial wait time multiplier (default: `1.0`) - min: Minimum wait time in seconds (default: `4.0`) - max: Maximum wait time in seconds (default: `10.0`) - exp_base: Exponent base to use (default: `2.0`) credentials: Optional[google.auth.credentials.Credentials] The default custom credentials to use when making API calls. If not provided, credentials will be ascertained from the environment. project: Optional[str] The default GCP project to use when making Vertex API calls. location: str = "us-central1" The default location to use when making API calls. request_parallelism: int = 5 The amount of parallelism allowed for requests issued to VertexAI models. base_url: Optional[str] Base URL for API requests. See full list of supported init args and their descriptions in the params section. **Instantiate:** ```python from langchain_google_vertexai import ChatVertexAI llm = ChatVertexAI( model="gemini-2.5-flash", temperature=0, max_tokens=None, max_retries=6, stop=None, # other params... ) ``` **Thinking:** For thinking models, you have the option to adjust the number of internal thinking tokens used (`thinking_budget`) or to disable thinking altogether. Note that not all models allow disabling thinking. See the [Gemini API docs](https://ai.google.dev/gemini-api/docs/thinking) for more details on thinking models. To see a thinking model's thoughts, set `include_thoughts=True` to have the model's reasoning summaries included in the response. ```python llm = ChatVertexAI( model="gemini-2.5-flash", include_thoughts=True, ) ai_msg = llm.invoke("How many 'r's are in the word 'strawberry'?") ``` **Invoke:** ```python messages = [ ( "system", "You are a helpful translator. Translate the user sentence to French.", ), ("human", "I love programming."), ] llm.invoke(messages) ``` ```python AIMessage( content="J'adore programmer. ", response_metadata={ "is_blocked": False, "safety_ratings": [ { "category": "HARM_CATEGORY_HATE_SPEECH", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_HARASSMENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, ], "citation_metadata": None, "usage_metadata": { "prompt_token_count": 17, "candidates_token_count": 7, "total_token_count": 24, }, }, id="run-925ce305-2268-44c4-875f-dde9128520ad-0", ) ``` **Stream:** ```python for chunk in llm.stream(messages): print(chunk) ``` ```python AIMessageChunk( content="J", response_metadata={ "is_blocked": False, "safety_ratings": [], "citation_metadata": None, }, id="run-9df01d73-84d9-42db-9d6b-b1466a019e89", ) AIMessageChunk( content="'adore programmer. ", response_metadata={ "is_blocked": False, "safety_ratings": [ { "category": "HARM_CATEGORY_HATE_SPEECH", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_HARASSMENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, ], "citation_metadata": None, }, id="run-9df01d73-84d9-42db-9d6b-b1466a019e89", ) AIMessageChunk( content="", response_metadata={ "is_blocked": False, "safety_ratings": [], "citation_metadata": None, "usage_metadata": { "prompt_token_count": 17, "candidates_token_count": 7, "total_token_count": 24, }, }, id="run-9df01d73-84d9-42db-9d6b-b1466a019e89", ) ``` ```python stream = llm.stream(messages) full = next(stream) for chunk in stream: full += chunk full ``` ```python AIMessageChunk( content="J'adore programmer. ", response_metadata={ "is_blocked": False, "safety_ratings": [ { "category": "HARM_CATEGORY_HATE_SPEECH", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_HARASSMENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, ], "citation_metadata": None, "usage_metadata": { "prompt_token_count": 17, "candidates_token_count": 7, "total_token_count": 24, }, }, id="run-b7f7492c-4cb5-42d0-8fc3-dce9b293b0fb", ) ``` **Async invocation:** ```python await llm.ainvoke(messages) # stream async for chunk in (await llm.astream(messages)) # batch await llm.abatch([messages]) ``` **Context Caching:** Context caching allows you to store and reuse content (e.g., PDFs, images) for faster processing. The `cached_content` parameter accepts a cache name created via the Google Generative AI API with Vertex AI. !!! example "Content caching" This caches content from GCS and queries it. ```python from google import genai from google.genai.types import ( Content, CreateCachedContentConfig, HttpOptions, Part, ) from langchain_google_vertexai import ChatVertexAI from langchain_core.messages import HumanMessage client = genai.Client(http_options=HttpOptions(api_version="v1beta1")) contents = [ Content( role="user", parts=[ Part.from_uri( file_uri="gs://your-bucket/file1", mime_type="application/pdf", ), Part.from_uri( file_uri="gs://your-bucket/file2", mime_type="image/jpeg", ), ], ) ] cache = client.caches.create( model="gemini-2.5-flash", config=CreateCachedContentConfig( contents=contents, system_instruction="You are an expert content analyzer.", display_name="content-cache", ttl="300s", ), ) llm = ChatVertexAI( model="gemini-2.5-flash", cached_content=cache.name, ) message = HumanMessage( content="Provide a summary of the key information across the content." ) llm.invoke([message]) ``` **Tool calling:** ```python from pydantic import BaseModel, Field class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field( ..., description="The city and state, e.g. San Francisco, CA" ) class GetPopulation(BaseModel): '''Get the current population in a given location''' location: str = Field( ..., description="The city and state, e.g. San Francisco, CA" ) llm_with_tools = llm.bind_tools([GetWeather, GetPopulation]) ai_msg = llm_with_tools.invoke( "Which city is hotter today and which is bigger: LA or NY?" ) ai_msg.tool_calls ``` ```python [ { "name": "GetWeather", "args": {"location": "Los Angeles, CA"}, "id": "2a2401fa-40db-470d-83ce-4e52de910d9e", }, { "name": "GetWeather", "args": {"location": "New York City, NY"}, "id": "96761deb-ab7f-4ef9-b4b4-6d44562fc46e", }, { "name": "GetPopulation", "args": {"location": "Los Angeles, CA"}, "id": "9147d532-abee-43a2-adb5-12f164300484", }, { "name": "GetPopulation", "args": {"location": "New York City, NY"}, "id": "c43374ea-bde5-49ca-8487-5b83ebeea1e6", }, ] ``` See `bind_tools` for more. **Built-in search:** ```python from google.cloud.aiplatform_v1beta1.types import Tool as VertexTool from langchain_google_vertexai import ChatVertexAI llm = ChatVertexAI(model="gemini-2.5-flash") resp = llm.invoke( "When is the next total solar eclipse in US?", tools=[VertexTool(google_search={})], ) ``` **Built-in code execution:** ```python from google.cloud.aiplatform_v1beta1.types import Tool as VertexTool from langchain_google_vertexai import ChatVertexAI llm = ChatVertexAI(model="gemini-2.5-flash") resp = llm.invoke( "What is 3^3?", tools=[VertexTool(code_execution={})], ) ``` **Structured output:** ```python from typing import Optional from pydantic import BaseModel, Field class Joke(BaseModel): '''Joke to tell user.''' setup: str = Field(description="The setup of the joke") punchline: str = Field(description="The punchline to the joke") rating: Optional[int] = Field( default=None, description="How funny the joke is, from 1 to 10" ) structured_llm = llm.with_structured_output(Joke) structured_llm.invoke("Tell me a joke about cats") ``` ```python Joke( setup="What do you call a cat that loves to bowl?", punchline="An alley cat!", rating=None, ) ``` See `with_structured_output` for more. **Image input:** ```python import base64 import httpx from langchain_core.messages import HumanMessage image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8") message = HumanMessage( content=[ {"type": "text", "text": "describe the weather in this image"}, { "type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}, }, ], ) ai_msg = llm.invoke([message]) ai_msg.content ``` ```txt The weather in this image appears to be sunny and pleasant. The sky is a bright blue with scattered white clouds, suggesting a clear and mild day. The lush green grass indicates recent rainfall or sufficient moisture. The absence of... ``` You can also point to GCS files which is faster / more efficient because bytes are transferred back and forth. ```python llm.invoke( [ HumanMessage( [ "What's in the image?", { "type": "media", "file_uri": "gs://cloud-samples-data/generative-ai/image/scones.jpg", "mime_type": "image/jpeg", }, ] ) ] ).content ``` ```txt The image is of five blueberry scones arranged on a piece of baking paper. Here is a list of what is in the picture:* **Five blueberry scones:** They are scattered across the parchment paper, dusted with powdered sugar. * **Two... ``` **PDF input:** ```python import base64 from langchain_core.messages import HumanMessage pdf_bytes = open("/path/to/your/test.pdf", "rb").read() pdf_base64 = base64.b64encode(pdf_bytes).decode("utf-8") message = HumanMessage( content=[ {"type": "text", "text": "describe the document in a sentence"}, { "type": "file", "mime_type": "application/pdf", "base64": pdf_base64, }, ] ) ai_msg = llm.invoke([message]) ai_msg.content ``` ```txt This research paper describes a system developed for SemEval-2025 Task 9, which aims to automate the detection of food hazards from recall reports, addressing the class imbalance problem by leveraging LLM-based data augmentation... ``` You can also point to GCS files. ```python llm.invoke( [ HumanMessage( [ "describe the document in a sentence", { "type": "media", "file_uri": "gs://cloud-samples-data/generative-ai/pdf/1706.03762v7.pdf", "mime_type": "application/pdf", }, ] ) ] ).content ``` ```txt The article introduces Transformer, a new model architecture for sequence transduction based solely on attention mechanisms, outperforming previous models in machine translation tasks and demonstrating good generalization to English... ``` **Video input:** ```python import base64 from langchain_core.messages import HumanMessage video_bytes = open("/path/to/your/video.mp4", "rb").read() video_base64 = base64.b64encode(video_bytes).decode("utf-8") message = HumanMessage( content=[ { "type": "text", "text": "describe what's in this video in a sentence", }, { "type": "file", "mime_type": "video/mp4", "base64": video_base64, }, ] ) ai_msg = llm.invoke([message]) ai_msg.content ``` ```txt Tom and Jerry, along with a turkey, engage in a chaotic Thanksgiving-themed adventure involving a corn-on-the-cob chase, maze antics, and a disastrous attempt to prepare a turkey dinner. ``` You can also pass YouTube URLs directly: ```python from langchain_core.messages import HumanMessage message = HumanMessage( content=[ {"type": "text", "text": "summarize the video in 3 sentences."}, { "type": "media", "file_uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg", "mime_type": "video/mp4", }, ] ) ai_msg = llm.invoke([message]) ai_msg.content ``` ```txt The video is a demo of multimodal live streaming in Gemini 2.0. The narrator is sharing his screen in AI Studio and asks if the AI can see it. The AI then reads text that is highlighted on the screen, defines the word “multimodal,” and... ``` You can also point to GCS files. ```python llm = ChatVertexAI(model="gemini-2.5-pro") llm.invoke( [ HumanMessage( [ "What's in the video?", { "type": "media", "file_uri": "gs://cloud-samples-data/video/animals.mp4", "mime_type": "video/mp4", }, ] ) ] ).content ``` ```txt The video is about a new feature in Google Photos called "Zoomable Selfies". The feature allows users to take selfies with animals at the zoo. The video shows several examples of people taking selfies with animals, including a tiger,... ``` **Audio input:** ```python import base64 from langchain_core.messages import HumanMessage audio_bytes = open("/path/to/your/audio.mp3", "rb").read() audio_base64 = base64.b64encode(audio_bytes).decode("utf-8") message = HumanMessage( content=[ {"type": "text", "text": "summarize this audio in a sentence"}, { "type": "file", "mime_type": "audio/mp3", "base64": audio_base64, }, ] ) ai_msg = llm.invoke([message]) ai_msg.content ``` ```python "In this episode of the Made by Google podcast, Stephen Johnson and Simon Tokumine discuss NotebookLM, a tool designed to help users understand complex material in various modalities, with a focus on its unexpected uses, the development of audio overviews, and the implementation of new features like mind maps and source discovery." ``` You can also point to GCS files. ```python from langchain_core.messages import HumanMessage llm = ChatVertexAI(model="gemini-2.5-flash") llm.invoke( [ HumanMessage( [ "What's this audio about?", { "type": "media", "file_uri": "gs://cloud-samples-data/generative-ai/audio/pixel.mp3", "mime_type": "audio/mpeg", }, ] ) ] ).content ``` ```python "This audio is an interview with two product managers from Google who work on Pixel feature drops. They discuss how feature drops are important for showcasing how Google devices are constantly improving and getting better. They also discuss some of the highlights of the January feature drop and the new features coming in the March drop for Pixel phones and Pixel watches. The interview concludes with discussion of how user feedback is extremely important to them in deciding which features to include in the feature drops." ``` **Token usage:** ```python ai_msg = llm.invoke(messages) ai_msg.usage_metadata ``` ```python {"input_tokens": 17, "output_tokens": 7, "total_tokens": 24} ``` **Logprobs:** ```python llm = ChatVertexAI(model="gemini-2.5-flash", logprobs=True) ai_msg = llm.invoke(messages) ai_msg.response_metadata["logprobs_result"] ``` ```python [ {"token": "J", "logprob": -1.549651415189146e-06, "top_logprobs": []}, {"token": "'", "logprob": -1.549651415189146e-06, "top_logprobs": []}, {"token": "adore", "logprob": 0.0, "top_logprobs": []}, { "token": " programmer", "logprob": -1.1922384146600962e-07, "top_logprobs": [], }, {"token": ".", "logprob": -4.827636439586058e-05, "top_logprobs": []}, {"token": " ", "logprob": -0.018011733889579773, "top_logprobs": []}, {"token": "\\n", "logprob": -0.0008687592926435173, "top_logprobs": []}, ] ``` **Response metadata:** ```python ai_msg = llm.invoke(messages) ai_msg.response_metadata ``` ```python { "is_blocked": False, "safety_ratings": [ { "category": "HARM_CATEGORY_HATE_SPEECH", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_HARASSMENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, ], "usage_metadata": { "prompt_token_count": 17, "candidates_token_count": 7, "total_token_count": 24, }, } ``` **Safety settings:** ```python from langchain_google_vertexai import HarmBlockThreshold, HarmCategory llm = ChatVertexAI( model="gemini-2.5-pro", safety_settings={ HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_ONLY_HIGH, }, ) llm.invoke(messages).response_metadata ``` ```python { "is_blocked": False, "safety_ratings": [ { "category": "HARM_CATEGORY_HATE_SPEECH", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_HARASSMENT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, { "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "probability_label": "NEGLIGIBLE", "probability_score": 0.1, "blocked": False, "severity": "HARM_SEVERITY_NEGLIGIBLE", "severity_score": 0.1, }, ], "usage_metadata": { "prompt_token_count": 17, "candidates_token_count": 7, "total_token_count": 24, }, } ``` ## Extends - `_VertexAICommon` - `BaseChatModel` ## Constructors ```python __init__( self, *, model_name: str | None = None, **kwargs: Any = {}, ) -> None ``` | Name | Type | |------|------| | `model_name` | `str \| None` | ## Properties - `model_name` - `examples` - `response_mime_type` - `response_schema` - `cached_content` - `logprobs` - `labels` - `perform_literal_eval_on_string_raw_content` - `wait_exponential_kwargs` - `model_kwargs` - `model_config` ## Methods - [`is_lc_serializable()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/is_lc_serializable) - [`get_lc_namespace()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/get_lc_namespace) - [`build_extra()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/build_extra) - [`validate_labels()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/validate_labels) - [`validate_environment()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/validate_environment) - [`get_num_tokens()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/get_num_tokens) - [`get_num_tokens_from_messages()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/get_num_tokens_from_messages) - [`with_structured_output()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/with_structured_output) - [`bind_tools()`](https://reference.langchain.com/python/langchain-google-vertexai/chat_models/ChatVertexAI/bind_tools) ## ⚠️ Deprecated Use [`ChatGoogleGenerativeAI`][langchain_google_genai.ChatGoogleGenerativeAI] instead. --- [View source on GitHub](https://github.com/langchain-ai/langchain-google/blob/982e4015b249de8b9ba1e787746d8cc1f6d6b790/libs/vertexai/langchain_google_vertexai/chat_models.py#L917)