Enables calls to the Google Cloud's Vertex AI API to access Large Language Models.

To use, you will need to have one of the following authentication methods in place:

  • You are logged into an account permitted to the Google Cloud project using Vertex AI.
  • You are running this on a machine using a service account permitted to the Google Cloud project using Vertex AI.
  • The GOOGLE_APPLICATION_CREDENTIALS environment variable is set to the path of a credentials file for a service account permitted to the Google Cloud project using Vertex AI.
const model = new GoogleVertexAI({
temperature: 0.7,
});
const stream = await model.stream(
"What would be a good company name for a company that makes colorful socks?",
);
for await (const chunk of stream) {
console.log(chunk);
}

Hierarchy

  • BaseGoogleVertexAI<GoogleAuthOptions>
    • GoogleVertexAI

Constructors

Properties

maxOutputTokens: number = 1024

Maximum number of tokens to generate in the completion.

model: string = "text-bison"

Model to use

temperature: number = 0.7

Sampling temperature to use

topK: number = 40

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

topP: number = 0.8

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

connection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAILLMInstance, TextPrediction, GoogleAuthOptions<JSONClient>>
streamedConnection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAILLMInstance, TextPrediction, GoogleAuthOptions<JSONClient>>

Methods

  • Extracts the prediction from the API response.

    Parameters

    • result: GoogleVertexAILLMResponse<TextPrediction>

      The API response from which to extract the prediction.

    Returns TextPrediction

    A TextPrediction object representing the extracted prediction.

  • Formats the input instance for the Google Vertex AI model based on the model type (text or code).

    Parameters

    • prompt: string

      Prompt to be formatted as an instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted instance.

  • Formats the input instance as a code instance for the Google Vertex AI model.

    Parameters

    • prompt: string

      Prompt to be formatted as a code instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted code instance.

  • Formats the input instance as a text instance for the Google Vertex AI model.

    Parameters

    • prompt: string

      Prompt to be formatted as a text instance.

    Returns GoogleVertexAILLMInstance

    A GoogleVertexAILLMInstance object representing the formatted text instance.

""