Class ChatModel (1.90.0)

ChatModel(model_id: str, endpoint_name: typing.Optional[str] = None)

ChatModel represents a language model that is capable of chat.

Examples::

chat_model = ChatModel.from_pretrained("chat-bison@001")
chat = chat_model.start_chat(
 context="My name is Ned. You are my personal assistant. My favorite movies are Lord of the Rings and Hobbit.",
 examples=[
 InputOutputTextPair(
 input_text="Who do you work for?",
 output_text="I work for Ned.",
 ),
 InputOutputTextPair(
 input_text="What do I like?",
 output_text="Ned likes watching movies.",
 ),
 ],
 temperature=0.3,
)
chat.send_message("Do you know any cool events this weekend?")

Methods

ChatModel

ChatModel(model_id: str, endpoint_name: typing.Optional[str] = None)

Creates a LanguageModel.

This constructor should not be called directly. Use LanguageModel.from_pretrained(model_name=...) instead.

from_pretrained

from_pretrained(model_name: str) -> vertexai._model_garden._model_garden_models.T

Loads a _ModelGardenModel.

Exceptions
Type	Description
`ValueError`	If model_name is unknown.
`ValueError`	If model does not support this class.

get_tuned_model

get_tuned_model(
 tuned_model_name: str,
) -> vertexai.language_models._language_models._LanguageModel

Loads the specified tuned language model.

list_tuned_model_names

list_tuned_model_names() -> typing.Sequence[str]

Lists the names of tuned models.

start_chat

start_chat(
 *,
 context: typing.Optional[str] = None,
 examples: typing.Optional[
 typing.List[vertexai.language_models.InputOutputTextPair]
 ] = None,
 max_output_tokens: typing.Optional[int] = None,
 temperature: typing.Optional[float] = None,
 top_k: typing.Optional[int] = None,
 top_p: typing.Optional[float] = None,
 message_history: typing.Optional[
 typing.List[vertexai.language_models.ChatMessage]
 ] = None,
 stop_sequences: typing.Optional[typing.List[str]] = None
) -> vertexai.language_models.ChatSession

Starts a chat session with the model.

tune_model

tune_model(
 training_data: typing.Union[str, pandas.core.frame.DataFrame],
 *,
 train_steps: typing.Optional[int] = None,
 learning_rate_multiplier: typing.Optional[float] = None,
 tuning_job_location: typing.Optional[str] = None,
 tuned_model_location: typing.Optional[str] = None,
 model_display_name: typing.Optional[str] = None,
 default_context: typing.Optional[str] = None,
 accelerator_type: typing.Optional[typing.Literal["TPU", "GPU"]] = None,
 tuning_evaluation_spec: typing.Optional[
 vertexai.language_models.TuningEvaluationSpec
 ] = None
) -> vertexai.language_models._language_models._LanguageModelTuningJob

Tunes a model based on training data.

This method launches and returns an asynchronous model tuning job. Usage:

tuning_job = model.tune_model(...)
... do some other work
tuned_model = tuning_job.get_tuned_model() # Blocks until tuning is complete

Exceptions
Type	Description
`ValueError`	If the "tuning_job_location" value is not supported
`ValueError`	If the "tuned_model_location" value is not supported
`RuntimeError`	If the model does not support tuning
`AttributeError`	If any attribute in the "tuning_evaluation_spec" is not supported

tune_model_rlhf

tune_model_rlhf(
 *,
 prompt_data: typing.Union[str, pandas.core.frame.DataFrame],
 preference_data: typing.Union[str, pandas.core.frame.DataFrame],
 model_display_name: typing.Optional[str] = None,
 prompt_sequence_length: typing.Optional[int] = None,
 target_sequence_length: typing.Optional[int] = None,
 reward_model_learning_rate_multiplier: typing.Optional[float] = None,
 reinforcement_learning_rate_multiplier: typing.Optional[float] = None,
 reward_model_train_steps: typing.Optional[int] = None,
 reinforcement_learning_train_steps: typing.Optional[int] = None,
 kl_coeff: typing.Optional[float] = None,
 default_context: typing.Optional[str] = None,
 tuning_job_location: typing.Optional[str] = None,
 accelerator_type: typing.Optional[typing.Literal["TPU", "GPU"]] = None,
 tuning_evaluation_spec: typing.Optional[
 vertexai.language_models.TuningEvaluationSpec
 ] = None
) -> vertexai.language_models._language_models._LanguageModelTuningJob

Tunes a model using reinforcement learning from human feedback.