In LangChain4j, while you can configure a ChatModel with default parameters during its creation, there are scenarios where you need to override these settings for a specific interaction. The following method of ChatModel provides this flexibility.

ChatResponse chat(ChatRequest chatRequest)

Why use ChatRequest?

The ChatRequest object acts as a container for both the conversation history (messages) and the generation parameters. It is essential for:

Definition of ChatRequest

 package dev.langchain4j.model.chat.request;
 ...
 public class ChatRequest {
     private final List<ChatMessage> messages;
     private final ChatRequestParameters parameters;
     ...
 }

Definition of ChatRequestParameters

 package dev.langchain4j.model.chat.request;
 public interface ChatRequestParameters {
     String modelName();
     Double temperature();
     Double topP();
     Integer topK();
     Double frequencyPenalty();
     Double presencePenalty();
     Integer maxOutputTokens();
     List<String> stopSequences();
     List<ToolSpecification> toolSpecifications();
     ToolChoice toolChoice();
     ResponseFormat responseFormat();
     ChatRequestParameters overrideWith(ChatRequestParameters parameters); 
     default ChatRequestParameters defaultedBy(ChatRequestParameters parameters); 
     static DefaultChatRequestParameters.Builder<?> builder();
 }

ChatRequest not only helps overriding model parameters, it is also used to specify toolSpecifications, stopSequences, or responseFormat (like JSON) dynamically. It also helps with model switching via modelName method.

Example

The following example demonstrates how to build a ChatRequest to override the default temperature and set a limit on the response length using the phi3:mini-128k model via ollama.

package com.logicbig.example;

import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.ollama.OllamaChatModel;

public class ChatRequestExample {

    public static void main(String[] args) {
        ChatModel model = OllamaChatModel.builder()
                                         .baseUrl("http://localhost:11434")
                                         .modelName("phi3:mini-128k")
                                         .numCtx(4096)
                                         .temperature(0.0)
                                         .build();

        ChatRequest chatRequest = ChatRequest.builder()
                                             .messages(UserMessage.from("What is Java?"))
                                             .temperature(0.7)
                                             .maxOutputTokens(50)
                                             .build();

        ChatResponse response = model.chat(chatRequest);

        System.out.println("Response: " + response.aiMessage().text());
    }
}

Output

Response: Java is a high-level, class-based, object-oriented programming language. It was designed to have few of the arcane details associated with imperative programming languages such as C and Pascal, which also happens to be highly popular for client

Conclusion

By using ChatRequest, we successfully controlled the LLM behavior on a per-call basis. As seen in the output, the response respects the constraints defined in our builder, demonstrating that per-request configuration effectively overrides the model's default settings.

Example Project

Dependencies and Technologies Used:

• langchain4j 1.10.0 (Build LLM-powered applications in Java: chatbots, agents, RAG, and much more)
  
• langchain4j-ollama 1.10.0 (LangChain4j :: Integration :: Ollama)
  
• JDK 17
• Maven 3.9.11