Introduction to Structured Outputs

Structured outputs allow Large Language Models (LLMs) to return data in a predictable, machine-readable format like JSON. This is essential when you need to extract specific information from unstructured text and use it in your Java application.

LangChain4j provides a ChatModel API that supports JSON Schema specification, enabling you to define exactly what structure the LLM should return.

Why JSON Schema?

When working with LLMs, you often need to extract specific data points from text. For example, you might want to extract person information from a paragraph. Without structured output, the LLM might return free-form text that's difficult to parse programmatically. JSON Schema solves this by:

• Defining the exact JSON structure expected
• Ensuring type safety for extracted data
• Making responses predictable and easy to map to Java objects
• Reducing parsing errors and manual text processing

Key Classes

The main classes you'll work with are:

JsonSchema jsonSchema = JsonSchema.builder()
     .name("Coordinate")
     .rootElement(JsonObjectSchema.builder()
         .addNumberProperty("lat")
         .addNumberProperty("lng")
         .required("lat", "lng")
         .build())
     .build();

ResponseFormat responseFormat = ResponseFormat.builder()
    .type(ResponseFormatType.JSON)
    .jsonSchema(jsonSchema)
    .build();

ChatRequest chatRequest = ChatRequest.builder()
    .responseFormat(responseFormat)
    .messages(userMessage)
    .build();

ChatResponse chatResponse = chatModel.chat(chatRequest);

Supported LLM Providers

JSON Schema for structured outputs is currently supported by:

• OpenAI (GPT-4o, GPT-4, GPT-3.5-turbo)
• Azure OpenAI
• Google AI Gemini
• Mistral AI
• Ollama (with supported models)

Example: Extracting Person Information

Let's create a complete example that extracts structured person data from unstructured text. We'll:

1. Define a Person record
2. Create a JSON Schema matching the record
3. Configure a ChatModel
4. Send a request and parse the JSON response

package com.logicbig.example;

public record Person(
        String name,
        int age,
        double height,
        boolean married,
        EmploymentStatus employmentStatus) {}

package com.logicbig.example;

import java.util.Arrays;
import java.util.List;

public enum EmploymentStatus {
    EMPLOYED,
    UNEMPLOYED,
    SELF_EMPLOYED;

    public static List<String> toStringList() {
        return Arrays.stream(EmploymentStatus.values())
                     .map(Enum::name)
                     .toList();
    }
}

package com.logicbig.example;

import com.fasterxml.jackson.databind.ObjectMapper;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.request.ResponseFormat;
import dev.langchain4j.model.chat.request.ResponseFormatType;
import dev.langchain4j.model.chat.request.json.JsonObjectSchema;
import dev.langchain4j.model.chat.request.json.JsonSchema;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.ollama.OllamaChatModel;

public class PersonExtractionExample {
    public static void main(String[] args) {

        ChatModel chatModel = OllamaChatModel.builder()
                                             .baseUrl("http://localhost:11434")
                                             .modelName("phi3:mini-128k")
                                             .logRequests(true)
                                             .logResponses(true)
                                             .build();

        // Define JSON Schema for Person
        JsonSchema jsonSchema =
                JsonSchema.builder()
                          .name("Person")
                          .rootElement(JsonObjectSchema
                                               .builder()
                                               .addStringProperty("name")
                                               .addIntegerProperty("age")
                                               .addNumberProperty("height")
                                               .addBooleanProperty("married")
                                               .addEnumProperty("employmentStatus",
                                                                EmploymentStatus.toStringList())
                                               .required("name",
                                                         "age",
                                                         "height",
                                                         "married",
                                                         "employmentStatus")
                                               .build())
                          .build();

        // Create response format with JSON Schema
        ResponseFormat responseFormat = ResponseFormat.builder()
                                                      .type(ResponseFormatType.JSON)
                                                      .jsonSchema(jsonSchema)
                                                      .build();

        // Create user message with text to analyze
        String text = """
                John is 42 years old. 
                   He stands 1.75 meters tall and carries himself with confidence. 
                   He is employed in a local company as a Software Engineer. 
                   He's married with 2 kids.
                """;

        // Create chat request
        ChatRequest chatRequest =
                ChatRequest.builder()
                           .responseFormat(responseFormat)
                           .messages(UserMessage.from(text))
                           .build();

        // Send request and get response
        ChatResponse chatResponse = chatModel.chat(chatRequest);

        // Extract and parse JSON response
        String jsonResponse = chatResponse.aiMessage().text();
        System.out.println("JSON Response: " + jsonResponse);

        // Parse JSON to Person object
        ObjectMapper mapper = new ObjectMapper();
        try {
            Person person = mapper.readValue(jsonResponse, Person.class);
            System.out.println(person);
        } catch (Exception e) {
            System.err.println("Error parsing JSON: " + e.getMessage());
        }
    }
}

Output

JSON Response: {
    "name": "John",
    "age": 42,
    "height": 1.75,
    "married" : true,
    "employmentStatus":"SELF_EMPLOYED"
}
Person[name=John, age=42, height=1.75, married=true, employmentStatus=SELF_EMPLOYED]

How It Works

The JsonSchema defines the exact structure we expect from the LLM. When we send this schema along with our request, the LLM attempts to extract matching information from the provided text and returns it in the specified JSON format.

Key points in the example:

• ResponseFormat.builder().type(JSON) tells the LLM to return JSON
• JsonObjectSchema defines an object with specific properties
• .required() specifies which fields must be present
• The LLM's JSON response is parsed directly into our Person record

Understanding the Output

The example demonstrates how unstructured text about John is transformed into structured data. The LLM identifies:

• name: "John" from the text
• age: 42 from "42 years old"
• height: 1.75 from "1.75 meters tall"
• married: false from "Currently unmarried"

Working with Different Data Types

Here's how to define different property types in your JSON Schema:

package com.logicbig.example;

import com.fasterxml.jackson.databind.ObjectMapper;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.request.ResponseFormat;
import dev.langchain4j.model.chat.request.ResponseFormatType;
import dev.langchain4j.model.chat.request.json.*;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.ollama.OllamaChatModel;

public class SchemaTypesExample {
    public static void main(String[] args) {
        ChatModel chatModel = OllamaChatModel.builder()
                                             .baseUrl("http://localhost:11434")
                                             .modelName("phi3:mini-128k")
                                             .logRequests(true)
                                             .logResponses(true)
                                             .build();

        // Example 1: String schema with description
        JsonSchemaElement stringSchema = JsonStringSchema.builder()
            .description("The full name of the person")
            .build();

        // Example 2: Integer schema
        JsonSchemaElement integerSchema = JsonIntegerSchema.builder()
            .description("Age in years")
            .build();

        // Example 3: Number schema for decimal values
        JsonSchemaElement numberSchema = JsonNumberSchema.builder()
            .description("Height in meters")
            .build();

        // Example 4: Boolean schema
        JsonSchemaElement booleanSchema = JsonBooleanSchema.builder()
            .description("Marital status")
            .build();

        // Example 5: Enum schema
        JsonSchemaElement enumSchema = JsonEnumSchema.builder()
            .description("Employment status")
            .enumValues(EmploymentStatus.toStringList())
            .build();

        // Example 6: Complete object schema
        JsonSchemaElement personSchema = JsonObjectSchema.builder()
            .addProperty("name", stringSchema)
            .addProperty("age", integerSchema)
            .addProperty("height", numberSchema)
            .addProperty("married", booleanSchema)
            .addProperty("employmentStatus", enumSchema)
            .required("name", "age")
            .build();

        // Create JSON Schema
        JsonSchema jsonSchema = JsonSchema.builder()
            .name("DetailedPerson")
            .rootElement(personSchema)
            .build();

        // Create response format
        ResponseFormat responseFormat = ResponseFormat.builder()
            .type(ResponseFormatType.JSON)
            .jsonSchema(jsonSchema)
            .build();

        String text = """
                John is 42 years old and lives an independent life. 
                He stands 1.75 meters tall. He has a his own business and works from home. 
                Currently unmarried he enjoys the freedom to focus on his personal goals.
                """;

        // Create chat request
        ChatRequest chatRequest =
                ChatRequest.builder()
                           .responseFormat(responseFormat)
                           .messages(UserMessage.from(text))
                           .build();

        // Send request and get response
        ChatResponse chatResponse = chatModel.chat(chatRequest);

        // Extract and parse JSON response
        String jsonResponse = chatResponse.aiMessage().text();
        System.out.println("JSON Response: " + jsonResponse);

        // Parse JSON to Person object
        ObjectMapper mapper = new ObjectMapper();
        try {
            Person person = mapper.readValue(jsonResponse, Person.class);
            System.out.println(person);
        } catch (Exception e) {
            System.err.println("Error parsing JSON: " + e.getMessage());
        }

    }
}

Output

JSON Response: {
   "name": "John",
   "age": 42,
   "height": 175,
   "employmentStatus": "SELF_EMPLOYED"
}

Person[name=John, age=42, height=175.0, married=false, employmentStatus=SELF_EMPLOYED]

Conclusion

In this tutorial, we have demonstrated how to use LangChain4j's ChatModel API with JSON Schema to extract structured data from unstructured text. The output shows that the LLM successfully extracted all required fields from the input text and returned them in the exact JSON structure we specified. This approach provides type-safe, predictable results that can be directly mapped to Java objects, eliminating the need for error-prone text parsing.

JSON Schema with ChatModel is the most reliable method for structured outputs when working with supported LLM providers, as it gives you complete control over the response format and ensures data consistency.

Example Project

Dependencies and Technologies Used:

• langchain4j 1.10.0 (Build LLM-powered applications in Java: chatbots, agents, RAG, and much more)
  
• langchain4j-ollama 1.10.0 (LangChain4j :: Integration :: Ollama)
  
• jackson-databind 2.18.2 (General data-binding functionality for Jackson: works on core streaming API)
  
• JDK 17
• Maven 3.9.11