Build a GenAI App With Java Using Spring AI and Docker Model Runner

docker model pull ai/llama3.1
docker model pull ai/mxbai-embed-large

@RestController
public class ChatController {

	private final ChatClient chatClient;

	public ChatController(ChatModel chatModel) {
		this.chatClient = ChatClient.builder(chatModel).build();
	}

	@GetMapping("/chat")
	public String generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
		return this.chatClient.prompt().user(message).call().content();
	}

}

spring.ai.openai.base-url=http://localhost:12434/engines
spring.ai.openai.api-key=test
spring.ai.openai.chat.options.model=ai/llama3.1

http :8080/chat message=="What’s testcontainers?"

Testcontainers is a fantastic and increasingly popular library for **local testing with containers**. Let's break down what it is, why it's useful, and how it works:

**What is Testcontainers?**

Testcontainers provides a way to run real, fully functional containerized services (like databases, message queues, web servers, etc.) directly within your tests. Instead of relying on mocked or stubbed versions of these services, you're using the *actual* services, which leads to much more realistic and reliable test results.

**Why Use Testcontainers?**

* **Realistic Testing:** This is the biggest benefit.  Mocking databases or message queues can be brittle and difficult to maintain.  Testcontainers provides a service that behaves exactly like the real thing, leading to tests that more accurately reflect how your application will perform in production.
* **Simplified Test Setup:**  Forget about manually setting up and configuring databases or other services on your test machine. Testcontainers automatically handles the container creation, configuration, and cleanup for you.
* **Faster Tests:** Because the services are running locally, there’s no network latency involved, resulting in significantly faster test execution times.
* **Consistent Environments:**  You eliminate the "it works on my machine" problem. Everyone running the tests will be using the same, pre-configured environment.
* **Supports Many Services:** Testcontainers supports a huge range of services, including:
    * **Databases:** PostgreSQL, MySQL, MongoDB, Redis, Cassandra, MariaDB
    * **Message Queues:** RabbitMQ, Kafka, ActiveMQ
    * **Web Servers:**  Tomcat, Jetty, H2 (for in-memory databases)
    * **And many more!**  The list is constantly growing.


**How Does It Work?**

1. **Client Library:** Testcontainers provides client libraries for various programming languages (Java, Python, JavaScript, Ruby, Go, .NET, and more).
2. **Container Run:** When you use the Testcontainers client library in your test, it automatically starts the specified container (e.g., a PostgreSQL database) in the background.
3. **Connection:** Your test code then connects to the running container using standard protocols (e.g., JDBC for PostgreSQL, HTTP for a web server).
4. **Test Execution:**  You execute your tests as usual.
5. **Cleanup:**  When the tests are finished, Testcontainers automatically shuts down the container, ensuring a clean state for the next test run.

**Example (Conceptual - Python):**

```python
from testcontainers.postgresql import PostgreSQLEnvironment

# Create a PostgreSQL environment
env = PostgreSQLEnvironment()

# Start the container
env.start()

# Connect to the database
db = env.db()  #  This creates a connection object to the running PostgreSQL container

# Perform database operations in your test
# ...

# Stop the container (cleanup)
env.shutdown()
```

**Key Concepts:**

* **Environment:**  A Testcontainers environment is a configuration that defines which containers to run and how they should be configured.
* **Container:**  A running containerized service (e.g., a database instance).
* **Connection:** An object that represents a connection to a specific container.

**Resources to Learn More:**

* **Official Website:** [https://testcontainers.io/](https://testcontainers.io/) - This is the best place to start.
* **GitHub Repository:** [https://github.com/testcontainers/testcontainers](https://github.com/testcontainers/testcontainers) -  See the source code and contribute.
* **Documentation:** [https://testcontainers.io/docs/](https://testcontainers.io/docs/) - Comprehensive documentation with examples for various languages.

**In short, Testcontainers is a powerful tool that dramatically improves the quality and reliability of your local tests by allowing you to test against real, running containerized services.**

Do you want me to delve deeper into a specific aspect of Testcontainers, such as:

*   A specific language implementation (e.g., Python)?
*   A particular service it supports (e.g., PostgreSQL)?
*   How to integrate it with a specific testing framework (e.g., JUnit, pytest)?

@RestController
public class RagController {

	private final ChatClient chatClient;

	private final VectorStore vectorStore;

	public RagController(ChatModel chatModel, VectorStore vectorStore) {
		this.chatClient = ChatClient.builder(chatModel).build();
		this.vectorStore = vectorStore;
	}

	@GetMapping("/rag")
	public String generate(@RequestParam(value = "message", defaultValue = "What's Testcontainers?") String message) {
		return callResponseSpec(this.chatClient, this.vectorStore, message).content();
	}

	static ChatClient.CallResponseSpec callResponseSpec(ChatClient chatClient, VectorStore vectorStore,
			String question) {
		QuestionAnswerAdvisor questionAnswerAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
			.searchRequest(SearchRequest.builder().topK(1).build())
			.build();
		return chatClient.prompt().advisors(questionAnswerAdvisor).user(question).call();
	}

}

@TestConfiguration(proxyBeanMethods = false)
public class IngestionConfiguration {

	@Value("classpath:/docs/testcontainers.txt")
	private Resource testcontainersDoc;

	@Bean
	ApplicationRunner init(VectorStore vectorStore) {
		return args -> {
			var javaTextReader = new TextReader(this.testcontainersDoc);
			javaTextReader.getCustomMetadata().put("language", "java");

			var tokenTextSplitter = new TokenTextSplitter();
			var testcontainersDocuments = tokenTextSplitter.apply(javaTextReader.get());

			vectorStore.add(testcontainersDocuments);
		};
	}

}

Testcontainers is a library that provides easy and lightweight APIs for bootstrapping local development and test dependencies with real services wrapped in Docker containers. Using Testcontainers, you can write tests that depend on the same services you use in production without mocks or in-memory services.

Testcontainers provides modules for a wide range of commonly used infrastructure dependencies including relational databases, NoSQL datastores, search engines, message brokers, etc. See https://testcontainers.com/modules/ for a complete list.

Technology-specific modules are a higher-level abstraction on top of GenericContainer which help configure and run these technologies without any boilerplate, and make it easy to access their relevant parameters.

Official website: https://testcontainers.com/
Getting Started: https://testcontainers.com/getting-started/
Module Catalog: https://testcontainers.com/modules/

spring.ai.openai.embedding.options.model=ai/mxbai-embed-large
spring.ai.vectorstore.qdrant.initialize-schema=true
spring.ai.vectorstore.qdrant.collection-name=test

public class TestDemoApplication {

	public static void main(String[] args) {
		SpringApplication.from(DemoApplication::main)
			.with(TestcontainersConfiguration.class, IngestionConfiguration.class)
			.run(args);
	}

}

http :8080/rag message=="What’s testcontainers?"

Testcontainers is a library that helps you write tests for your applications by bootstrapping real services in Docker containers, rather than using mocks or in-memory services. This allows you to test your applications as they would run in production, but in a controlled and isolated environment.

It provides modules for commonly used infrastructure dependencies such as relational databases, NoSQL datastores, search engines, and message brokers.

If you have any specific questions about how to use Testcontainers or its features, I'd be happy to help.

@SpringBootTest(classes = { TestcontainersConfiguration.class, IngestionConfiguration.class },
		webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class RagControllerTest {

	@LocalServerPort
	private int port;

	@Autowired
	private VectorStore vectorStore;

	@Autowired
	private ChatClient.Builder chatClientBuilder;

	@Test
	void verifyTestcontainersAnswer() {
		var question = "Tell me about Testcontainers";
		var answer = retrieveAnswer(question);

		assertFactCheck(question, answer);
	}

	private String retrieveAnswer(String question) {
		RestClient restClient = RestClient.builder().baseUrl("http://localhost:%d".formatted(this.port)).build();
		return restClient.get().uri("/rag?message={question}", question).retrieve().body(String.class);
	}

	private void assertFactCheck(String question, String answer) {
		FactCheckingEvaluator factCheckingEvaluator = new FactCheckingEvaluator(this.chatClientBuilder);
		EvaluationResponse evaluate = factCheckingEvaluator.evaluate(new EvaluationRequest(docs(question), answer));
		assertThat(evaluate.isPass()).isTrue();
	}

	private List<Document> docs(String question) {
		var response = RagController
			.callResponseSpec(this.chatClientBuilder.build(), this.vectorStore, question)
			.chatResponse();
		return response.getMetadata().get(QuestionAnswerAdvisor.RETRIEVED_DOCUMENTS);
	}

}

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
	<groupId>io.micrometer</groupId>
	<artifactId>micrometer-registry-otlp</artifactId>
</dependency>
<dependency>
	<groupId>io.micrometer</groupId>
	<artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
	<groupId>io.opentelemetry</groupId>
	<artifactId>opentelemetry-exporter-otlp</artifactId>
</dependency>
<dependency>
	<groupId>org.testcontainers</groupId>
	<artifactId>grafana</artifactId>
       <scope>test</scope>
</dependency>

@TestConfiguration(proxyBeanMethods = false)
public class GrafanaContainerConfiguration {

	@Bean
	@ServiceConnection
	LgtmStackContainer lgtmContainer() {
		return new LgtmStackContainer("grafana/otel-lgtm:0.11.4");
	}

}

spring.application.name=demo
management.tracing.sampling.probability=1

public class TestDemoApplication {

	public static void main(String[] args) {
		SpringApplication.from(DemoApplication::main)
			.with(TestcontainersConfiguration.class, IngestionConfiguration.class, GrafanaContainerConfiguration.class)
			.run(args);
	}

}

http :8080/rag message=="What’s testcontainers?"

o.t.grafana.LgtmStackContainer           : Access to the Grafana dashboard: 
http://localhost:64908