Local Development of Go Applications with Testcontainers

Md

Manuel de la Peña

When building applications, it’s important to have an enjoyable developer experience, regardless of the programming language. This experience includes having a great build tool to perform any task related to the development lifecycle of the project. This includes compiling it, building the release artifacts, and running tests.

Often times our build tool doesn’t support all our local development tasks, such as starting the runtime dependencies for our application. We’re then forced to manage them manually with a Makefile, a shell script, or an external Docker Compose file. This might involve calling them in a separated terminal or even maintaining code for that purpose. Thankfully, there’s a better way.

In this post, I’m going to show you how to use Testcontainers for Go. You’ll learn how to start and stop the runtime dependencies of your Go application while building it and how to run the tests simply and consistently. We’ll build a super simple Go app using the Fiber web framework, which will connect to a PostgreSQL database to store its users. Then, we’ll leverage Go’s built-in capabilities and use Testcontainers for Go to start the dependencies of the application.

Local development of go applications with testcontainers

You can find the source code in the testcontainers-go-fiber repository.

If you’re new to Testcontainers for Go, then watch this video to get started with Testcontainers for Go.

NOTE: I’m not going to show the code to interact with the users database, as the purpose of this post is to show how to start the dependencies of the application, not how to interact with them.

Introducing Fiber

From their Fiber website:

Fiber is a Go web framework built on top of Fasthttp, the fastest HTTP engine for Go. It’s designed to ease things up for fast development with zero memory allocation and performance in mind.

Why Fiber? There are various frameworks for working with HTTP in Go, such as gin, or gobuffalo. And many Gophers directly stay in the net/http package of the Go’s standard library. In the end, it doesn’t matter which library of framework we choose, as it’s independent of what we’re going to demonstrate here.

Let’s create the default Fiber application:

package main

import (
   "log"
   "os"

   "github.com/gofiber/fiber/v2"
)

func main() {
   app := fiber.New()

   app.Get("/", func(c *fiber.Ctx) error {
       return c.SendString("Hello, World!")
   })

   log.Fatal(app.Listen(":8000"))
}

As we said, our application will connect to a Postgres database to store its users. In order to share state across the application, we’re going to create a new type representing the App. This App type will include information about the Fiber application, and the connection string for the users database.

// MyApp is the main application, including the fiber app and the postgres container
type MyApp struct {
   // The name of the app
   Name string
   // The version of the app
   Version string
   // The fiber app
   FiberApp *fiber.App
   // The database connection string for the users database. The application will need it to connect to the database,
   // reading it from the USERS_CONNECTION environment variable in production, or from the container in development.
   UsersConnection string
}

var App *MyApp = &MyApp{
   Name:            "my-app",
   Version:         "0.0.1",
   // in production, the URL will come from the environment
   UsersConnection: os.Getenv("USERS_CONNECTION"),
}

func main() {
   app := fiber.New()

   app.Get("/", func(c *fiber.Ctx) error {
      return c.SendString("Hello, World!")
   })

   // register the fiber app
   App.FiberApp = app

   log.Fatal(app.Listen(":8000"))
}

For demonstration purposes, we’re going to use the main package to define the access to the users in the Postgres database. In the real-world application, this code wouldn’t be in the main package.

Running the application for local development would be this:

Testcontainers for Go

Testcontainers for Go is a Go library that allows us to start and stop Docker containers from our Go tests. It provides us with a way to define our own containers, so we can start and configure any container we want. It also provides us with a set of predefined containers in the form of Go modules that we can use to start those dependencies of our application.

Therefore, with Testcontainers, we’ll be able to interact with our dependencies in an abstract manner, as we could be interacting with databases, message brokers, or any other kind of dependency in a Docker container.

Starting the dependencies for development mode

Now that we have a library for it, we need to start the dependencies of our application. Remember that we’re talking about the local experience of building the application. So, we would like to start the dependencies only under certain build conditions, not on the production environment.

Go build tags

Go provides us with a way to define build tags that we can use to define build conditions. We can define a build tag in the form of a comment at the top of our Go files. For example, we can define a build tag called dev like this:

// +build dev
// go:build dev

Adding this build tag to a file will mean that the file will only be compiled when the dev build tag is passed to the go build command, not landing into the release artifact. The power of the go toolchain is that this build tag applies to any command that uses the go toolchain, such as go run. Therefore, we can still use this build tag when running our application with go run -tags dev ..

Go init functions

The init functions in Go are special functions that are executed before the main function. We can define an init function in a Go file like this:

func init() {
   // Do something
}

They aren’t executed in a deterministic order, so please consider this when defining init functions.

For our example, in which we want to improve the local development experience in our Go application, we’re going to use an init function in a dev_dependencies.go file protected by a dev build tag. From there, we’ll start the dependencies of our application, which in our case is the PostgreSQL database for the users.

We’ll use Testcontainers for Go to start this Postgres database. Let’s combine all this information in the dev_dependencies.go file:

//go:build dev
// +build dev

package main

import (
   "context"
   "log"
   "path/filepath"
   "time"

   "github.com/jackc/pgx/v5"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/modules/postgres"
   "github.com/testcontainers/testcontainers-go/wait"
)

func init() {
   ctx := context.Background()

   c, err := postgres.RunContainer(ctx,
       testcontainers.WithImage("postgres:15.3-alpine"),
       postgres.WithInitScripts(filepath.Join(".", "testdata", "dev-db.sql")),
       postgres.WithDatabase("users-db"),
       postgres.WithUsername("postgres"),
       postgres.WithPassword("postgres"),
       testcontainers.WithWaitStrategy(
           wait.ForLog("database system is ready to accept connections").
               WithOccurrence(2).WithStartupTimeout(5*time.Second)),
   )
   if err != nil {
       panic(err)
   }

   connStr, err := c.ConnectionString(ctx, "sslmode=disable")
   if err != nil {
       panic(err)
   }

   // check the connection to the database
   conn, err := pgx.Connect(ctx, connStr)
   if err != nil {
       panic(err)
   }
   defer conn.Close(ctx)

   App.UsersConnection = connStr
   log.Println("Users database started successfully")
}

The c container is defined and started using Testcontainers for Go. We’re using:

The WithInitScripts option to copy and run a SQL script that creates the database and the tables. This script is located in the testdata folder.
The WithWaitStrategy option to wait for the database to be ready to accept connections, checking database logs.
The WithDatabase, WithUsername and WithPassword options to configure the database.
The ConnectionString method to get the connection string to the database directly from the started container.

The App variable will be of the type we defined earlier, representing the application. This type included information about the Fiber application and the connection string for the users database. Therefore, after the container is started, we’re filling the connection string to the database directly from the container we just started.

So far so good! We’ve leveraged the built-in capabilities in Go to execute the init functions defined in the dev_dependencies.go file only when the -tags dev flag is added to the go run command.

With this approach, running the application and its dependencies takes a single command!

go run -tags dev .

We’ll see that the Postgres database is started and the tables are created. We can also see that the App variable is filled with the information about the Fiber application and the connection string for the users database.

Stopping the dependencies for development mode

Now that the dependencies are started, if and only if the build tags are passed to the go run command, we need to stop them when the application is stopped.

We’re going to reuse what we did with the build tags to register a graceful shutdown to stop the dependencies of the application before stopping the application itself only when the dev build tag is passed to the go run command.

Our Fiber app stays untouched, and we’ll need to only update the dev_dependencies.go file:

//go:build dev
// +build dev

package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "os/signal"
    "path/filepath"
    "syscall"
    "time"

    "github.com/jackc/pgx/v5"
    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/postgres"
    "github.com/testcontainers/testcontainers-go/wait"
)

func init() {
    ctx := context.Background()

    c, err := postgres.RunContainer(ctx,
        testcontainers.WithImage("postgres:15.3-alpine"),
        postgres.WithInitScripts(filepath.Join(".", "testdata", "dev-db.sql")),
        postgres.WithDatabase("users-db"),
        postgres.WithUsername("postgres"),
        postgres.WithPassword("postgres"),
        testcontainers.WithWaitStrategy(
            wait.ForLog("database system is ready to accept connections").
                WithOccurrence(2).WithStartupTimeout(5*time.Second)),
    )
    if err != nil {
        panic(err)
    }

    connStr, err := c.ConnectionString(ctx, "sslmode=disable")
    if err != nil {
        panic(err)
    }

    // check the connection to the database
    conn, err := pgx.Connect(ctx, connStr)
    if err != nil {
        panic(err)
    }
    defer conn.Close(ctx)

    App.UsersConnection = connStr
    log.Println("Users database started successfully")

    // register a graceful shutdown to stop the dependencies when the application is stopped
    // only in development mode
    var gracefulStop = make(chan os.Signal)
    signal.Notify(gracefulStop, syscall.SIGTERM)
    signal.Notify(gracefulStop, syscall.SIGINT)
    go func() {
        sig := <-gracefulStop
        fmt.Printf("caught sig: %+v\n", sig)
        err := shutdownDependencies()
        if err != nil {
            os.Exit(1)
        }
        os.Exit(0)
    }()
}

// helper function to stop the dependencies
func shutdownDependencies(containers ...testcontainers.Container) error {
    ctx := context.Background()
    for _, c := range containers {
        err := c.Terminate(ctx)
        if err != nil {
            log.Println("Error terminating the backend dependency:", err)
            return err
        }
    }

    return nil
}

In this code, at the bottom of the init function and right after setting the database connection string, we’re starting a goroutine to handle the graceful shutdown. We’re also listening for the defining SIGTERM and SIGINT signals. When a signal is put into the gracefulStop channel the shutdownDependencies helper function will be called to stop the dependencies of the application. This helper function will internally call the Testcontainers for Go’s Terminate method of the database container, resulting in the container being stopped on signals.

What’s especially great about this approach is how dynamic the created environment is. Testcontainers takes extra effort to allow parallelization and binds containers on high-level available ports. This means the dev mode won’t collide with running the tests. Or you can have multiple instances of your application running without any problems!

Hey, what will happen in production?

Because our app is initializing the connection to the database from the environment:

var App *MyApp = &MyApp{
   Name:            "my-app",
   Version:         "0.0.1",
   DevDependencies: []DevDependency{},
   // in production, the URL will come from the environment
   UsersConnection: os.Getenv("USERS_CONNECTION"),
}

We don’t have to worry about that value being overridden by our custom code for the local development. The UsersConnection won’t be set because everything that we showed here is protected by the dev build tag.

NOTE: Are you using Gin or net/http directly? You could directly benefit from everything that we explained here: init functions and build tags to start and graceful shutdown the runtime dependencies.

Conclusion

In this post, we’ve learned how to use Testcontainers for Go to start and stop the dependencies of our application while building it and running the tests. And all we needed to leverage was the built-in capabilities of the Go language and the go toolchain.

The result is that we can start the dependencies of our application while building it and running the application. And we can stop them when the application is stopped. This means that our local development experience is improved, as we don’t need to start the dependencies in a Makefile, shell script, or an external Docker Compose file. And the most important thing, it only happens for development mode, passing the -tags dev flag to the go run command.

Learn more

Find the source code for this blog on GitHub.
Connect on the Testcontainers Slack.
Learn about Testcontainers best practices.
Get started with the Testcontainers guide.
Subscribe to the Docker Newsletter.
Have questions? The Docker community is here to help.
New to Docker? Get started.

Go, golang, Testcontainers

Feedback

0 thoughts on "Local Development of Go Applications with Testcontainers"

Categories