Cloud SQL Go Connector

The Cloud SQL Go Connector is a Cloud SQL connector designed for use with the Go language. Using a Cloud SQL connector provides a native alternative to the Cloud SQL Auth Proxy while providing the following benefits:

• IAM Authorization: uses IAM permissions to control who/what can connect to your Cloud SQL instances
• Improved Security: uses robust, updated TLS 1.3 encryption and identity verification between the client connector and the server-side proxy, independent of the database protocol.
• Convenience: removes the requirement to use and distribute SSL certificates, as well as manage firewalls or source/destination IP addresses.
• (optionally) IAM DB Authentication: provides support for Cloud SQL’s automatic IAM DB AuthN feature.

For users migrating from the Cloud SQL Proxy drivers, see the migration guide.

For a quick example, try out the Go Connector in a Codelab.

You can install this repo with go get:

go get cloud.google.com/go/cloudsqlconn

This package provides several functions for authorizing and encrypting connections. These functions can be used with your database driver to connect to your Cloud SQL instance.

The instance connection name for your Cloud SQL instance is always in the format project:region:instance.

This package requires the following to successfully make Cloud SQL Connections:

• IAM principal (user, service account, etc.) with the Cloud SQL Client role or equivalent. This IAM principal will be used for credentials.
• The Cloud SQL Admin API to be enabled within your Google Cloud Project. By default, the API will be called in the project associated with the IAM principal.

This project uses the Application Default Credentials (ADC) strategy for resolving credentials. Please see these instructions for how to set your ADC (Google Cloud Application vs Local Development, IAM user vs service account credentials), or consult the golang.org/x/oauth2/google documentation.

To explicitly set a specific source for the Credentials, see Using Options below.

Postgres users have the option of using the database/sql interface or using pgx directly. See pgx's advice on which to choose.

To use the dialer with pgx, we recommend using connection pooling with pgxpool by configuring a Config.DialFunc like so:

import (
    "context"
    "net"

    "cloud.google.com/go/cloudsqlconn"
    "github.com/jackc/pgx/v5/pgxpool"
)

func connect() {
    // Configure the driver to connect to the database
    dsn := "user=myuser password=mypass dbname=mydb sslmode=disable"
    config, err := pgxpool.ParseConfig(dsn)
    if err != nil {
        /* handle error */
    }

    // Create a new dialer with any options
    d, err := cloudsqlconn.NewDialer(context.Background())
    if err != nil {
        /* handle error */
    }

    // Tell the driver to use the Cloud SQL Go Connector to create connections
    config.ConnConfig.DialFunc = func(ctx context.Context, _ string, instance string) (net.Conn, error) {
        return d.Dial(ctx, "project:region:instance")
    }

    // Interact with the driver directly as you normally would
    pool, err := pgxpool.NewWithConfig(context.Background(), config)
    if err != nil {
        /* handle error */
    }

    // call cleanup when you're done with the database connection
    cleanup := func() error { return d.Close() }
    // ... etc
}

To use database/sql, call pgxv5.RegisterDriver with any necessary Dialer configuration. Note: the connection string must use the keyword/value format with host set to the instance connection name. The returned cleanup func will stop the dialer's background refresh goroutine and so should only be called when you're done with the Dialer.

import (
    "database/sql"

    "cloud.google.com/go/cloudsqlconn"
    "cloud.google.com/go/cloudsqlconn/postgres/pgxv5"
)

func connect() {
    cleanup, err := pgxv5.RegisterDriver("cloudsql-postgres", cloudsqlconn.WithIAMAuthN())
    if err != nil {
        // ... handle error
    }
    // call cleanup when you're done with the database connection
    defer cleanup()

    db, err := sql.Open(
        "cloudsql-postgres",
        "host=project:region:instance user=myuser password=mypass dbname=mydb sslmode=disable",
    )
    if err != nil {
        // ... handle error
    }
    // close the pool before cleanup() so that connection-close metrics are
    // recorded by the Dialer before it shuts down
    defer db.Close()
    // ... etc
}

To use database/sql, use mysql.RegisterDriver with any necessary Dialer configuration. The returned cleanup func will stop the dialer's background refresh goroutine and so should only be called when you're done with the Dialer.

import (
    "database/sql"

    "cloud.google.com/go/cloudsqlconn"
    "cloud.google.com/go/cloudsqlconn/mysql/mysql"
)

func connect() {
    cleanup, err := mysql.RegisterDriver("cloudsql-mysql", cloudsqlconn.WithCredentialsFile("key.json"))
    if err != nil {
        // ... handle error
    }
    // call cleanup when you're done with the database connection
    defer cleanup()

    db, err := sql.Open(
        "cloudsql-mysql",
        "myuser:mypass@cloudsql-mysql(project:region:instance)/mydb",
    )
    // ... etc
}

To use database/sql, use mssql.RegisterDriver with any necessary Dialer configuration. The returned cleanup func will stop the dialer's background refresh goroutine and so should only be called when you're done with the Dialer.

import (
    "database/sql"

    "cloud.google.com/go/cloudsqlconn"
    "cloud.google.com/go/cloudsqlconn/sqlserver/mssql"
)

func connect() {
    cleanup, err := mssql.RegisterDriver("cloudsql-sqlserver", cloudsqlconn.WithCredentialsFile("key.json"))
    if err != nil {
        // ... handle error
    }
    // call cleanup when you're done with the database connection
    defer cleanup()

    db, err := sql.Open(
        "cloudsql-sqlserver",
        "sqlserver://user:password@localhost?database=mydb&cloudsql=project:region:instance",
    )
    // ... etc
}

The connector can be configured to use DNS to look up an instance. Use a DNS name managed by Cloud SQL Advanced Disaster Recovery, or a domain name that you manage.

Advanced Disaster Recovery creates geographically distributed replicas of your Cloud SQL database instance. When you perform switchover or failover on the database instance, the connector will gracefully disconnect from the old primary instance and reconnect to the new primary instance.

Follow the instructions in Connect using Write Endpoint to get the write endpoint DNS name for your primary instance. Then, use this write endpoint DNS name to configure the connector.

The connector may be configured to use DNS that you define as well.

Add a DNS TXT record for the Cloud SQL instance to a private DNS server or a private Google Cloud DNS Zone used by your application.

Note: You are strongly discouraged from adding DNS records for your Cloud SQL instances to a public DNS server. This would allow anyone on the internet to discover the Cloud SQL instance name.

For example: suppose you wanted to use the domain name prod-db.mycompany.example.com to connect to your database instance my-project:region:my-instance. You would create the following DNS record:

• Record type: TXT
• Name: prod-db.mycompany.example.com – This is the domain name used by the application
• Value: my-project:region:my-instance – This is the instance name

Configure the connector as described above, replacing the connector ID with the DNS name.

Adapting the MySQL + database/sql example above:

package main

import (
	"database/sql"

	"cloud.google.com/go/cloudsqlconn"
	"cloud.google.com/go/cloudsqlconn/mysql/mysql"
)

func connect() {
	cleanup, err := mysql.RegisterDriver("cloudsql-mysql",
		cloudsqlconn.WithDNSResolver())
	if err != nil {
		// ... handle error
	}
	// call cleanup when you're done with the database connection
	defer cleanup()

	db, err := sql.Open(
		"cloudsql-mysql",
		"myuser:mypass@cloudsql-mysql(prod-db.mycompany.example.com)/mydb",
	)
	// ... etc
}

When the connector is configured using a domain name, the connector will periodically check if the DNS record for an instance changes. When the connector detects that the domain name refers to a different instance, the connector will close all open connections to the old instance. Subsequent connection attempts will be directed to the new instance.

For example: suppose application is configured to connect using the domain name prod-db.mycompany.example.com. Initially the corporate DNS zone has a TXT record with the value my-project:region:my-instance. The application establishes connections to the my-project:region:my-instance Cloud SQL instance.

Then, to reconfigure the application to use a different database instance, change the value of the prod-db.mycompany.example.com DNS record from my-project:region:my-instance to my-project:other-region:my-instance-2

The connector inside the application detects the change to this DNS record. Now, when the application connects to its database using the domain name prod-db.mycompany.example.com, it will connect to the my-project:other-region:my-instance-2 Cloud SQL instance.

The connector will automatically close all existing connections to my-project:region:my-instance. This will force the connection pools to establish new connections. Also, it may cause database queries in progress to fail.

The connector will poll for changes to the DNS name every 30 seconds by default. You may configure the frequency of the connections using the option WithFailoverPeriod(d time.Duration). When this is set to 0, the connector will disable polling and only check if the DNS record changed when it is creating a new connection.

If you need to customize something about the Dialer, you can initialize directly with NewDialer:

d, err := cloudsqlconn.NewDialer(
    ctx,
    cloudsqlconn.WithCredentialsFile("key.json"),
)
if err != nil {
    log.Fatalf("unable to initialize dialer: %s", err)
}

conn, err := d.Dial(ctx, "project:region:instance")

For a full list of customizable behavior, see Option.

If you want to customize things about how the connection is created, use Option:

conn, err := d.Dial(
    ctx,
    "project:region:instance",
    cloudsqlconn.WithPrivateIP(),
)

You can also use the WithDefaultDialOptions Option to specify DialOptions to be used by default:

d, err := cloudsqlconn.NewDialer(
    ctx,
    cloudsqlconn.WithDefaultDialOptions(
        cloudsqlconn.WithPrivateIP(),
    ),
)

Connections using Automatic IAM database authentication are supported when using Postgres or MySQL drivers.

Make sure to configure your Cloud SQL Instance to allow IAM authentication and add an IAM database user.

A Dialer can be configured to connect to a Cloud SQL instance using automatic IAM database authentication with the WithIAMAuthN Option (recommended) or the WithDialIAMAuthN DialOption.

d, err := cloudsqlconn.NewDialer(ctx, cloudsqlconn.WithIAMAuthN())

When configuring the DSN for IAM authentication, the password field can be omitted and the user field should be formatted as follows:

Postgres: For an IAM user account, this is the user's email address. For a service account, it is the service account's email without the .gserviceaccount.com domain suffix.

MySQL: For an IAM user account, this is the user's email address, without the @ or domain name. For example, for test-user@gmail.com, set the user field to test-user. For a service account, this is the service account's email address without the @project-id.iam.gserviceaccount.com suffix.

Example DSNs using the test-sa@test-project.iam.gserviceaccount.com service account to connect can be found below.

Postgres:

dsn := "user=test-sa@test-project.iam dbname=mydb sslmode=disable"

MySQL:

dsn := "user=test-sa dbname=mydb sslmode=disable"

The connector reports client-side connection metrics to Cloud Monitoring under the cloudsql.googleapis.com/client/connector/ prefix, using the cloudsql.googleapis.com/InstanceClient monitored resource. These metrics help you (and Cloud SQL) spot connectivity regressions, for example a rise in failed dial attempts or in dial latency. They are enabled by default.

Metrics are written to the Cloud SQL instance's project using the same identity the connector uses for the Cloud SQL Admin API. That identity needs the roles/monitoring.metricWriter role on that project. If you pass your own token source or credentials (WithTokenSource, WithCredentials, or an impersonated credential), they must also carry the https://www.googleapis.com/auth/monitoring.write (or cloud-platform) scope; credentials the connector detects itself are scoped automatically. Export is strictly fail-open: it never affects connectivity. If the identity lacks the role or the scope, the connector logs one line per process explaining what to fix and retries with exponential backoff (up to once per hour); all further diagnostics go to the debug logger only.

In a non-default Cloud universe (a universe domain other than googleapis.com), built-in metrics are off unless you pass WithBuiltInMetrics(true).

The export is billed to the caller's project (for a service account, the project it belongs to), which must have the Cloud Monitoring API enabled. WithQuotaProject changes the billed project, as it does for the Cloud SQL Admin API.

Hermetic tests that construct a Dialer against a mock Admin API (WithAdminAPIEndpoint, WithHTTPClient) should pass WithBuiltInMetrics(false) so the test does not try to reach Cloud Monitoring.

To turn built-in metrics off:

d, err := cloudsqlconn.NewDialer(ctx, cloudsqlconn.WithBuiltInMetrics(false))

Two optional labels can be set by the application:

d, err := cloudsqlconn.NewDialer(ctx,
    cloudsqlconn.WithApplicationName("my-service"),                      // application_name
    cloudsqlconn.WithComputePlatform(cloudsqlconn.ComputePlatformCloudRun), // overrides auto-detection
)

application_name identifies your application and groups its metrics across all of its instances. It is only ever the value you set here and is left empty otherwise; the connector does not infer it from the environment (instances are already told apart by client_uid, and compute_platform is auto-detected).

WithComputePlatform accepts only the ComputePlatform* constants. To take the platform from a string such as a flag value, use cloudsqlconn.ParseComputePlatform(s); cloudsqlconn.ComputePlatforms() lists the valid values for help text.

Metrics are exported every 60 seconds and flushed when Dialer.Close() is called, so short-lived processes should close the Dialer before exiting.

These metrics are in BETA: label values are documented in the metric descriptors and pinned by tests, but the feature is subject to the Cloud SQL pre-GA terms.

This library includes support for metrics and tracing using OpenCensus. To enable metrics or tracing, you need to configure an exporter. OpenCensus supports many backends for exporters.

Supported metrics include:

• cloudsqlconn/dial_latency: The distribution of dialer latencies (ms)
• cloudsqlconn/open_connections: The current number of open Cloud SQL connections
• cloudsqlconn/dial_failure_count: The number of failed dial attempts
• cloudsqlconn/refresh_success_count: The number of successful certificate refresh operations
• cloudsqlconn/refresh_failure_count: The number of failed refresh operations
• cloudsqlconn/bytes_sent: The number of bytes sent to Cloud SQL
• cloudsqlconn/bytes_received: The number of bytes received from Cloud SQL

Supported traces include:

• cloud.google.com/go/cloudsqlconn.Dial: The dial operation including refreshing an ephemeral certificate and connecting the instance
• cloud.google.com/go/cloudsqlconn/internal.InstanceInfo: The call to retrieve instance metadata (e.g., database engine type, IP address, etc)
• cloud.google.com/go/cloudsqlconn/internal.Connect: The connection attempt using the ephemeral certificate
• SQL Admin API client operations

For example, to use Cloud Monitoring and Cloud Trace, you would configure an exporter like so:

import (
    "contrib.go.opencensus.io/exporter/stackdriver"
    "go.opencensus.io/trace"
)

func main() {
    sd, err := stackdriver.NewExporter(stackdriver.Options{
        ProjectID: "mycoolproject",
    })
    if err != nil {
        // handle error
    }
    defer sd.Flush()
    trace.RegisterExporter(sd)

    sd.StartMetricsExporter()
    defer sd.StopMetricsExporter()

    // Use cloudsqlconn as usual.
    // ...
}

As OpenTelemetry has now reached feature parity with OpenCensus, the migration from OpenCensus to OpenTelemetry is strongly encouraged. OpenTelemetry bridge can be leveraged to migrate to OpenTelemetry without the need of replacing the OpenCensus APIs in this library. Example code is shown below for migrating an application using the OpenTelemetry bridge for traces.

import (
	texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace"
	"go.opencensus.io/trace"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/bridge/opencensus"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	"google.golang.org/api/option"
)

func main() {
	// trace.AlwaysSample() is expensive. Replacing it with your own
	// sampler for production environments is recommended.
	trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})

	exporter, err := texporter.New(
		texporter.WithTraceClientOptions([]option.ClientOption{option.WithTelemetryDisabled()}),
		texporter.WithProjectID("mycoolproject"),
	)
	if err != nil {
		// Handle error
	}

	tp := sdktrace.NewTracerProvider(sdktrace.WithSyncer(exporter))
	otel.SetTracerProvider(tp)
	tracer := tp.Tracer("Cloud SQL Go Connector Trace")
	trace.DefaultTracer = opencensus.NewTracer(tracer)

	// Use cloudsqlconn as usual.
	// ...
}

A known OpenTelemetry issue has been reported here. It shouldn't impact database operations.

The Go Connector supports optional debug logging to help diagnose problems with the background certificate refresh. To enable it, provide a logger that implements the debug.ContextLogger interface when initializing the Dialer.

For example:

import (
    "context"
    "net"

    "cloud.google.com/go/cloudsqlconn"
)

type myLogger struct{}

func (l *myLogger) Debugf(ctx context.Context, format string, args ...interface{}) {
    // Log as you like here
}

func connect() {
    l := &myLogger{}

    d, err := NewDialer(
        context.Background(),
        cloudsqlconn.WithContextDebugLogger(l),
    )
    // use dialer as usual...
}

This project uses semantic versioning, and uses the following lifecycle regarding support for a major version:

Active - Active versions get all new features and security fixes (that wouldn’t otherwise introduce a breaking change). New major versions are guaranteed to be "active" for a minimum of 1 year.

Deprecated - Deprecated versions continue to receive security and critical bug fixes, but do not receive new features. Deprecated versions will be supported for 1 year.

Unsupported - Any major version that has been deprecated for >=1 year is considered unsupported.

We follow the Go Version Support Policy used by Google Cloud Libraries for Go.

The minimum supported Go version is Go 1.26, as declared in go.mod.

This project aims for a release on at least a monthly basis. If no new features or fixes have been added, a new PATCH version with the latest dependencies is released.