5 Feb 2023 Terraform Plugin Framework: Optional Attributes with Defaults

My Second Custom Terraform Provider

This post is part of a series on the Terraform Plugin Framework:

Terraform Plugin Framework: Optional Attributes With Defaults
Terraform Plugin Framework: Computed Attributes Known Only After Apply
Terraform Plugin Framework: Use State For Unknown

TL;DR skip to The Problem or right to The Solution:

Example Resource Schema
Example DefaultPlanModifier

I went searching for documentation on writing a custom terraform provider last week and noticed a few things:

Hashicorp’s docs on Plugin Framework Benefits recommend using the newers Terraform Plugin Framework for new provider development:
HashiCorp offers two Go programming language Software Development Kits (SDKs) for building Terraform providers:
- Terraform Plugin Framework: The most recent SDK that is easier to use and more extensible than SDKv2.
- SDKv2: SDKv2 is the prior SDK that many existing providers use. It is maintained for Terraform versions 1.x and earlier, but we have stopped most feature development so we can focus on improving the framework.
We recommend using the framework for new provider development because it offers significant advantages as compared to the SDKv2. We also recommend migrating existing providers to the framework when possible.
The newer Terraform Plugin Framework has some breaking changes from the older SDKv2 specifically around the syntax for providing default values and
These Hashicorp learning and reference implementations are still using the older SDKv2 framework:
- Call APIs with Custom SDK Providers
  (a step-by-step walkthrough from Hashicorp about building a custom provider)
- terraform-provider-hashicups (on github)
Several “My First Terraform Provider” style blog posts which I found are either using the SDKv2 (or even older patterns) or, if using the newer Terraform Plugin Framework, are simple enough where each attribute is only Required or Computed.

All of made it hard to find working examples of using the newer Schema pattern to configure Optional attributes with a default.

The Problem: Optional with a Default value is not trivial

Beyond not finding any clear examples it turns out the the semantics of how to handle Optional HCL attributes with Default values is complicated.

Computed, Optional, RequiresReplace behaviour: state of play #189 includes a two long tables showing the multiple possible combinations of Optional, Computed, Current value, Planned value, etc.

In my case I wanted the behavior which @jaloren called out in Improve Ergonomics of Setting Default Values #516:

if the field is not set in terraform during creation, then the provider supplies the value of 1 using tfsdk. AttributePlanModifier and terraform creates the infra where the property is set to one in the infra.

if the field is not in the terraform config during update, then the provider supplies the value of 1 using tfsdk. AttributePlanModifier and terraform should not flag the resource as needing to be updated.

if the user adds the field in the terraform config during either creation or update, the provider does not supply a value and terraform creates/updates the resource to have the user supplied value.

if the user adds the field in the terraform config, successfully performs a terraform apply and then removes the field from the config, then terraform will update the infrastructure to use the default value since the user-supplied value no longer exists.

Until I found @jaloren’s issue i couldn’t find any reliable examples of how to configure this using the newer terraform-plugin-framework patterns.

Here is a more complete list of the Github issues that I read through in my search to figure this out:

terraform-plugin-framework
terraform-provider-aws
- Optional/Computed Sets with Terraform Plugin Framework #28638

The Solution

The issue and documentation that cracked this open for me are the discussion here:

Improve Ergonomics of Setting Default Values #516

and the detailed documentation here:

Terraform Resource Instance Change Lifecycle

which states:

Proposed New State: Terraform Core uses some built-in logic to perform an initial basic merger of the Configuration and the Prior State which a provider may use as a starting point for its planning operation.

The built-in logic primarily deals with the expected behavior for attributes marked in the schema as both “optional” and “computed”, which means that the user may either set it or may leave it unset to allow the provider to choose a value instead.

Terraform Core therefore constructs the proposed new state by taking the attribute value from Configuration if it is non-null, and then using the Prior State as a fallback otherwise, thereby helping a provider to preserve its previously-chosen value for the attribute where appropriate.

That helped me understand when the PlanModifiers run and recognize that the signature of the modification function includes a plan request (what the initial draft plan was) and a plan response (what the modified plan should be).

Here is a the pattern which worked for me:

Example Resource Schema
Example DefaultPlanModifier

Example Resource Schema

Define an Optional attribute with a default as both Optional (meaning it’s okay to leave it out of your HCL) and Computed (meaning that the provider is allowed to modify the value)

func (r *MyResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
	resp.Schema = schema.Schema{
		// This description is used by the documentation generator and the language server.
		MarkdownDescription: "my resource",

		Attributes: map[string]schema.Attribute{
			"id": schema.StringAttribute{
				Computed:            true,
				MarkdownDescription: "internal resource identifier (set by the provider)",
				PlanModifiers: []planmodifier.String{
					stringplanmodifier.UseStateForUnknown(),
				},
			},
			"name": schema.StringAttribute{
				MarkdownDescription: "friendly name of the resource",
				Required:            true,
			},
			"description": schema.StringAttribute{
				MarkdownDescription: "optional description of the resource",
				Optional:            true,
				Computed:            true,
				PlanModifiers: []planmodifier.String{
					mystringplanmodifiers.DefaultValue(""),
				},
			},
		},
	}
}

This defines a resource schema with three attributes:

id is managed internally by the provider, so it is entirely Computed
- the stringplanmodifier.UseStateForUnknown() modifier marks this attribute as the one which signifies whether the resource is saved
name is used to uniquely identify this example resource, so it is entirely Required
description is Optional with a Default value provided so it must be marked both Optional and Computed
- the mystringplanmodifiers.DefaultValue("") is the magic sauce which inspects the plan before it is presented and applies the default if and only if the configured value is null

Example DefaultPlanModifier

While the ability to provide a default value is straightforward and baked into the SDKv2 schema, it is enabled but missing from the terraform-plugin-framework (see AttributePlanModifier for default value #285).

Here is the one I finally got working, cribbed together from:

The stringmodifier package from the terraform-provider-local provider
The modifiers package from the terraform-community-providers/terraform-plugin-framework-utils repo
The stringplanmodifier package from the terraform-provider-aws repo

package stringplanmodifier

import (
	"context"
	"fmt"

	"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
	"github.com/hashicorp/terraform-plugin-framework/types"
)

// DefaultValue return a string plan modifier that sets the specified value if the planned value is Null.
func DefaultValue(s string) planmodifier.String {
	return defaultValue{
		val: s,
	}
}

// defaultValue holds our default value and allows us to implement the `planmodifier.String` interface
type defaultValue struct {
	val string
}

// Description implements the `planmodifier.String` interface
func (m defaultValue) Description(context.Context) string {
	return fmt.Sprintf("If value is not configured, defaults to %s", m.val)
}

// MarkdownDescription implements the `planmodifier.String` interface
func (m defaultValue) MarkdownDescription(ctx context.Context) string {
	return m.Description(ctx) // reuse our plaintext Description
}

// PlanModifyString implements the `planmodifier.String` interface
func (m defaultValue) PlanModifyString(ctx context.Context, req planmodifier.StringRequest, resp *planmodifier.StringResponse) {
	// If the attribute configuration is not null it is explicit; we should apply the planned value.
	if !req.ConfigValue.IsNull() {
		return
	}

	// Otherwise, the configuration is null, so apply the default value to the response.
	resp.PlanValue = types.StringValue(m.val)
}

Test Cases

Building my provider locally I can configure a developer override so that Terraform can find it.

Set ~/.terraformrc to:

provider_installation {
  dev_overrides {
    "registry.terraform.io/davidalpert/myprovider" = "/local-path-to-my-provider/bin"
  }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

Now I can create a new folder for terraform source (e.g. myprovider) and add a providers.tf with

provider "myprovider" {
  // provider configuration properties go here
}

And a main.tf file which defines a resource:

resource "myprovider_my_resource" "test" {
  name = "my-test"
  description = "created by terraform"
}

Running terraform apply successfully creates the resource and saves it to terraform state.

Running terraform plan then shows no changes are needed.

Remove the description:

resource "myprovider_my_resource" "test" {
  name = "my-test"
}

and the plan shows that the resource is

terraform plan
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│  - davidalpert/myprovider in /local-path-to-my-provider/bin
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵
myprovider_my_resource.test: Refreshing state... [id=test]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # myprovider_my_resource.test will be updated in-place
  ~ resource "myprovider_my_resource" "test" {
      - description = "created by terraform" -> null
        id          = "test"
        # (1 unchanged attribute hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

Applying that plan then properly updates the resource and terraform state.

Happy terraforming!