Follow me at @dane_albaugh
At Augno, we maintain an API with 400+ endpoints and dozens of resources. Managing an API with this much surface area requires writing a lot of boring boilerplate. You have to read query params, decode JSON, validate fields, call business logic, serialize the response, and handle errors over and over again. Before LLMs this was boring to write. And after LLMs, it became apparent that it was even more boring to review. These kinds of repetitive programming tasks have never been fun and consequently always seem to introduce inconsistencies and bugs.
We decided to eliminate the boilerplate by building a declarative, generic endpoint system where each endpoint is defined and configured with a struct literal. Then, a single shared Execute method handles all the HTTP transport concerns automatically.
The old approach
Consider a typical Go HTTP handler. Even with a framework, you end up writing something like this for every endpoint:
func CreateAPIKeyHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
orgID, ok := auth.OrgIDFromContext(ctx)
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
var req CreateAPIKeyRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, 400, "invalid json")
return
}
defer r.Body.Close()
if req.Name == "" {
writeError(w, 400, "name is required")
return
}
if req.RoleID == "" {
writeJSONError(w, http.StatusBadRequest, map[string]any{
"code": "validation_error",
"message": "role_id is required",
})
return
}
result, err := apiKeySvc.CreateAPIKey(ctx, orgID, &req)
if err != nil {
switch e := err.(type) {
case *apierror.APIError:
writeJSONError(w, e.StatusCode, e)
default:
log.Printf("create api key: %v", err)
http.Error(w, "internal server error", http.StatusInternalServerError)
}
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
if err := json.NewEncoder(w).Encode(result); err != nil {
log.Printf("encode create api key response: %v", err)
}
}
func CreateAPIKeyHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
orgID, ok := auth.OrgIDFromContext(ctx)
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
var req CreateAPIKeyRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, 400, "invalid json")
return
}
defer r.Body.Close()
if req.Name == "" {
writeError(w, 400, "name is required")
return
}
if req.RoleID == "" {
writeJSONError(w, http.StatusBadRequest, map[string]any{
"code": "validation_error",
"message": "role_id is required",
})
return
}
result, err := apiKeySvc.CreateAPIKey(ctx, orgID, &req)
if err != nil {
switch e := err.(type) {
case *apierror.APIError:
writeJSONError(w, e.StatusCode, e)
default:
log.Printf("create api key: %v", err)
http.Error(w, "internal server error", http.StatusInternalServerError)
}
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
if err := json.NewEncoder(w).Encode(result); err != nil {
log.Printf("encode create api key response: %v", err)
}
}
I realize I could dramatically improve the readability of this handler with some helper functions. Humor me.
What is this endpoint doing? It is a bit hard to figure that out at a glance - no thanks to Go's verbosity. And this is a relatively simple endpoint.
Now, duplicate this across 400+ endpoints and you will get subtle differences in error handling, inconsistent validation, and forgotten headers. On top of that, some moron (me) would end up copying sections of this boilerplate to a new endpoint and fail to notice some inconsistency.
The desired solution shape
I wanted to be able to express only the unique features of an endpoint and let our shared infrastructure handle the rest. After some research, I landed on an approach that fit our needs123. Here is what defining an endpoint looks like now.
The request:
// Request to create an API key.
type CreateAPIKeyRequest struct {
// Role ID assigned to the API key.
RoleID string `json:"role_id" validate:"required"`
// Human-readable name for the API key.
Name string `json:"name" validate:"required,max=255"`
// Expiration timestamp. If not set, the key does not expire.
ExpiresAt field.Optional[time.Time] `json:"expires_at,omitzero"`
}
// Request to create an API key.
type CreateAPIKeyRequest struct {
// Role ID assigned to the API key.
RoleID string `json:"role_id" validate:"required"`
// Human-readable name for the API key.
Name string `json:"name" validate:"required,max=255"`
// Expiration timestamp. If not set, the key does not expire.
ExpiresAt field.Optional[time.Time] `json:"expires_at,omitzero"`
}
The response:
// Result of creating an API key, with the full secret value.
type CreatedAPIKey struct {
// Resource type identifier.
Object constants.ObjectType `json:"object" validate:"required,enum=created_api_key"`
// Full secret value. Returned once and cannot be retrieved later. Learn more about [managing your API keys](https://docs.augno.com/api/managing-api-keys).
APIKeySecret string `json:"api_key_secret" validate:"required" sensitive:"true"`
// API key metadata.
APIKeyInfo APIKey `json:"api_key_info" validate:"required"`
}
// APIKey represents an API key for authenticating API requests.
type APIKey struct {
// API key ID.
ID string `json:"id" validate:"required"`
// Resource type identifier.
Object constants.ObjectType `json:"object" validate:"required,enum=api_key"`
// Human-readable name for the API key.
Name string `json:"name" validate:"required"`
// Redacted key value safe for display.
RedactedValue string `json:"redacted_value" validate:"required"`
// Assigned role.
Role *Role `json:"role" expandable:"true"`
// Creation timestamp.
CreatedAt time.Time `json:"created_at" validate:"required"`
// Last updated timestamp.
UpdatedAt time.Time `json:"updated_at" validate:"required"`
// Last used timestamp.
LastUsedAt *time.Time `json:"last_used_at"`
// Expiration timestamp.
ExpiresAt *time.Time `json:"expires_at"`
// Revocation timestamp.
RevokedAt *time.Time `json:"revoked_at"`
}
// Result of creating an API key, with the full secret value.
type CreatedAPIKey struct {
// Resource type identifier.
Object constants.ObjectType `json:"object" validate:"required,enum=created_api_key"`
// Full secret value. Returned once and cannot be retrieved later. Learn more about [managing your API keys](https://docs.augno.com/api/managing-api-keys).
APIKeySecret string `json:"api_key_secret" validate:"required" sensitive:"true"`
// API key metadata.
APIKeyInfo APIKey `json:"api_key_info" validate:"required"`
}
// APIKey represents an API key for authenticating API requests.
type APIKey struct {
// API key ID.
ID string `json:"id" validate:"required"`
// Resource type identifier.
Object constants.ObjectType `json:"object" validate:"required,enum=api_key"`
// Human-readable name for the API key.
Name string `json:"name" validate:"required"`
// Redacted key value safe for display.
RedactedValue string `json:"redacted_value" validate:"required"`
// Assigned role.
Role *Role `json:"role" expandable:"true"`
// Creation timestamp.
CreatedAt time.Time `json:"created_at" validate:"required"`
// Last updated timestamp.
UpdatedAt time.Time `json:"updated_at" validate:"required"`
// Last used timestamp.
LastUsedAt *time.Time `json:"last_used_at"`
// Expiration timestamp.
ExpiresAt *time.Time `json:"expires_at"`
// Revocation timestamp.
RevokedAt *time.Time `json:"revoked_at"`
}
And the endpoint itself:
// Creates an [API key](https://docs.augno.com/api/api-keys) to authenticate API requests.
//
// The secret key is returned once and cannot be retrieved later, so you should store it securely. We provide some [recommendations](https://docs.augno.com/api/managing-api-keys) on how you can manage your API keys.
type CreateAPIKeyEndpoint struct{}
func (e *CreateAPIKeyEndpoint) Materialize() *apiendpoint.APIEndpoint[*CreateAPIKeyRequest, *apiresource.CreatedAPIKey] {
return (&apiendpoint.APIEndpoint[*CreateAPIKeyRequest, *apiresource.CreatedAPIKey]{
Title: "Create API Key",
Method: http.MethodPost,
Route: "/v1/auth/api-keys",
ContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
Public: true,
Preview: true,
ObjectType: constants.ObjectTypeCreatedAPIKey,
ServiceHandler: func(svc any) func(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError) {
return svc.(APIKeySvc).CreateAPIKey
},
LocationFunc: func(resp *apiresource.CreatedAPIKey) string {
return "/v1/auth/api-keys/" + resp.APIKeyInfo.ID
},
IncludeConfig: apiendpoint.IncludesFor(apiendpoint.IncludesParams{
ObjectType: constants.ObjectTypeAPIKey,
Fields: []string{"role", "role.permissions"},
}),
})
}
// Creates an [API key](https://docs.augno.com/api/api-keys) to authenticate API requests.
//
// The secret key is returned once and cannot be retrieved later, so you should store it securely. We provide some [recommendations](https://docs.augno.com/api/managing-api-keys) on how you can manage your API keys.
type CreateAPIKeyEndpoint struct{}
func (e *CreateAPIKeyEndpoint) Materialize() *apiendpoint.APIEndpoint[*CreateAPIKeyRequest, *apiresource.CreatedAPIKey] {
return (&apiendpoint.APIEndpoint[*CreateAPIKeyRequest, *apiresource.CreatedAPIKey]{
Title: "Create API Key",
Method: http.MethodPost,
Route: "/v1/auth/api-keys",
ContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
Public: true,
Preview: true,
ObjectType: constants.ObjectTypeCreatedAPIKey,
ServiceHandler: func(svc any) func(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError) {
return svc.(APIKeySvc).CreateAPIKey
},
LocationFunc: func(resp *apiresource.CreatedAPIKey) string {
return "/v1/auth/api-keys/" + resp.APIKeyInfo.ID
},
IncludeConfig: apiendpoint.IncludesFor(apiendpoint.IncludesParams{
ObjectType: constants.ObjectTypeAPIKey,
Fields: []string{"role", "role.permissions"},
}),
})
}
This is the entire endpoint definition. We do not write any HTTP handler code, and there is no manual parsing or error handling. You simply declare your definition and write your service handler. The same definition drives OpenAPI spec generation. So, how does this work?
Step 1: The generic API endpoint type
The foundation is a single generic struct that represents any endpoint in the system (I've simplified this a bit):
type APIEndpoint[TReq, TResp any] struct {
Title string
Method string
Route string
SDKMethodKey string
ContentType string
SuccessStatusCode int
Public bool
Preview bool
ServiceHandler func(svc any) ServiceHandler[TReq, TResp]
Extras APIEndpointExtras
MinVersion *version.APIVersion
ObjectType constants.ObjectType
// LocationFunc returns the Location header value for 201 Created responses.
LocationFunc func(TResp) string
// IncludeConfig declares which sub-objects can be expanded via the include query parameter. When nil, no include support is provided.
IncludeConfig *IncludeConfig
}
type APIEndpoint[TReq, TResp any] struct {
Title string
Method string
Route string
SDKMethodKey string
ContentType string
SuccessStatusCode int
Public bool
Preview bool
ServiceHandler func(svc any) ServiceHandler[TReq, TResp]
Extras APIEndpointExtras
MinVersion *version.APIVersion
ObjectType constants.ObjectType
// LocationFunc returns the Location header value for 201 Created responses.
LocationFunc func(TResp) string
// IncludeConfig declares which sub-objects can be expanded via the include query parameter. When nil, no include support is provided.
IncludeConfig *IncludeConfig
}
There is no Request or Response field. The request and response types come from the generic parameters and are recovered when needed via reflection:
func (e *APIEndpoint[TReq, TResp]) GetRequestType() reflect.Type { return reflect.TypeFor[TReq]() }
func (e *APIEndpoint[TReq, TResp]) GetResponseType() reflect.Type { return reflect.TypeFor[TResp]() }
func (e *APIEndpoint[TReq, TResp]) GetRequestType() reflect.Type { return reflect.TypeFor[TReq]() }
func (e *APIEndpoint[TReq, TResp]) GetResponseType() reflect.Type { return reflect.TypeFor[TResp]() }
reflect.TypeFor[T]() gives us the runtime type descriptor for T without storing T in a field or creating a dummy value.
If you have not used reflection in Go before, this is a good primer.
Step 2: Struct tags as a declarative schema
Each field on a request struct declares where its value comes from. A query tag binds from the query string, a path tag from a URL segment, and a json tag from the request body. Additional tags handle defaults, validation, and a few specialized binding modes. At runtime, the binding layer walks the struct and fills in a typed Go value before the handler runs.
Here are some tags consumed by the transport layer:
| Tag | Source |
|---|---|
json | request body |
query | query string |
path | URL path segment |
header | request header |
cookie | cookie (used on just a few endpoints to get a refresh token) |
scheme | required scheme prefix when extracting from Authorization (e.g. scheme:"Bearer") |
rawbody | bind the entire request body to a []byte field |
time_layout | overrides the default layout used to parse a time.Time from a string source |
default | default value when no source provided one |
validate | downstream go-playground/validator rules |
expandable | marks a response sub-object as supporting include/expand |
sensitive | marks a field as sensitive so it is not saved in the request/response logs |
Most endpoints combine two or three binding sources.
The json tag, presence, and Go's two-state problem
json tag, presence, and Go's two-state problemAs we built out this system, we had a hard time figuring out what to do with the json tag. The request body is encoded in JSON and gives clients the ability to express three separate intents for any field:
| What the client sends | What they mean |
|---|---|
the key is absent ({}) | "I'm not saying anything about this field." |
the key is null ({"note": null}) | "Set this to nothing / clear it." |
the key has a value ({"note": "hi"}) | "Set this to this value." |
However, Go's encoding/json (the standard library decoder) can only natively represent two of these states, and which two it loses depends on whether your field is a value or a pointer. Let's go through some examples.
When the client sends a request, Go takes that JSON and fills in a struct.
If a field is a value type (a plain string), then both {} and {"note": ""} result in Note="". Go's zero value for a string is "". Therefore, absent and empty cannot be distinguished after unmarshaling.
// A generic request object we will receive from the client
type CreateThing struct {
Name string `json:"name"`
Note string `json:"note,omitempty"`
}
// A generic request object we will receive from the client
type CreateThing struct {
Name string `json:"name"`
Note string `json:"note,omitempty"`
}
We will create some thing that has a Name and Note field. Our intention is that Note is a purely optional field. The caller may include it or leave it out, but if they do include it, it has to be a real value. An explicit empty string ("") is meaningless and we want to reject it with a 400 and an explicit error.
We can run a simple test to see how a request with a note omitted and a note set to "" will be handled:
package main
import (
"encoding/json"
"fmt"
)
type CreateThing struct {
Name string `json:"name"`
Note string `json:"note,omitempty"`
}
func main() {
// A caller who omits the optional field entirely.
omitted := []byte(`{ "name": "bill" }`)
// A caller who sends it, but blank.
blank := []byte(`{ "name": "bill", "note": "" }`)
var a, b CreateThing
json.Unmarshal(omitted, &a)
json.Unmarshal(blank, &b)
fmt.Printf("omitted: Name=%q Note=%q\n", a.Name, a.Note)
fmt.Printf("blank: Name=%q Note=%q\n", b.Name, b.Note)
fmt.Printf("Note equal? %v\n", a.Note == b.Note)
}
package main
import (
"encoding/json"
"fmt"
)
type CreateThing struct {
Name string `json:"name"`
Note string `json:"note,omitempty"`
}
func main() {
// A caller who omits the optional field entirely.
omitted := []byte(`{ "name": "bill" }`)
// A caller who sends it, but blank.
blank := []byte(`{ "name": "bill", "note": "" }`)
var a, b CreateThing
json.Unmarshal(omitted, &a)
json.Unmarshal(blank, &b)
fmt.Printf("omitted: Name=%q Note=%q\n", a.Name, a.Note)
fmt.Printf("blank: Name=%q Note=%q\n", b.Name, b.Note)
fmt.Printf("Note equal? %v\n", a.Note == b.Note)
}
go run value_field_test.go
omitted: Name="bill" Note=""
blank: Name="bill" Note=""
Note equal? true
$ go run value_field_test.go
omitted: Name="bill" Note=""
blank: Name="bill" Note=""
Note equal? true
Both requests will result in Note == "", but each represents two different intents. The first is a caller correctly omitting an optional field, which we want to accept. The second is a caller sending a blank value, which we want to reject.
If you are wondering why we can't use
validate:"required", keep in mind that validation runs after unmarshaling and would therefore be unable to distinguish between these two states.
If a field is a pointer (*string), then both {} and {"note": null} result in Note=nil. Go does not allocate the pointer for null, and it does not allocate it for a missing key. Therefore, absent and explicit-null cannot be distinguished after unmarshaling. Consider a PATCH request where we want the client to be able to selectively update some fields:
// A generic request object we will receive from the client
type UpdateThing struct {
Name *string `json:"name,omitempty"`
Note *string `json:"note,omitempty"`
}
// A generic request object we will receive from the client
type UpdateThing struct {
Name *string `json:"name,omitempty"`
Note *string `json:"note,omitempty"`
}
A client who sends { "name": "bob" } is communicating that they want to update the name field while leaving the note field unchanged. A client who sends {"note": null} is communicating that they wish to erase the note field. After json.Unmarshal, both are nil:
package main
import (
"encoding/json"
"fmt"
)
type UpdateThing struct {
Name *string `json:"name,omitempty"`
Note *string `json:"note,omitempty"`
}
func main() {
// A caller updating name, leaving note out entirely.
absent := []byte(`{ "name": "bill" }`)
// A caller updating name, explicitly nulling note.
null := []byte(`{ "name": "bill", "note": null }`)
var a, b UpdateThing
json.Unmarshal(absent, &a)
json.Unmarshal(null, &b)
fmt.Printf("absent: Note=%v (nil? %v)\n", a.Note, a.Note == nil)
fmt.Printf("null: Note=%v (nil? %v)\n", b.Note, b.Note == nil)
fmt.Printf("both nil? %v\n", a.Note == nil && b.Note == nil)
}
package main
import (
"encoding/json"
"fmt"
)
type UpdateThing struct {
Name *string `json:"name,omitempty"`
Note *string `json:"note,omitempty"`
}
func main() {
// A caller updating name, leaving note out entirely.
absent := []byte(`{ "name": "bill" }`)
// A caller updating name, explicitly nulling note.
null := []byte(`{ "name": "bill", "note": null }`)
var a, b UpdateThing
json.Unmarshal(absent, &a)
json.Unmarshal(null, &b)
fmt.Printf("absent: Note=%v (nil? %v)\n", a.Note, a.Note == nil)
fmt.Printf("null: Note=%v (nil? %v)\n", b.Note, b.Note == nil)
fmt.Printf("both nil? %v\n", a.Note == nil && b.Note == nil)
}
go run pointer_field_test.go
absent: Note=<nil> (nil? true)
null: Note=<nil> (nil? true)
both nil? true
$ go run pointer_field_test.go
absent: Note=<nil> (nil? true)
null: Note=<nil> (nil? true)
both nil? true
Once again, it is difficult to ascertain the original intention of the request.
How we fixed this
Rather than fighting the decoder or creating a custom UnmarshalJSON per request struct, we give fields a type that carries the missing bit of information. Each one has a custom UnmarshalJSON method (this is a hook Go calls instead of its default decoding) so it can record exactly what the client sent.
-
field.Optional[T]: Used on fields that may be optionally set, but not cleared. ItsUnmarshalJSONrecords a value when one is present, and returns an error for an explicitnullso we can reject it with a precise message. An absent key leaves it unset. -
field.Clearable[T]: Used on fields that may be set, cleared (null), or absent. This is the only request shape that acceptsnull.
These types are always used as a value (i.e. field.Optional[string], not *field.Optional[string]) with json:"<name>,omitzero". We enforce this convention with a unit test and at startup. When an endpoint is registered, we walk its request struct (including embedded structs) and panic if any Optional or Clearable field is declared as a pointer. A *field.Clearable[T] would silently drop null clears, and a *field.Optional[T] would silently accept a null it is supposed to reject, so it is necessary to add protections against this footgun.
Go 1.24 added
omitzero, which is likeomitemptybut uses zero-value semantics rather than JSON “empty” semantics. If the field type has anIsZero() boolmethod,encoding/jsoncalls it to decide whether the field should be omitted. Otherwise, it uses the type's ordinary zero value. This is handy becausefield.Optional[T]andfield.Clearable[T]are structs, soomitemptyalone would still encode an unset value as{}. Withomitzero, we can omit unset records:// IsZero reports whether the field is unset so encoding/json omitempty omits it. func (n Optional[T]) IsZero() bool { return n.IsUnset() }// IsZero reports whether the field is unset so encoding/json omitempty omits it. func (n Optional[T]) IsZero() bool { return n.IsUnset() }
In summary:
| Field declaration | Context | What the caller may send |
|---|---|---|
Name string `json:"name" validate:"required"` | create / update | A required value. Omitting it or sending null/"" is a 400. |
Note field.Optional[string] `json:"note,omitzero"` | create / update | An optional value. An explicit null (or blank "") is rejected. |
Note field.Clearable[string] `json:"note,omitzero"` | update PATCH | A value to set, null to clear, or omit to leave unchanged. |
On responses
On a request we no longer use pointers (e.g. *string) for optional fields at all. Instead, we use field.Optional and field.Clearable since they are unambiguous. On a response we can use a pointer, because we decided responses will only ever have two states (a value or null). That way clients can rely on a certain response shape without worrying that sometimes fields may be omitted from the response. If this rule ever needed to be broken in the future, it would be trivial to migrate all endpoints to use field.Optional and field.Clearable.
I'm not sure this is the best solution, but so far it has helped clear up the ambiguity of the json tag and prevent footguns.
Step 3: The binding pipeline
Under the hood, the transport layer precomputes a bind plan once per request type and caches it. The plan walks the struct shape, records every field that can be bound from headers/path/query:
type bindPlan struct {
fields []bindField
allowedQuery map[string]struct{}
}
func planFor(dst any) (*bindPlan, error) {
rv := reflect.ValueOf(dst)
if rv.Kind() != reflect.Pointer || rv.IsNil() {
return nil, errors.New("destination must be a non-nil pointer")
}
t := rv.Type().Elem()
if t.Kind() != reflect.Struct {
return nil, errors.New("destination must point to a struct")
}
if cached, ok := bindPlanCache.Load(t); ok {
return cached.(*bindPlan), nil
}
plan := buildBindPlan(t)
if actual, loaded := bindPlanCache.LoadOrStore(t, plan); loaded {
return actual.(*bindPlan), nil
}
return plan, nil
}
type bindPlan struct {
fields []bindField
allowedQuery map[string]struct{}
}
func planFor(dst any) (*bindPlan, error) {
rv := reflect.ValueOf(dst)
if rv.Kind() != reflect.Pointer || rv.IsNil() {
return nil, errors.New("destination must be a non-nil pointer")
}
t := rv.Type().Elem()
if t.Kind() != reflect.Struct {
return nil, errors.New("destination must point to a struct")
}
if cached, ok := bindPlanCache.Load(t); ok {
return cached.(*bindPlan), nil
}
plan := buildBindPlan(t)
if actual, loaded := bindPlanCache.LoadOrStore(t, plan); loaded {
return actual.(*bindPlan), nil
}
return plan, nil
}
buildBindPlan recurses into embedded structs, follows pointers to structs when they are being used as nested objects, and treats tagged leaf fields as the actual binding targets. A small list keeps track of how to navigate from the root value to each leaf field later on.
Step 4: The Execute method
Execute methodOnce the plan exists, Execute calls BindIncomingRequest which applies headers, path params, and query params in a single pass. Type conversion is handled in a single setFromString helper so endpoints can simply declare their types and tags.
Execute is a single method shared by every endpoint. It handles the entire HTTP request lifecycle. Here is an idea of how this works (simplified a bit):
func (e *APIEndpoint[TReq, TResp]) Execute(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
e.ensureSensitivePaths()
// 1. Check API version requirements
if e.MinVersion != nil {
if requestVersion, ok := appctx.GetAPIVersionFromContext(ctx); ok {
if requestVersion.Before(*e.MinVersion) {
respondWithError(w, apierror.NewAPIVersionTooOldError(...))
return
}
}
}
// 2. Stamp the request log with sensitive-response paths so it can redact
if rl, ok := appctx.GetRequestLog(ctx); ok {
if e.Extras.SkipRequestLogging { rl.SkipSave = true }
if len(e.sensitiveRespPaths) > 0 {
rl.SensitiveResponseFields = maps.Clone(e.sensitiveRespPaths)
}
}
// 3. Extract idempotency key for downstream services (falls back to request log ID)
if idempotencyKey := r.Header.Get(header.IdempotencyKeyHeader); idempotencyKey != "" {
ctx = appctx.WithIdempotencyKey(ctx, idempotencyKey)
} else if rl, ok := appctx.GetRequestLog(ctx); ok && rl != nil && rl.ID != "" {
ctx = appctx.WithIdempotencyKey(ctx, rl.ID)
}
// 4. Allocate the request struct
var req TReq
req = httptransport.AllocIfPtr(req) // ensures *SomeStruct isn't nil
// 5. Bind from headers/path/query in a single pass
includesEnabled := e.IncludeConfig != nil
if err := httptransport.BindIncomingRequest(r, any(req), includesEnabled); err != nil {
respondWithError(w, err)
return
}
// 6. Parse & validate include parameters against e.IncludeConfig
requestedIncludes, apiErr := e.parseIncludeTree(r)
if apiErr != nil { respondWithError(w, apiErr); return }
// 7. Decode JSON body (if present)
if !e.Extras.SkipRequestBodyParsing && shouldDecodeBody(r) {
// Buffer body (1 MiB cap) for logging, null detection, and version transformation
jsonBodyBytes, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20))
// Optional: transform body from older API version to latest format
if e.ObjectType != "" {
if reqVersion, ok := getAPIVersion(ctx); ok && !reqVersion.Equal(version.Latest) {
r, _ = e.transformRequestBody(r, reqVersion, version.Latest)
}
}
// Decode with strict unknown field rejection + Levenshtein suggestions
if err := httptransport.DecodeJSONInto(any(req), r, true); err != nil {
respondWithError(w, err)
return
}
// Record slice presence (value Clearable/Optional fields already captured null/absent during decode)
validate.ApplySlicePresenceFlags(jsonBodyBytes, any(req))
// Reject explicit `null` on fields that don't support it
if apiErr := validate.RejectExplicitJSONNulls(jsonBodyBytes, any(req)); apiErr != nil {
respondWithError(w, apiErr); return
}
} else if e.Extras.SkipRequestBodyParsing {
// For endpoints that handle raw bodies (e.g. file uploads)
httptransport.BindRawBody(r, any(req))
}
// 8. Reject empty PATCH bodies
if r.Method == http.MethodPatch && len(jsonBodyBytes) > 0 {
if apiErr := validate.RejectEmptyPatchBody(jsonBodyBytes, any(req)); apiErr != nil {
respondWithError(w, apiErr); return
}
}
// 9. Validate enum fields and the fully-populated struct
if apiErr := httptransport.ValidateEnumFields(any(req)); apiErr != nil { respondWithError(w, apiErr); return }
if apiErr := validate.Validate(any(req)); apiErr != nil { respondWithError(w, apiErr); return }
// 10. Put include set in context for downstream service to use
if requestedIncludes != nil { ctx = appctx.WithRequestedIncludes(ctx, requestedIncludes) }
// 11. Call the business logic
resp, err := e.boundServiceHandler(ctx, req)
// If the client disconnected during processing, report a 499
if r.Context().Err() == context.Canceled {
respondWithError(w, apierror.NewClientClosedRequestError("Client closed the connection."))
return
}
if err != nil { respondWithError(w, err); return }
// 12. Handle response
var respondOpts []httptransport.RespondOption
if e.SuccessStatusCode == http.StatusCreated && e.LocationFunc != nil {
respondOpts = append(respondOpts, httptransport.WithLocation(e.LocationFunc(resp)))
}
// File-download response (e.g. Excel export)
if fd, ok := any(resp).(*httptransport.FileDownload); ok {
httptransport.RespondWithFile(ctx, w, e.SuccessStatusCode, fd, respondOpts...)
return
}
if e.IncludeConfig != nil {
// resourcekit-driven: batch-load requested sub-resources, stitch onto the response, and collapse anything not requested to null.
e.respondWithIncludes(ctx, w, resp, requestedIncludes, respondOpts...)
} else {
httptransport.RespondWithJSON(ctx, w, e.SuccessStatusCode, resp, respondOpts...)
}
}
func (e *APIEndpoint[TReq, TResp]) Execute(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
e.ensureSensitivePaths()
// 1. Check API version requirements
if e.MinVersion != nil {
if requestVersion, ok := appctx.GetAPIVersionFromContext(ctx); ok {
if requestVersion.Before(*e.MinVersion) {
respondWithError(w, apierror.NewAPIVersionTooOldError(...))
return
}
}
}
// 2. Stamp the request log with sensitive-response paths so it can redact
if rl, ok := appctx.GetRequestLog(ctx); ok {
if e.Extras.SkipRequestLogging { rl.SkipSave = true }
if len(e.sensitiveRespPaths) > 0 {
rl.SensitiveResponseFields = maps.Clone(e.sensitiveRespPaths)
}
}
// 3. Extract idempotency key for downstream services (falls back to request log ID)
if idempotencyKey := r.Header.Get(header.IdempotencyKeyHeader); idempotencyKey != "" {
ctx = appctx.WithIdempotencyKey(ctx, idempotencyKey)
} else if rl, ok := appctx.GetRequestLog(ctx); ok && rl != nil && rl.ID != "" {
ctx = appctx.WithIdempotencyKey(ctx, rl.ID)
}
// 4. Allocate the request struct
var req TReq
req = httptransport.AllocIfPtr(req) // ensures *SomeStruct isn't nil
// 5. Bind from headers/path/query in a single pass
includesEnabled := e.IncludeConfig != nil
if err := httptransport.BindIncomingRequest(r, any(req), includesEnabled); err != nil {
respondWithError(w, err)
return
}
// 6. Parse & validate include parameters against e.IncludeConfig
requestedIncludes, apiErr := e.parseIncludeTree(r)
if apiErr != nil { respondWithError(w, apiErr); return }
// 7. Decode JSON body (if present)
if !e.Extras.SkipRequestBodyParsing && shouldDecodeBody(r) {
// Buffer body (1 MiB cap) for logging, null detection, and version transformation
jsonBodyBytes, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20))
// Optional: transform body from older API version to latest format
if e.ObjectType != "" {
if reqVersion, ok := getAPIVersion(ctx); ok && !reqVersion.Equal(version.Latest) {
r, _ = e.transformRequestBody(r, reqVersion, version.Latest)
}
}
// Decode with strict unknown field rejection + Levenshtein suggestions
if err := httptransport.DecodeJSONInto(any(req), r, true); err != nil {
respondWithError(w, err)
return
}
// Record slice presence (value Clearable/Optional fields already captured null/absent during decode)
validate.ApplySlicePresenceFlags(jsonBodyBytes, any(req))
// Reject explicit `null` on fields that don't support it
if apiErr := validate.RejectExplicitJSONNulls(jsonBodyBytes, any(req)); apiErr != nil {
respondWithError(w, apiErr); return
}
} else if e.Extras.SkipRequestBodyParsing {
// For endpoints that handle raw bodies (e.g. file uploads)
httptransport.BindRawBody(r, any(req))
}
// 8. Reject empty PATCH bodies
if r.Method == http.MethodPatch && len(jsonBodyBytes) > 0 {
if apiErr := validate.RejectEmptyPatchBody(jsonBodyBytes, any(req)); apiErr != nil {
respondWithError(w, apiErr); return
}
}
// 9. Validate enum fields and the fully-populated struct
if apiErr := httptransport.ValidateEnumFields(any(req)); apiErr != nil { respondWithError(w, apiErr); return }
if apiErr := validate.Validate(any(req)); apiErr != nil { respondWithError(w, apiErr); return }
// 10. Put include set in context for downstream service to use
if requestedIncludes != nil { ctx = appctx.WithRequestedIncludes(ctx, requestedIncludes) }
// 11. Call the business logic
resp, err := e.boundServiceHandler(ctx, req)
// If the client disconnected during processing, report a 499
if r.Context().Err() == context.Canceled {
respondWithError(w, apierror.NewClientClosedRequestError("Client closed the connection."))
return
}
if err != nil { respondWithError(w, err); return }
// 12. Handle response
var respondOpts []httptransport.RespondOption
if e.SuccessStatusCode == http.StatusCreated && e.LocationFunc != nil {
respondOpts = append(respondOpts, httptransport.WithLocation(e.LocationFunc(resp)))
}
// File-download response (e.g. Excel export)
if fd, ok := any(resp).(*httptransport.FileDownload); ok {
httptransport.RespondWithFile(ctx, w, e.SuccessStatusCode, fd, respondOpts...)
return
}
if e.IncludeConfig != nil {
// resourcekit-driven: batch-load requested sub-resources, stitch onto the response, and collapse anything not requested to null.
e.respondWithIncludes(ctx, w, resp, requestedIncludes, respondOpts...)
} else {
httptransport.RespondWithJSON(ctx, w, e.SuccessStatusCode, resp, respondOpts...)
}
}
Step 5: Grouping and registration
Endpoints are organized into groups by resource. Each group knows how to create its service and wire up its endpoints. apiendpoint.From(...) calls the wrapper's Materialize() method.
func (*APIKeysEndpointGroup) Materialize(config *APIKeysEndpointGroupConfig) *APIKeysEndpointGroup {
if err := config.validate(); err != nil {
panic(err)
}
apiKeySvc := apikeyep.NewAPIKeySvc(&apikeyep.APIKeySvcConfig{
AuthClient: config.AuthClient.Client,
})
authMw := middleware.AuthMiddleware(&middleware.AuthMiddlewareConfig{
AuthClient: config.AuthClient,
})
inner := &apiendpoint.APIEndpointGroup{
Title: "API Key Management",
Description: "Create and manage API keys for programmatic access.",
ResourceType: &apiresource.APIKey{},
}
getAPIKeyEndpoint := apiendpoint.From(&apikeyep.RetrieveAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
listAPIKeysEndpoint := apiendpoint.From(&apikeyep.ListAPIKeysEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
createAPIKeyEndpoint := apiendpoint.From(&apikeyep.CreateAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
rotateAPIKeyEndpoint := apiendpoint.From(&apikeyep.RotateAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
revokeAPIKeyEndpoint := apiendpoint.From(&apikeyep.RevokeAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
getDocAPIKeyEndpoint := apiendpoint.From(&apikeyep.GetDocAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
inner.Endpoints = []apiendpoint.APIEndpointer{
getAPIKeyEndpoint,
listAPIKeysEndpoint,
createAPIKeyEndpoint,
rotateAPIKeyEndpoint,
revokeAPIKeyEndpoint,
getDocAPIKeyEndpoint,
}
return &APIKeysEndpointGroup{inner}
}
func (*APIKeysEndpointGroup) Materialize(config *APIKeysEndpointGroupConfig) *APIKeysEndpointGroup {
if err := config.validate(); err != nil {
panic(err)
}
apiKeySvc := apikeyep.NewAPIKeySvc(&apikeyep.APIKeySvcConfig{
AuthClient: config.AuthClient.Client,
})
authMw := middleware.AuthMiddleware(&middleware.AuthMiddlewareConfig{
AuthClient: config.AuthClient,
})
inner := &apiendpoint.APIEndpointGroup{
Title: "API Key Management",
Description: "Create and manage API keys for programmatic access.",
ResourceType: &apiresource.APIKey{},
}
getAPIKeyEndpoint := apiendpoint.From(&apikeyep.RetrieveAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
listAPIKeysEndpoint := apiendpoint.From(&apikeyep.ListAPIKeysEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
createAPIKeyEndpoint := apiendpoint.From(&apikeyep.CreateAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
rotateAPIKeyEndpoint := apiendpoint.From(&apikeyep.RotateAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
revokeAPIKeyEndpoint := apiendpoint.From(&apikeyep.RevokeAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
getDocAPIKeyEndpoint := apiendpoint.From(&apikeyep.GetDocAPIKeyEndpoint{}).WithMiddleware(authMw).WithService(inner, apiKeySvc)
inner.Endpoints = []apiendpoint.APIEndpointer{
getAPIKeyEndpoint,
listAPIKeysEndpoint,
createAPIKeyEndpoint,
rotateAPIKeyEndpoint,
revokeAPIKeyEndpoint,
getDocAPIKeyEndpoint,
}
return &APIKeysEndpointGroup{inner}
}
At startup, all groups are materialized and registered with the router:
func (r *router) InitAuthEndpointGroups(config AuthRouterConfig) {
registry := NewRegistry()
r.AddMiddleware(middleware.TracingMiddleware())
r.AddMiddleware(middleware.CORSMiddleware())
r.AddMiddleware(middleware.RateLimitMiddleware())
r.AddMiddleware(middleware.VersionMiddleware())
r.AddMiddleware(middleware.IdempotencyMiddleware(config))
r.AddMiddleware(middleware.RecoverMiddleware())
apiKeysGroup := (&httpgroup.APIKeysEndpointGroup{}).Materialize(&httpgroup.APIKeysEndpointGroupConfig{
AuthClient: config.AuthClient,
CoreClient: config.CoreClient,
})
registry.RegisterGroup(apiKeysGroup.APIEndpointGroup)
// ... more groups ...
registry.RegisterEndpoints(r)
}
func (r *router) InitAuthEndpointGroups(config AuthRouterConfig) {
registry := NewRegistry()
r.AddMiddleware(middleware.TracingMiddleware())
r.AddMiddleware(middleware.CORSMiddleware())
r.AddMiddleware(middleware.RateLimitMiddleware())
r.AddMiddleware(middleware.VersionMiddleware())
r.AddMiddleware(middleware.IdempotencyMiddleware(config))
r.AddMiddleware(middleware.RecoverMiddleware())
apiKeysGroup := (&httpgroup.APIKeysEndpointGroup{}).Materialize(&httpgroup.APIKeysEndpointGroupConfig{
AuthClient: config.AuthClient,
CoreClient: config.CoreClient,
})
registry.RegisterGroup(apiKeysGroup.APIEndpointGroup)
// ... more groups ...
registry.RegisterEndpoints(r)
}
The registry simply iterates and calls GetHandler() on each endpoint:
func (r *Registry) RegisterEndpoints(router *router) {
for _, group := range r.groups {
for _, endpointer := range group.Endpoints {
router.HandleEndpoint(
endpointer.GetMethod(),
endpointer.GetRoute(),
endpointer.GetHandler(),
endpointer.IsPublic(),
)
}
}
}
func (r *Registry) RegisterEndpoints(router *router) {
for _, group := range r.groups {
for _, endpointer := range group.Endpoints {
router.HandleEndpoint(
endpointer.GetMethod(),
endpointer.GetRoute(),
endpointer.GetHandler(),
endpointer.IsPublic(),
)
}
}
}
Step 6: Write your service handlers
So far, everything we have seen stops at the HTTP boundary, but we still need to call the backend and shape the result into the API resource types.
To do this, we define a small service interface with one method per endpoint, making sure the signatures mirror the endpoint's generic types:
type APIKeySvc interface {
CreateAPIKey(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError)
// ...
}
type APIKeySvc interface {
CreateAPIKey(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError)
// ...
}
The ServiceHandler field wires this at compile time. If the signature doesn't match the endpoint's generic parameters, the code will not compile.
When Execute calls your handler, the request is already bound and validated so you only need to do a few things:
- Map the typed request into a protobuf.
- Call the backend through a shared gRPC helper.
- Map the response into an
apiresourcetype and return it.
func (m *apiKeySvcImpl) CreateAPIKey(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError) {
// ...
resp, apiErr := grpcutil.CallRPC(ctx, apiKeySvcTracer, "service.api_keys.create", domain.ServiceName,
func(ctx context.Context, opts ...grpc.CallOption) (*pb.CreateAPIKeyResponse, error) {
return m.authClient.CreateAPIKey(ctx, pbReq, opts...)
})
if apiErr != nil {
return nil, apiErr
}
return &apiresource.CreatedAPIKey{
Object: constants.ObjectTypeCreatedAPIKey,
APIKeySecret: resp.ApiKeySecret,
APIKeyInfo: resp.ApiKey,
}, nil
}
func (m *apiKeySvcImpl) CreateAPIKey(ctx context.Context, req *CreateAPIKeyRequest) (*apiresource.CreatedAPIKey, *apierror.APIError) {
// ...
resp, apiErr := grpcutil.CallRPC(ctx, apiKeySvcTracer, "service.api_keys.create", domain.ServiceName,
func(ctx context.Context, opts ...grpc.CallOption) (*pb.CreateAPIKeyResponse, error) {
return m.authClient.CreateAPIKey(ctx, pbReq, opts...)
})
if apiErr != nil {
return nil, apiErr
}
return &apiresource.CreatedAPIKey{
Object: constants.ObjectTypeCreatedAPIKey,
APIKeySecret: resp.ApiKeySecret,
APIKeyInfo: resp.ApiKey,
}, nil
}
Conclusion
We have really enjoyed operating this framework and have noticed a number of advantages:
- Consistency: Since every endpoint handles errors, validation, and responses identically, the API as a whole starts to feel very predictable.
- Cross-cutting concerns are free: When we added API versioning, we added it once in
Executeand every endpoint got it for free. Same for idempotency keys, unknown parameter rejection, include/expand support, and client-disconnect 499s. - Testability: You can test
Executein isolation by providing mock service handlers, which makes it really simple to test all your transport-layer concerns across all your endpoints. - No copy-paste drift: Developers cannot accidentally implement a subtly different error-handling path because they never write handler code.
- Readability: Defining everything about an endpoint in one file makes it really easy to understand how a particular endpoint operates at a glance.
- There is only one source of truth: The endpoint definitions can be used to drive behavior and the OpenAPI spec generation process.