Code Generation
Droplet ORM uses no reflection at runtime. You describe entities as plain Go structs (see Entities), register them in a Registry, and run orm.Generate(). The generator emits one Go file per entity containing a typed Provider (loading, searching, factory and metadata methods) and a typed Entity (getters, setters, dirty tracking and the SQL it needs to persist itself), plus a shared enums package. This page is a reference for what is generated and how to call it.
Running the generator
func Generate(engine Engine, outputDirectory string) error
Generate needs a validated Engine. Use registry.ValidateForCodeGen() rather than Validate(): it builds the entity schemas without opening a single connection to MySQL, Redis or NATS, so the generator can run on a machine that has no access to the databases (a developer laptop, CI). Pools still have to be registered so that tags such as orm:"mysql=logs" or orm:"redisCache" can be resolved, but the addresses are never dialled.
Put the call in a small program and run it whenever an entity struct changes:
// cmd/generate/main.go
package main
import (
"os"
"github.com/DropletLive/orm"
"myapp/model"
)
func main() {
registry := orm.NewRegistry()
registry.RegisterMySQL("user:password@tcp(localhost:3306)/db", orm.DefaultPoolCode, &orm.MySQLOptions{})
registry.RegisterRedis("localhost:6379", 0, orm.DefaultPoolCode, nil)
registry.RegisterEntity(model.CategoryEntity{}, model.UserEntity{})
engine, err := registry.ValidateForCodeGen()
if err != nil {
panic(err)
}
if err = os.MkdirAll("entities", 0o755); err != nil {
panic(err)
}
if err = orm.Generate(engine, "entities"); err != nil {
panic(err)
}
}
go run ./cmd/generate
What Generate does, in order:
- Checks the output directory: it must be non-empty (
output directory is empty), exist (output directory does not exist: <path>), be a directory and be writable. - Walks up from the output directory until it finds a
go.modand reads itsmoduleline. The import path of the generatedenumspackage is<module>/<relative path of the output dir>/enums, so the output directory must live inside a Go module. - Deletes every regular file in the output directory, whatever its extension. Sub-directories (such as
enums/) are kept: the file of every enum still in use is rewritten, but a file left over from an enum that no longer exists is not removed. Never put hand-written files in the output directory. - Writes one file per registered entity, then
providers.go, then the consumer files when consumers or tasks are registered. - Deletes every regular file in
<outputDirectory>/esfields/and, when Elasticsearch documents are registered, writes one file per document there. - Runs every file through
go/format.
The package name of the generated code is the base name of the output directory (entities in the example above). Commit the generated files together with the entity structs that produced them.
Upgrading Droplet ORM
Regenerate all entities after updating the ORM dependency, even when your model structs have not changed. The generator and runtime must agree on the entity persistence protocol.
The transaction save fix requires generated snapshot and rollback support, including PrivateSnapshot, PrivateRollback and PrivateIsSnapshot. Older generated entities can still compile, but Save returns orm.ErrEntityNeedsRegeneration rather than using an entity without this support. Run your generator with the updated dependency, then rebuild and test the application. Save, Transaction and the public setters keep their existing signatures.
The change feed requires regenerated code as well: <Entity>DirtyEvent is now orm.DirtyEvent[<Entity>] with read-only entities in Before/After, and the generated init() calls orm.RegisterCDCEntity. Code generated for the previous event format does not compile against the current runtime; handlers that read map[string]any snapshots must be rewritten to use getters (see Entity Events).
The GetByID and GetByIDs allocation improvements are emitted into generated providers. Rerun your application generator with the updated ORM dependency to adopt them; changing the dependency alone does not rewrite existing providers. These optimizations preserve method signatures, cache behavior, and missing-row behavior, as well as input order and duplicate handling in GetByIDs. Newly generated readers require the updated ORM runtime for numeric scan adapters and Redis command batching; database/sql conversions, NULL handling, overflow errors, and the Redis row-cache format are preserved.
Output layout
| File | Content |
|---|---|
<TableName>.go | One file per entity. The file name is the table name verbatim: UserEntity.go for a struct UserEntity, users.go when the struct carries orm:"table=users". |
providers.go | var AllProviders = []orm.EntityProvider{...}, pointers to every provider sorted by table name. |
consumers.go, <consumer>_consumer.go | Only when consumers or tasks are registered. See Consumers and Tasks. |
enums/<EnumName>.go | Package enums, one file per enum name (see Enums). |
esfields/<document_name>.go | Package esfields, one file per registered Elasticsearch document (see Elasticsearch fields). |
Naming rules
- The table name is the struct type name unless the
IDfield carriesorm:"table=...". - The entity type is the table name with its first letter capitalised and underscores removed, capitalising each segment:
UserEntitystaysUserEntity, a tableuser_accountsbecomesUserAccounts. - The provider variable is
<Entity>Provider(UserEntityProvider). Its type, the fields struct and the SQL row struct are unexported (userEntityProvider,userEntityFields,userEntitySQLRow). - The cache index stored in the provider is the fully qualified Go type name of the source struct (
model.UserEntity). It is assigned at validation time and used as the key of the context cache and of the lifecycle handlers.
The examples on this and the following pages use these two structs, defined in a package model and generated into a package entities:
package model
import (
"time"
"github.com/DropletLive/orm"
)
type CategoryEntity struct {
ID uint64 `orm:"redisCache"`
Code string `orm:"required;length=10"`
Name string `orm:"required;length=100"`
}
func (e CategoryEntity) UniqueIndexes() [][]string {
return [][]string{{"Code"}}
}
func (e CategoryEntity) CachedUniqueIndexes() [][]string {
return [][]string{{"Code"}}
}
type UserEntity struct {
ID uint64 `orm:"redisCache"`
Name string `orm:"required;length=100"`
Email string `orm:"length=255"`
Age *uint8
Status string `orm:"enum=active,blocked;required;enumName=UserStatus"`
Category orm.Reference[CategoryEntity] `orm:"required"`
CreatedAt time.Time
UpdatedAt time.Time
}
Anatomy of a generated file
The excerpt below is what entities/UserEntity.go looks like (the hex values are computed from the table and column names and differ for every entity; method bodies are omitted).
// Code generated by orm; DO NOT EDIT.
package entities
type userEntityFields struct {
ID orm.UintField
Category orm.ReferenceField
CreatedAt orm.TimeField
UpdatedAt orm.TimeField
Name orm.StringField
Email orm.NullableStringField
Age orm.NullableUintField
Status orm.EnumField
}
type userEntityProvider struct {
tableName string
dbCode string
redisCode string
cacheIndex string
redisCachePrefix string
redisCacheStamp string
redisCacheTTL int
Fields userEntityFields
}
var UserEntityProvider = userEntityProvider{
tableName: "UserEntity",
dbCode: "default",
redisCode: "default",
cacheIndex: "model.UserEntity",
redisCachePrefix: "e11dd246:",
redisCacheStamp: "e69f360206e0533c",
redisCacheTTL: 0,
Fields: userEntityFields{
ID: orm.UintField{Column: "ID"},
Category: orm.ReferenceField{Column: "Category"},
// ...
},
}
func (p userEntityProvider) TableName() string { return p.tableName }
func (p userEntityProvider) DBCode() string { return p.dbCode }
func (p userEntityProvider) RedisCode() string { return p.redisCode }
func (p userEntityProvider) RedisCachePrefix() string { return p.redisCachePrefix }
func (p userEntityProvider) ClearRedisCache(ctx orm.Context) (int, error)
type userEntitySQLRow struct {
F0 uint64
F1 uint64
F2 time.Time
F3 time.Time
F4 string
F5 sql.NullString
F6 sql.NullInt64
F7 string
}
func (p userEntityProvider) OnAfterInsert(engine orm.Engine, handler func(ctx orm.Context, entity *UserEntity) error)
func (p userEntityProvider) OnAfterUpdate(engine orm.Engine, handler func(ctx orm.Context, entity *UserEntity, changes map[string]any) error)
func (p userEntityProvider) OnAfterDelete(engine orm.Engine, handler func(ctx orm.Context, entity *UserEntity) error)
func RegisterUserEntityBeforeInsert(cb func(entity *UserEntity))
func RegisterUserEntityBeforeUpdate(cb func(entity *UserEntity))
func RegisterUserEntityBeforeDelete(cb func(entity *UserEntity))
func (p userEntityProvider) GetByID(ctx orm.Context, id uint64) (entity *UserEntity, err error)
func (p userEntityProvider) GetByIDs(ctx orm.Context, id ...uint64) ([]*UserEntity, error)
func (p userEntityProvider) New(ctx orm.Context) *UserEntity
func (p userEntityProvider) NewWithID(ctx orm.Context, id uint64) *UserEntity
func (p userEntityProvider) SearchOne(ctx orm.Context, query *orm.DBQuery) (*UserEntity, error)
func (p userEntityProvider) SearchMany(ctx orm.Context, query *orm.DBQuery) ([]*UserEntity, error)
func (p userEntityProvider) SearchManyWithTotal(ctx orm.Context, query *orm.DBQuery) ([]*UserEntity, int, error)
func (p userEntityProvider) Count(ctx orm.Context, query *orm.DBQuery) (int, error)
type UserEntity struct {
ctx orm.Context
id uint64
new bool
deleted bool
removed bool
snapshot bool
originDatabaseValues *userEntitySQLRow
databaseBind map[string]any
originRedisValues []string
flushType uint8
flushChanges map[string]any
}
func (e *UserEntity) GetID() uint64
func (e *UserEntity) GetName() string
func (e *UserEntity) SetName(value string) *UserEntity
func (e *UserEntity) GetEmail() string
func (e *UserEntity) SetEmail(value string) *UserEntity
func (e *UserEntity) GetAge() *uint64
func (e *UserEntity) SetAge(value *uint64) *UserEntity
func (e *UserEntity) GetStatus() enums.UserStatus
func (e *UserEntity) SetStatus(value enums.UserStatus) *UserEntity
func (e *UserEntity) GetCategoryID() uint64
func (e *UserEntity) SetCategory(value uint64) *UserEntity
func (e *UserEntity) GetCategory(ctx orm.Context) (reference *CategoryEntity, err error)
func (e *UserEntity) GetCreatedAt() time.Time
func (e *UserEntity) SetCreatedAt(value time.Time) *UserEntity
func (e *UserEntity) GetUpdatedAt() time.Time
func (e *UserEntity) SetUpdatedAt(value time.Time) *UserEntity
The redisCachePrefix, redisCacheStamp and redisCacheTTL provider fields and the entity's originRedisValues exist only for entities tagged orm:"redisCache"; an entity that only declares CachedUniqueIndexes() gets redisCachePrefix alone. An entity tagged orm:"redisCache;cached" also gets a redisAllKey field and the GetAll method. Entities tagged cdc or outbox get the change-feed block. Entities without redisCache get two unexported row readers (entityFromRow, readRows) used by their Search* methods.
Methods whose name starts with Private (PrivateFlush, PrivateFlushed, PrivateSnapshot, PrivateRollback, PrivateAdvance, PrivateIsSnapshot, PrivateIsDeleted, PrivateDatabasePool, PrivateReload, PrivateDelete, PrivateForceDelete (the latter only on FakeDelete entities), PrivateFlushEvent, PrivateGetDatabaseBind, PrivateCacheIndex, PrivateIsNew, PrivateContext, and on cdc/outbox entities PrivateCDCRow, PrivateCDCChange, PrivateColumnValue) implement the orm.Entity interface or internal capabilities used by ctx.Save, ctx.Delete, ctx.Reload and transactions. They are exported only because the ORM lives in another package; never call them yourself.
The Provider
Field descriptors
Provider.Fields holds one typed descriptor per column. Descriptors build the conditions passed to orm.NewQuery().Filter(...) and the sort arguments of SortByASC/SortByDESC:
users, err := entities.UserEntityProvider.SearchMany(ctx,
orm.NewQuery().
Filter(
entities.UserEntityProvider.Fields.Category.Eq(categoryID),
entities.UserEntityProvider.Fields.Status.Is(enums.UserStatusList.Active),
).
SortByASC(entities.UserEntityProvider.Fields.Name).
Pager(orm.NewPager(1, 20)),
)
| Go field | Descriptor |
|---|---|
uint* (including ID) | orm.UintField |
*uint* | orm.NullableUintField |
int* / *int* | orm.IntField / orm.NullableIntField |
float* / *float* | orm.FloatField / orm.NullableFloatField |
bool / *bool | orm.BoolField / orm.NullableBoolField |
time.Time / *time.Time | orm.TimeField / orm.NullableTimeField |
string with required | orm.StringField |
string without required | orm.NullableStringField |
string with enum= and required | orm.EnumField |
string with enum= without required | orm.NullableEnumField |
orm.Reference[T] with required | orm.ReferenceField |
orm.Reference[T] without required | orm.NullableUintField |
FakeDelete, []uint8, set= fields, orm.References[T] and JSON struct fields have no descriptor: they cannot be used in Filter. Fields of embedded or nested structs are flattened with the nested field name as prefix (TestSubSize for TestSub.Size), anonymous embedding adds no prefix. The available condition methods are documented in Search.
Metadata methods and interfaces
Every provider implements orm.EntityProvider:
type EntityProvider interface {
TableName() string
DBCode() string
}
Entities with orm:"redisCache" or with CachedUniqueIndexes() additionally implement orm.RedisCacheEntityProvider (an entity that only caches unique index lookups still needs a way to clear those keys):
type RedisCacheEntityProvider interface {
RedisCode() string
RedisCachePrefix() string
ClearRedisCache(ctx Context) (int, error)
}
providers.go collects every provider so that tooling can iterate over them and pick the capabilities it needs:
for _, provider := range entities.AllProviders {
if cached, ok := provider.(orm.RedisCacheEntityProvider); ok {
deleted, err := cached.ClearRedisCache(ctx)
// ...
}
}
Loading, factory and search methods
| Method | Notes |
|---|---|
GetByID(ctx, id) (*E, error) | Context cache, then Redis row cache, then MySQL. A missing row is a *orm.NotFoundError. See CRUD. |
GetByIDs(ctx, id ...uint64) ([]*E, error) | Input order preserved, duplicates collapsed, missing ids skipped. |
New(ctx) *E | New entity with a snowflake ID from ctx.Engine().NextID(); never fails. |
NewWithID(ctx, id) *E | New entity with a caller-chosen ID. |
SearchOne(ctx, *DBQuery) (*E, error) | Uses the cached unique index fast path when the filter matches one exactly. No match is a *orm.NotFoundError. |
SearchMany(ctx, *DBQuery) ([]*E, error) | |
SearchManyWithTotal(ctx, *DBQuery) ([]*E, int, error) | |
Count(ctx, *DBQuery) (int, error) | |
GetAll(ctx) ([]*E, error) | Only with orm:"redisCache;cached": every row, with the set of ids cached in Redis. |
NewSnapshot(id, fill func(*E)) *E | Only with cdc or outbox: a read-only entity built in memory, for tests that hand-build events. See Testing. |
A miss is detected with errors.AsType:
user, err := entities.UserEntityProvider.GetByID(ctx, id)
if _, missing := errors.AsType[*orm.NotFoundError](err); missing {
// no such row
} else if err != nil {
return err
}
orm.NotFoundError has the fields Entity (the table name) and ID (0 for SearchOne and for an unset reference). There is no found flag and no MustGet* variant.
GetByID, GetByIDs, New, NewWithID are described in CRUD; the Search* and Count methods in Search. There is no generated Delete on the provider and no GetBy<Index> methods: deleting goes through ctx.Delete, unique index lookups through SearchOne.
Lifecycle registration
OnAfterInsert, OnAfterUpdate and OnAfterDelete register one post-commit handler per event on the Engine. The package-level Register<Entity>BeforeInsert, Register<Entity>BeforeUpdate and Register<Entity>BeforeDelete functions register synchronous callbacks that run before the SQL statement is built. Both families are documented in Lifecycle Callbacks.
The Entity
An entity remembers the Context that created or loaded it, its ID, its latest loaded or successfully saved values (originDatabaseValues / originRedisValues) and the columns changed since (databaseBind). A successful Save advances that baseline immediately after SQL execution, even before a transaction commits. The ORM keeps separate snapshots for deferred events and rollback; see Transactions. The entity can only be saved on its own context; see CRUD for the write rules.
Getters and setters
Every setter returns the entity, so calls chain:
user := entities.UserEntityProvider.New(ctx)
user.SetName("Alice").
SetEmail("alice@example.com").
SetStatus(enums.UserStatusList.Active).
SetCategory(category.GetID())
| Go field | Getter | Setter |
|---|---|---|
uint* | Get<F>() uint64 | Set<F>(value uint64) |
int* | Get<F>() int64 | Set<F>(value int64) |
float* | Get<F>() float64 | Set<F>(value float64) |
bool | Get<F>() bool | Set<F>(value bool) |
time.Time | Get<F>() time.Time | Set<F>(value time.Time) — truncated to the second (orm:"time") or to the day (date) |
string with required | Get<F>() string | Set<F>(value string) |
string without required | Get<F>() string — "" for NULL | Set<F>(value string) — "" stores NULL |
*uint* | Get<F>() *uint64 | Set<F>(value *uint64) — nil stores NULL |
*int* | Get<F>() *int64 | Set<F>(value *int64) |
*float* | Get<F>() *float64 | Set<F>(value *float64) |
*bool | Get<F>() *bool | Set<F>(value *bool) |
*time.Time | Get<F>() *time.Time | Set<F>(value *time.Time) |
[]uint8 | Get<F>() []uint8 — nil for NULL | Set<F>(value []uint8) |
enum with required | Get<F>() enums.X | Set<F>(value enums.X) |
enum without required | Get<F>() *enums.X | Set<F>(value *enums.X) |
set with required | Get<F>() []enums.X | Set<F>(value ...enums.X) — stored sorted, comma-joined |
set without required | Get<F>() []enums.X — nil for NULL | Set<F>(value ...enums.X) — no arguments stores NULL |
JSON struct *pkg.T | Get<F>() *pkg.T | Set<F>(value *pkg.T) — nil stores NULL |
orm.Reference[T] | Get<F>ID() uint64 (0 for NULL), Get<F>(ctx) (*T, error) | Set<F>(value uint64) — 0 stores NULL on an optional reference |
orm.References[T] | Get<F>IDs() []uint64, Get<F>(ctx) ([]*T, error) | Set<F>IDs(ids []uint64) — empty stores []; nil stores NULL on an optional field |
Narrow Go integers widen in the API: a uint32 column has Get<F>() uint64, a *uint8 column has Get<F>() *uint64. The ID field has only GetID().
Reference getters load lazily through the referenced provider: Get<F>(ctx) returns a *orm.NotFoundError (with ID 0) when the ID is 0, otherwise the result of <Ref>Provider.GetByID(ctx, id), which is a *orm.NotFoundError carrying the ID when the referenced row is missing. Get<F>(ctx) on a References[T] field calls <Ref>Provider.GetByIDs(ctx, ids...).
Change events
Entities tagged orm:"cdc" or orm:"outbox" get a change-feed block at the end of their file. With orm:"cdc" on UserEntity:
// UserEntityDirtyEvent is one UserEntity row change as a listener receives it.
// Before and After are read-only entities: Insert sets only After, Delete only Before.
type UserEntityDirtyEvent = orm.DirtyEvent[UserEntity]
// One entry per column, in SQL row order, with the column's wire encoding.
var userEntityCDCColumns = []orm.CDCColumn{
{Name: "ID", Kind: orm.CDCNumber},
{Name: "Category", Kind: orm.CDCNumber},
{Name: "CreatedAt", Kind: orm.CDCDate},
// ...
}
func (r *userEntitySQLRow) cdcFields() []any
func (e *UserEntity) PrivateCDCRow(withChanges bool) map[string]any
func (e *UserEntity) PrivateCDCChange(op orm.DirtyOp) orm.CDCChange
func (e *UserEntity) PrivateColumnValue(column string) any
func decodeUserEntityCDCRow(ctx orm.Context, image map[string]json.RawMessage) (*UserEntity, error)
func (p userEntityProvider) NewSnapshot(id uint64, fill func(e *UserEntity)) *UserEntity
func init() {
orm.RegisterCDCEntity[UserEntity]("UserEntity")
}
| Member | Purpose |
|---|---|
<Entity>DirtyEvent | The event type handlers receive; Before/After are *<Entity>. |
<entity>CDCColumns, cdcFields() | The codec between the generated SQL row and the Debezium row image (orm.EncodeCDCRow / orm.DecodeCDCRow). Kind is one of orm.CDCNumber, CDCBool, CDCFloat, CDCDecimal (with Scale), CDCDatetime, CDCDate, CDCString, CDCBytes. |
PrivateCDCRow, PrivateCDCChange | Build the row image and the change (c/u/d) one write puts on the feed. |
PrivateColumnValue | Lets orm.WatchFields compare one column of two images. |
decode<Entity>CDCRow | Turns a row image into a read-only entity; the generated On<Entity> consumer methods pass it to orm.BuildDispatch. |
NewSnapshot | Builds a read-only entity in memory through the setters, for tests. |
init() | Puts the entity on the feed under its table name. |
See Entity Events for the envelope and the event semantics.
Elasticsearch fields
For every document registered with registry.RegisterElasticsearchDocument(...), Generate also writes <outputDirectory>/esfields/<document_name>.go (package esfields) with the document's typed field names and its mappings.properties map. The directory is wiped and rewritten on every run, like the output directory itself. See Elasticsearch.
Enums
Every enum= or set= field produces a file in the enums package. The type name is the enumName= tag value; without it the generator derives one from the entity and field name. Several fields — in the same or in different entities — may share one enum by giving them the same enumName; a field may even reference an enum that is defined elsewhere with orm:"enumName=UserStatus" alone.
// Code generated by orm; DO NOT EDIT.
package enums
type UserStatus string
var UserStatusList = struct {
Active UserStatus
Blocked UserStatus
}{
Active: "active",
Blocked: "blocked",
}
func (e UserStatus) Valid() bool {
switch e {
case "active", "blocked":
return true
}
return false
}
func (e UserStatus) Values() []UserStatus {
return []UserStatus{"active", "blocked"}
}
The constant names are the values with the first letter of every _-separated segment capitalised (in_review becomes InReview). A required enum or set field of a freshly created entity defaults to the first declared value, so entities.UserEntityProvider.New(ctx).GetStatus() is enums.UserStatusList.Active.