Custom-marshal Golang structs with flattening
How to marshal a struct field that doesn’t implement the Marshaler interface
Recently, I needed to marshal a Go struct to JSON and BSON (binary JSON, a serialization format developed by MongoDB), but one of the fields in my struct was an interface that needed special handling.
Typically, it’s easy enough to provide custom marshaling functions: just make
MyStruct needs to be marshaled, the marshaler will call our implementation of the marshal functions to marshal the value for
myStruct.Custom. All we need to do is provide
MarshalBSON functions for every implementation of
Unfortunately, in my case, this was not an option. I already had dozens of implementations of my custom type, and a handy function that took any
CustomType implementation and made it marshalable. (Note: for demo purposes, this function will just return the string constant “
field prepared by MarshalableCustomType”.)
I didn’t want to provide
MarshalBSON functions for each implementation of
CustomType; I just wanted the marshaler to call
A common way to do this is to create an auxiliary struct used just for marshaling
MarshalJSON function creates an instance of
myStructMirror that’s exactly the same as the original, except for the
Custom field, which is created by calling
MarshalableCustomType. Then, we marshal the
myStructMirror to get what we want.
This works well, but there’s one obvious annoyance: every time we want to add a field that doesn’t need custom marshaling to
MyStruct, we must also add it to
MarshalJSON (as well as
UnmarshalBSON) in order to ensure the new field gets marshaled and unmarshaled properly.
This makes adding a new field cumbersome and error-prone. While it’s possible to mitigate this with unit tests or compile-time checks, it would be preferable for future engineers to not have to add new fields to multiple places.
Ideally, our auxiliary struct would be modified such that we only specify the
Custom field and its special marshaling behavior. The struct should then fall back to the default behavior for all other fields. To do this, we’ll take advantage of the flattening feature of our marshaling libraries. The code is slightly different for JSON and BSON, so we’ll address them separately.
For every field in a struct, Go’s JSON marshaler will create a new key/value pair in the serialized JSON. There’s one exception to this rule: embedded structs. If a field is an embedded struct of a parent, the child struct’s fields will be flattened, and included on the parent’s level.
Here’s a simple example of flattening:
ChildStruct is embedded in
ParentStruct is marshaled, instead of creating a new key called “ChildStruct”, all the fields in
ChildStruct are simply treated as if they were part of the parent.
We can take advantage of flattening to improve upon the mirror struct strategy from before. Instead of adding the fields from
myStructMirror, we’ll start by creating a type alias for
myStructAlias, and embed it in the mirror struct instead.
Now, when we marshal
Field2 will be marshaled the way they normally would, but
Custom will get marshaled using
MarshalableCustomType. All will be placed at the same level in the output JSON.
We can use the same principle with BSON, but the specifics are a bit different. The MongoDB BSON library respects an
inline struct tag which provides the same behavior as JSON flattening on any struct field, so we don’t need to bother with the type alias for an embedded struct. The code is super simple:
Et voila! When we try to marshal into BSON, we get our new custom-marshal output:
Note: an older version of the mongo-driver BSON package would complain because
Custom shows up twice on the top-level. To solve this, suppress it from the original
MyStruct so that it doesn’t conflict with the marshaling of
In this post, we mostly addressed marshaling, but of course most things that are marshaled need to be unmarshaled as well. Assuming you have an
UnmarshalCustomType function that does the inverse of
MarshalCustomType, it should be fairly simple to write
UnmarshalBSON functions that unmarshal into the auxiliary struct and then convert into
MyStruct. Give it a try yourself!
When possible, it’s still preferable to implement the
bson.Marshaler interfaces so that you don’t need auxiliary structs for marshaling at all. However, when auxiliary structs are unavoidable, embedding and flattening structs can save you from having to add new fields in many different places — preventing major headaches.
Special thanks to Jarrett Gaddy and Chris Warth for reviewing an earlier version of this post.