dzejkop.space

Serde by Example 1: JSON-RPC

serde serde-by-example rust intermediate

I want this to be the serde tutorial I wish I had when I was starting out.

Don't get me wrong. Serde is great. It's easy to use, powerful and has fantastic documentation.

But, when I was just a lowly Rust beginner, that had recently moved from C# and C++ there were moments when I wished for a more by-example walk-through.

I've now been programming in Rust for quite a while and I like to think of myself as proficient with it - which includes familiarity with all the various serde macro attributes.

This is the first in (hopefully) many articles in the series in which I'll pick a real-life format (HTTP response, configuration file, serialization format, etc.) and show you how to design your data types to accommodate them.

In this first article, we'll take a look at the JSON-RPC response type. Specifically the JSON-RPC 2.0 specification - I won't bother with the 1.0 specification as it's now over 15+ years old and many APIs will use only the 2.0 format.

The JSON RPC specification says that the response must include the following:

  1. A jsonrpc field. With a value of "2.0".
  2. A result field, that contains the data dependent on the JSON RPC method.
  3. An error field, that contains the method error if such occurred.
  4. An id field which must have the same value as that in the request.

An important thing to note here is that the result and error fields are mutually exclusive - meaning only one of them can be present in the response at a given time.

Furthermore, the error field has the following format:

  1. A 1 field, an integer which indicates the error type.
  2. A 1 field, a string with a short description of the error.
  3. An optional 1 field, the format of which is arbitrary and dependent on the RPC server implementation.
  4. An example of such a response would therefore look like this
{
  "jsonrpc": "2.0",
  "result": { "server-defined-field": "Hello, World!" },
  "id": 1
}

or

{
  "jsonrpc": "2.0",
  "error": {
    "code": 123,
    "message": "Something failed"
  },
  "id": 1
}

To start off let's consider the most basic solution.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response {
    pub jsonrpc: String,
    pub result: Option<serde_json::Value>,
    pub error: Option<ResponseError>,
    pub id: u32,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResponseError {
    pub code: i32,
    pub message: String,
    pub data: Option<serde_json::Value>,
}

Something worth noting here, is that just because we use serde_json::Value doesn't lock us in with just JSON.

It's fine to serialize/deserialize serde_json::Value by any serde backend.

Looks good, right? And it's "correct" in the sense, that it'll successfully capture all the responses we get from the RPC. In fact, it'll be able to capture many more JSON documents than the ones compliant with the JSON-RPC specification. Many of the changes that we'll include in this article will actually constrain this type so that we'll reject incorrect values (like a value different than "2.0" for the jsonrpc field).

We can use the result and error fields as they are, but you'll note that for the result we'd have to run it through serde_json::from_value to parse the Value into our defined data format.

A simple improvement, therefore, is to swap out the serde_json::Value for a generic parameter.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: String,
    pub result: Option<T>,
    pub error: Option<ResponseError>,
    pub id: u32,
}

That way we can, for example, parse a Response with the result the field immediately ready to use. And we can still set the generic parameter to be serde_json::Value (or any other such type) if we'd like to do some response pre-processing.

But this solution is still far from perfect. According to the spec, the value of the jsonrpc field must always be "2.0" and not some arbitrary string.

Therefore it makes little sense to:

  1. Allocate an entire String to never use it.
  2. Allow any other values than "2.0" at parse time.

A quick way to constrain this field is to introduce a single value enum.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum JsonRpcVersion {
    V2_0
}

Unfortunately Rust won't allow us to name our field "2.0" (even with raw identifiers). But we can use the serde(rename = "...") attribute instead.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum JsonRpcVersion {
    #[serde(rename = "2.0")]
    V2_0
}

All that's left is to plug this type into our response struct

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    pub result: Option<T>,
    pub error: Option<ResponseError>,
    pub id: u32,
}

Moving on, you'll notice that we're breaking one invariant specified by the specification. Namely result and error are mutually exclusive.

Apart from being wrong from the specification's point of view it's also detrimental to your code - Rust gives us a powerful type system to constrain the values we're working with at runtime. If we keep the two Option fields here, we'll have to .uwrap or .expect most of the time we use these fields.

There has to be a better way!

And there is, this exclusivity should fit nicely into Rust's Result type - maybe it'd even be possible to just plug it in?

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    pub result: Result<T, ResponseError>,
    pub id: u32,
}

Unfortunately no, this type maps to a response like

{
  "jsonrpc": "2.0",
  "result": {
    "Ok": "Arbitrary data"
  },
  "id": 1
}

The first thing to do here is to flatten the result field.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    #[serde(flatten)]
    pub result: Result<T, ResponseError>,
    pub id: u32,
}

Now we can parse something like this

{
  "jsonrpc": "2.0",
  "Ok": "Arbitrary data",
  "id": 1
}

Which is almost what we want!

Serde allows us to rename variants of an enum. But Result is defined in the standard library out of our reach.

As far as I know, there are 3 ways of solving this:

  1. Use our own custom Result-like type
  2. Using the serde(remote) attribute we can define a remote type that'll handle the parsing
  3. Manually implementing Serialize and Deserialize

I'm the most partial to the first option, as from my experience it's the fastest. But I'll also entertain the second option for demonstration purposes - and it also appears to be the method recommended by serde's documentation.

I'd consider the third option to be the most complex in terms of written code. And I don't really see the benefit to it for this particular case. I'll make sure to cover it in a different article.

Option 1: Custom type

This is in my opinion the easiest, simplest and fastest solution.

With this solution instead of using the Result type we'll create our own, let's call it ResponseOrError for clarity.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum ResponseOrError<T> {
    Result(T),
    Error(ResponseError),
}

However in this form it'll expect a Result or an Error field. We could use the serde(rename = "...") attribute as before, to rename each variant. Or we could use the serde(rename_all = "...") attribute instead.

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ResponseOrError<T> {
    Result(T),
    Error(ResponseError),
}

Note the argument of the rename_all attribute. It describes the case of each field or variant in the type.

Something that I always found nice is that the value of this attribute is itself in that case. Some other values that are helpful to know are:

  1. kebab-case
  2. PascalCase
  3. snake_case
  4. SCREAMING_SNAKE_CASE

Putting it all together we have the following code

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    #[serde(flatten)]
    pub result: ResponseOrError<T>,
    pub id: u32,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum JsonRpcVersion {
    #[serde(rename = "2.0")]
    V2_0
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResponseError {
    pub code: i32,
    pub message: String,
    pub data: Option<serde_json::Value>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ResponseOrError<T> {
    Result(T),
    Error(ResponseError),
}

Option 2: Remote types

The remote attribute is serde's answer to the limitations imposed by the orphan rule.

The solution will actually be very similar to the previous option, with the only difference being that we'll actually have the result field be of Result type.

As before we have to define a new type that we'll derive the Serialize and Deserialize implementations for. However this time, this type must exactly match the std::result::Result type.

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(remote = "Result")]
pub enum ResultDef<T, E> {
    #[serde(rename = "result")]
    Ok(T),
    #[serde(rename = "error")]
    Err(E),
}

And now instead of using the same type in our struct, we'll use the Result type annotated with serde's with attribute.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    #[serde(flatten)]
    #[serde(with = "ResultDef")]
    pub result: Result<T, ResponseError>,
    pub id: u32,
}

If you now try to use this code you'll notice that it fails to compile and complains that the type T does not implement Serialize and Deserialize.

Normally, like in the first solution, serde would insert the appropriate trait bounds that would make serialization possible. But in this case we need to help it a little. We can annotate the Response type with the serde(bound) attribute. Like so:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(bound = "T: Serialize + DeserializeOwned")]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    #[serde(flatten)]
    #[serde(with = "ResultDef")]
    pub result: Result<T, ResponseError>,
    pub id: u32,
}

Putting it all together we have the following code:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(bound = "T: Serialize + DeserializeOwned")]
pub struct Response<T> {
    pub jsonrpc: JsonRpcVersion,
    #[serde(flatten)]
    #[serde(with = "ResultDef")]
    pub result: Result<T, ResponseError>,
    pub id: u32,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum JsonRpcVersion {
    #[serde(rename = "2.0")]
    V2_0
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResponseError {
    pub code: i32,
    pub message: String,
    pub data: Option<serde_json::Value>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(remote = "Result")]
pub enum ResultDef<T, E> {
    #[serde(rename = "result")]
    Ok(T),
    #[serde(rename = "error")]
    Err(E),
}