serialize_as/deserialize_as does not seem to work with Option<>

[ruseinov]

Backend

PostgreSQL

Diesel version

2.3.0

Diesel Features

"postgres",     "chrono",     "serde_json",     "uuid",

Operating System Version

Mac

Third party libraries

irrelevant

What do you want to do?

what works:

    #[diesel(serialize_as = DbAddress, deserialize_as = DbAddress)]
    pub wallet_address: Address,

what I'd like to achieve

    #[diesel(serialize_as = Option<DbAddress>, deserialize_as = Option<DbAddress>)]
    pub wallet_address: Option<Address>,

or even

    #[diesel(serialize_as = DbAddress, deserialize_as = DbAddress)]
    pub wallet_address: Option<Address>,

Compile time error

57 |     AsChangeset,
   |     ^^^^^^^^^^^ the trait `From<Address>` is not implemented for `std::option::Option<db_address::DbAddress>`

What code do you already have?

use alloy::primitives::{Address, private::derive_more};
use derive_more::{Deref, Display, From, Into};
use diesel::{
    AsExpression, FromSqlRow,
    backend::Backend,
    deserialize::FromSql,
    pg::Pg,
    serialize::{IsNull, Output, ToSql},
    sql_types::Bytea,
};
use serde::{Deserialize, Serialize};
use std::{fmt::Debug, io::Write};

/// Address wrapper for diesel.
#[derive(
    FromSqlRow,
    AsExpression,
    Serialize,
    Deserialize,
    Debug,
    Default,
    Display,
    Deref,
    From,
    Into,
    Copy,
    Eq,
    PartialEq,
    Clone,
    Hash,
)]
#[diesel(sql_type = Bytea)]
pub struct DbAddress(pub Address);

impl ToSql<Bytea, Pg> for DbAddress {
    fn to_sql<'b>(
        &'b self,
        out: &mut Output<'b, '_, Pg>,
    ) -> diesel::serialize::Result {
        // Write the Address as a byte slice to the SQL output
        out.write_all(self.0.as_slice())?;
        Ok(IsNull::No)
    }
}

impl FromSql<Bytea, Pg> for DbAddress {
    fn from_sql(
        bytes: <Pg as Backend>::RawValue<'_>,
    ) -> diesel::deserialize::Result<Self> {
        // Extract the raw bytes from the SQL value
        let byte_data: &[u8] = bytes.as_bytes();

        // Validate that the data consists of exactly 20 bytes
        if byte_data.len() != 20 {
            return Err(Box::from(
                "Failed to deserialize DbAddress: Expected 20 bytes but got a different length",
            ));
        }

        // Convert the byte slice into an Address
        Ok(Self(Address::from_slice(byte_data)))
    }
}

Additional details

The code that's already there is irrelevant, it seems that serialize_as/deserialize_as is broken for any kind of Option pair.

Checklist

• I have already looked over the issue tracker and the discussion forum for similar possible closed issues.

[weiznich]

Your assessment that this attribute is broken for any option type is wrong. You just need to have a existing TryFrom impl between both types. That can be done by for example using a custom non-option new type on one side.

What does not woraus converting from an option to an option type. That's a language limitation and nothing diesel can fix. If you have opinions about that please contact the rust project instead.

[ruseinov]

It was not my assumption that the attribute is broken, but rather that it does not support Option to Option conversion. The From implementations are present and they work as intended for non-optional fields.

Thanks for the clarification, I will then add a new type to wrap Options.

[ruseinov]

Once I have a somewhat elegant solution I’ll make sure to post it here.

This kind of thing is useful for anybody looking for an ergonomic solution to wrapper types that don’t get leaked to the rest of the codebase through model properties

[weiznich]

Essentially a duplicate of #3835

[ruseinov]

Yep, I guess I didn't use the right keywords. Though this solution is going to be a pretty ugly one in our case, because we're talking about stuff like

Address -> DbAddress(Address) and then if we would need to wrap it into another type, it would be something like OptionalAddress(Option<Address>) -> DbAddress(Address). Which eliminates the point of doing this in the first place, because then one could just use address: Option<DbAddress> in the model and it will just work.

The general idea behind the approach taken is to hide all of this extra complexity (newtypes) from the user, but with optionals it seems that we need to add one newtype to hide the other.

I'm wondering why macro attributes could not be parsed for Option and dealt with accordingly, for instance using Option.map.
This would make the serde way more powerful and ergonomic.

I'll give a full example to express better what the issue is:

what we have now

pub struct Blah {
  /// works perfectly for serde, needs no extra wrappers, but is a newtype DbAddress(Address), so it's usages in the consuming api would have to be `addr.0` or `addr.into()`, when `From` is implemented.
  pub address: Option<DbAddress>
}

what we'd like to achieve

pub struct Blah {
  /// Hides the newtype making Address usage more ergonomic, does not leak the db-specific newtype
  #[diesel(serialize_as = Option<DbAddress>, deserialize_as = Option<DbAddress>)]
  pub address: Option<Address>
}

what the solution would look like as far as I understand it:

pub struct Blah {
  /// This is OptionalAddress(Option<Address>). Which makes it even less ergonomic than the original design
  #[diesel(serialize_as = Option<DbAddress>, deserialize_as = Option<DbAddress>)]
  pub address: OptionalAddress 
}

I must admit that I haven't had the time to dig into how diesel deals with ser/de in detail yet, so all the thoughts I express here are purely UX-based.

[ruseinov]

#3835 (reply in thread) This looks like it could be a solution, I'm going to try something like that.