//! # Deserialization Flavors //! //! "Flavors" in `postcard` are used as modifiers to the serialization or deserialization //! process. Flavors typically modify one or both of the following: //! //! 1. The source medium of the deserialization, e.g. whether the data is serialized from a `[u8]` slice, or some other container //! 2. The format of the deserialization, such as if the original data is encoded in a COBS format, contains a CRC32 checksum //! appended to the message, etc. //! //! Flavors are implemented using the [`Flavor`] trait, which acts as a "middleware" for retrieving the bytes before they //! are passed to `serde` for deserialization //! //! Multiple flavors may be combined to obtain a desired combination of behavior and storage. //! When flavors are combined, it is expected that the storage flavor (such as [`Slice`]) is the innermost flavor. //! //! Custom flavors may be defined by users of the `postcard` crate, however some commonly useful flavors have been provided in //! this module. If you think your custom flavor would be useful to others, PRs adding flavors are very welcome! //! //! ## Usability //! //! Flavors may not always be convenient to use directly, as they may expose some implementation details of how the //! inner workings of the flavor behaves. It is typical to provide a convenience method for using a flavor, to prevent //! the user from having to specify generic parameters, setting correct initialization values, or handling the output of //! the flavor correctly. See `postcard::from_bytes()` for an example of this. //! //! ## When to use (multiple) flavors //! //! Combining flavors are nice for convenience, as they perform potentially multiple steps of //! serialization at one time. //! //! This can often be more memory efficient, as intermediate buffers are not typically required. //! //! ## When NOT to use (multiple) flavors //! //! The downside of passing deserialization through multiple steps is that it is typically slower than //! performing each step serially. Said simply, "cobs decoding while deserializing" is often slower //! than "cobs decode then deserialize", due to the ability to handle longer "runs" of data in each //! stage. The downside is that if these stages can not be performed in-place on the buffer, you //! will need additional buffers for each stage. //! //! Additionally, deserializating flavors can be more restrictive or difficult to work with than //! serialization flavors, as deserialization may require that the deserialized types borrow some //! portion of the original message. //! //! ## Examples //! //! ### Using a single flavor //! //! In the first example, we use the `Slice` flavor, to retrieve the serialized output from a `[u8]` slice. //! No other modification is made to the serialization process. //! //! ```rust //! use postcard::{ //! de_flavors::Slice, //! Deserializer, //! }; //! use serde::Deserialize; //! //! #[derive(Deserialize, Debug, PartialEq)] //! struct Tup(u8, u8, u8); //! //! let msg = [0x04, 0x00, 0x04, 0x01, 0x02, 0x03]; //! let slice = Slice::new(&msg); //! let mut deserializer = Deserializer::from_flavor(slice); //! let t = Tup::deserialize(&mut deserializer).unwrap(); //! assert_eq!(t, Tup(4, 0, 4)); //! let remainder = deserializer.finalize().unwrap(); //! assert_eq!(remainder, &[1, 2, 3]); //! ``` use crate::{Error, Result}; use core::marker::PhantomData; /// The deserialization Flavor trait /// /// This is used as the primary way to decode serialized data from some kind of buffer, /// or modify that data in a middleware style pattern. /// /// See the module level docs for an example of how flavors are used. pub trait Flavor<'de>: 'de { /// The remaining data of this flavor after deserializing has completed. /// /// Typically, this includes the remaining buffer that was not used for /// deserialization, and in cases of more complex flavors, any additional /// information that was decoded or otherwise calculated during /// the deserialization process. type Remainder: 'de; /// The source of data retrieved for deserialization. /// /// This is typically some sort of data buffer, or another Flavor, when /// chained behavior is desired type Source: 'de; /// Obtain the next byte for deserialization fn pop(&mut self) -> Result; /// Returns the number of bytes remaining in the message, if known. /// /// # Implementation notes /// /// It is not enforced that this number is exactly correct. /// A flavor may yield less or more bytes than the what is hinted at by /// this function. /// /// `size_hint()` is primarily intended to be used for optimizations such as /// reserving space for deserialized items, but must not be trusted to /// e.g., omit bounds checks in unsafe code. An incorrect implementation of /// `size_hint()` should not lead to memory safety violations. /// /// That said, the implementation should provide a correct estimation, /// because otherwise it would be a violation of the trait’s protocol. /// /// The default implementation returns `None` which is correct for any flavor. fn size_hint(&self) -> Option { None } /// Attempt to take the next `ct` bytes from the serialized message fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]>; /// Complete the deserialization process. /// /// This is typically called separately, after the `serde` deserialization /// has completed. fn finalize(self) -> Result; } /// A simple [`Flavor`] representing the deserialization from a borrowed slice pub struct Slice<'de> { // This string starts with the input data and characters are truncated off // the beginning as data is parsed. pub(crate) cursor: *const u8, pub(crate) end: *const u8, pub(crate) _pl: PhantomData<&'de [u8]>, } impl<'de> Slice<'de> { /// Create a new [Slice] from the given buffer pub fn new(sli: &'de [u8]) -> Self { Self { cursor: sli.as_ptr(), end: unsafe { sli.as_ptr().add(sli.len()) }, _pl: PhantomData, } } } impl<'de> Flavor<'de> for Slice<'de> { type Remainder = &'de [u8]; type Source = &'de [u8]; #[inline] fn pop(&mut self) -> Result { if self.cursor == self.end { Err(Error::DeserializeUnexpectedEnd) } else { unsafe { let res = Ok(*self.cursor); self.cursor = self.cursor.add(1); res } } } #[inline] fn size_hint(&self) -> Option { Some((self.end as usize) - (self.cursor as usize)) } #[inline] fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> { let remain = (self.end as usize) - (self.cursor as usize); if remain < ct { Err(Error::DeserializeUnexpectedEnd) } else { unsafe { let sli = core::slice::from_raw_parts(self.cursor, ct); self.cursor = self.cursor.add(ct); Ok(sli) } } } /// Return the remaining (unused) bytes in the Deserializer fn finalize(self) -> Result<&'de [u8]> { let remain = (self.end as usize) - (self.cursor as usize); unsafe { Ok(core::slice::from_raw_parts(self.cursor, remain)) } } } /// Support for [`std::io`] or `embedded-io` traits #[cfg(any( feature = "embedded-io-04", feature = "embedded-io-06", feature = "use-std" ))] pub mod io { use crate::{Error, Result}; use core::marker::PhantomData; struct SlidingBuffer<'de> { cursor: *mut u8, end: *const u8, _pl: PhantomData<&'de [u8]>, } impl<'de> SlidingBuffer<'de> { pub fn new(sli: &'de mut [u8]) -> Self { Self { cursor: sli.as_mut_ptr(), end: unsafe { sli.as_ptr().add(sli.len()) }, _pl: PhantomData, } } #[inline] fn size(&self) -> usize { (self.end as usize) - (self.cursor as usize) } #[inline] fn take_n(&mut self, ct: usize) -> Result<&'de mut [u8]> { let remain = (self.end as usize) - (self.cursor as usize); let buff = if remain < ct { return Err(Error::DeserializeUnexpectedEnd); } else { unsafe { let sli = core::slice::from_raw_parts_mut(self.cursor, ct); self.cursor = self.cursor.add(ct); sli } }; Ok(buff) } fn complete(self) -> Result<&'de mut [u8]> { let remain = (self.end as usize) - (self.cursor as usize); unsafe { Ok(core::slice::from_raw_parts_mut(self.cursor, remain)) } } } /// Support for [`embedded_io`](crate::eio::embedded_io) traits #[cfg(any(feature = "embedded-io-04", feature = "embedded-io-06"))] pub mod eio { use super::super::Flavor; use super::SlidingBuffer; use crate::{Error, Result}; /// Wrapper over a [`embedded_io`](crate::eio::embedded_io)::[`Read`](crate::eio::Read) and a sliding buffer to implement the [`Flavor`] trait pub struct EIOReader<'de, T> where T: crate::eio::Read, { reader: T, buff: SlidingBuffer<'de>, } impl<'de, T> EIOReader<'de, T> where T: crate::eio::Read, { /// Create a new [`EIOReader`] from a reader and a buffer. /// /// `buff` must have enough space to hold all data read during the deserialisation. pub fn new(reader: T, buff: &'de mut [u8]) -> Self { Self { reader, buff: SlidingBuffer::new(buff), } } } impl<'de, T> Flavor<'de> for EIOReader<'de, T> where T: crate::eio::Read + 'de, { type Remainder = (T, &'de mut [u8]); type Source = &'de [u8]; #[inline] fn pop(&mut self) -> Result { let mut val = [0; 1]; self.reader .read(&mut val) .map_err(|_| Error::DeserializeUnexpectedEnd)?; Ok(val[0]) } #[inline] fn size_hint(&self) -> Option { Some(self.buff.size()) } #[inline] fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> { let buff = self.buff.take_n(ct)?; self.reader .read_exact(buff) .map_err(|_| Error::DeserializeUnexpectedEnd)?; Ok(buff) } /// Return the remaining (unused) bytes in the Deserializer fn finalize(self) -> Result<(T, &'de mut [u8])> { let buf = self.buff.complete()?; Ok((self.reader, buf)) } } } /// Support for [`std::io`] traits #[allow(clippy::module_inception)] #[cfg(feature = "use-std")] pub mod io { use super::super::Flavor; use super::SlidingBuffer; use crate::{Error, Result}; /// Wrapper over a [`std::io::Read`] and a sliding buffer to implement the [Flavor] trait pub struct IOReader<'de, T> where T: std::io::Read, { reader: T, buff: SlidingBuffer<'de>, } impl<'de, T> IOReader<'de, T> where T: std::io::Read, { /// Create a new [`IOReader`] from a reader and a buffer. /// /// `buff` must have enough space to hold all data read during the deserialisation. pub fn new(reader: T, buff: &'de mut [u8]) -> Self { Self { reader, buff: SlidingBuffer::new(buff), } } } impl<'de, T> Flavor<'de> for IOReader<'de, T> where T: std::io::Read + 'de, { type Remainder = (T, &'de mut [u8]); type Source = &'de [u8]; #[inline] fn pop(&mut self) -> Result { let mut val = [0; 1]; self.reader .read(&mut val) .map_err(|_| Error::DeserializeUnexpectedEnd)?; Ok(val[0]) } #[inline] fn size_hint(&self) -> Option { Some(self.buff.size()) } #[inline] fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> { let buff = self.buff.take_n(ct)?; self.reader .read_exact(buff) .map_err(|_| Error::DeserializeUnexpectedEnd)?; Ok(buff) } /// Return the remaining (unused) bytes in the Deserializer fn finalize(self) -> Result<(T, &'de mut [u8])> { let buf = self.buff.complete()?; Ok((self.reader, buf)) } } } } //////////////////////////////////////// // CRC //////////////////////////////////////// /// This Cyclic Redundancy Check flavor applies [the CRC crate's `Algorithm`](https://docs.rs/crc/latest/crc/struct.Algorithm.html) struct on /// the serialized data. /// /// The flavor will check the CRC assuming that it has been appended to the bytes. /// CRCs are used for error detection when reading data back. /// Requires the `crc` feature. /// /// More on CRCs: . #[cfg(feature = "use-crc")] #[cfg_attr(docsrs, doc(cfg(feature = "use-crc")))] pub mod crc { use core::convert::TryInto; use crc::Digest; use crc::Width; use serde::Deserialize; use super::Flavor; use super::Slice; use crate::Deserializer; use crate::Error; use crate::Result; use paste::paste; /// Manages CRC modifications as a flavor. pub struct CrcModifier<'de, B, W> where B: Flavor<'de>, W: Width, { flav: B, digest: Digest<'de, W>, } impl<'de, B, W> CrcModifier<'de, B, W> where B: Flavor<'de>, W: Width, { /// Create a new Crc modifier Flavor. pub fn new(bee: B, digest: Digest<'de, W>) -> Self { Self { flav: bee, digest } } } macro_rules! impl_flavor { ($( $int:ty ),*) => { $( paste! { impl<'de, B> Flavor<'de> for CrcModifier<'de, B, $int> where B: Flavor<'de>, { type Remainder = B::Remainder; type Source = B::Source; #[inline] fn pop(&mut self) -> Result { match self.flav.pop() { Ok(byte) => { self.digest.update(&[byte]); Ok(byte) } e @ Err(_) => e, } } #[inline] fn size_hint(&self) -> Option { self.flav.size_hint() } #[inline] fn try_take_n(&mut self, ct: usize) -> Result<&'de [u8]> { match self.flav.try_take_n(ct) { Ok(bytes) => { self.digest.update(bytes); Ok(bytes) } e @ Err(_) => e, } } fn finalize(mut self) -> Result { match self.flav.try_take_n(core::mem::size_of::<$int>()) { Ok(prev_crc_bytes) => match self.flav.finalize() { Ok(remainder) => { let crc = self.digest.finalize(); let le_bytes = prev_crc_bytes .try_into() .map_err(|_| Error::DeserializeBadEncoding)?; let prev_crc = <$int>::from_le_bytes(le_bytes); if crc == prev_crc { Ok(remainder) } else { Err(Error::DeserializeBadCrc) } } e @ Err(_) => e, }, Err(e) => Err(e), } } } /// Deserialize a message of type `T` from a byte slice with a Crc. The unused portion (if any) /// of the byte slice is not returned. pub fn []<'a, T>(s: &'a [u8], digest: Digest<'a, $int>) -> Result where T: Deserialize<'a>, { let flav = CrcModifier::new(Slice::new(s), digest); let mut deserializer = Deserializer::from_flavor(flav); let r = T::deserialize(&mut deserializer)?; let _ = deserializer.finalize()?; Ok(r) } /// Deserialize a message of type `T` from a byte slice with a Crc. The unused portion (if any) /// of the byte slice is returned for further usage pub fn []<'a, T>(s: &'a [u8], digest: Digest<'a, $int>) -> Result<(T, &'a [u8])> where T: Deserialize<'a>, { let flav = CrcModifier::new(Slice::new(s), digest); let mut deserializer = Deserializer::from_flavor(flav); let t = T::deserialize(&mut deserializer)?; Ok((t, deserializer.finalize()?)) } } )* }; } impl_flavor![u8, u16, u32, u64, u128]; }