Rust 中的序列化和反序列化

  |   0 评论   |   0 浏览

序列化: 将数据结构或对象转换成二进制序列的过程

反序列化:将在序列化过程中所生成的二进制序列转换成数据结构或者对象的过程

Serde 是对 Rust 数据结构进行序列化和反序列化一种常用框架。

Serde 中的 trait

Serde 定义了 4 种 trait:

  • Deserialize A data structure that can be deserialized from any data format supported by Serde.
  • Deserializer A data format that can deserialize any data structure supported by Serde.
  • Serialize A data structure that can be serialized into any data format supported by Serde.
  • Serializer A data format that can serialize any data structure supported by Serde.

SerializeDeserialize

SerializeDeserialize 的定义:

 1pub trait Serialize {
 2    pub fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
 3    where
 4        S: Serializer;
 5}
 6pub trait Deserialize<'de>: Sized {
 7    pub fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
 8    where
 9        D: Deserializer<'de>;
10}

若要数据类型支持序列化和反序列化,则该类型需要实现 SerializeDeserialize trait。Serde 提供了 Rust 基础类型和标准库类型的 SerializeDeserialize 实现。

对于自定义类型,我们可以自行实现 SerializeDeserialize trait。另外,Serde 提供一个宏 serde_derive 来自动为结构体类型和枚举类型生成 SerializeDeserialize 实现。该特性需要 Rust 编译器版本在 1.31及以上,并且在 Cargo.toml 文件中配置 Serde 依赖时,需要使用 features 指定该特性。例如:

1[dependencies]
2serde = { version = "1.0", features = ["derive"] }

然后就可以在代码中引入并使用了,例如:

1use serde::{Serialize, Deserialize};
2
3#[derive(Serialize, Deserialize, Debug)]
4struct Point {
5    x: i32,
6    y: i32,
7}

SerializerDeserializer

SerializerDeserializer 的实现由第三方 crate 提供,例如:serde_json, serde_yamlbincode

社区实现的部分数据格式:

  • JSON, the ubiquitous JavaScript Object Notation used - by many HTTP APIs.
  • Bincode, a compact binary format used for IPC within - the Servo rendering engine.
  • CBOR, a Concise Binary Object Representation designed - for small message size without the need for version - negotiation.
  • YAML, a self-proclaimed human-friendly configuration - language that ain't markup language.
  • MessagePack, an efficient binary format that resembles - a compact JSON.
  • TOML, a minimal configuration format used by Cargo.
  • Pickle, a format common in the Python world.
  • RON, a Rusty Object Notation.
  • BSON, the data storage and network transfer format - used by MongoDB.
  • Avro, a binary format used within Apache Hadoop, with - support for schema definition.
  • JSON5, A superset of JSON including some productions - from ES5.
  • Postcard, a no_std and embedded-systems friendly - compact binary format.
  • URL query strings, in the x-www-form-urlencoded format.
  • Envy, a way to deserialize environment variables into - Rust structs. (deserialization only)
  • Envy Store, a way to deserialize AWS Parameter Store - parameters into Rust structs. (deserialization only)
  • S-expressions, the textual representation of code and - data used by the Lisp language family.
  • D-Bus's binary wire format.
  • FlexBuffers, the schemaless cousin of Google's - FlatBuffers zero-copy serialization format.
  • DynamoDB Items, the format used by rusoto_dynamodb to - transfer data to and from DynamoDB.

Serde JSON

Cargo.toml 文件中添加依赖:

1[dependencies]
2serde_json = "1.0"

序列化

serde_json Crate 提供了 serde_json::to_stringserde_json::to_vecserde_json::to_writer 三个函数将 Rust 数据类型转为 JSON StringVec<u8>io::Write(比如文件或者 TCP 流)。

示例

 1use serde::{Deserialize, Serialize};
 2use serde_json::Result;
 3
 4#[derive(Serialize, Deserialize)]
 5struct Address {
 6    street: String,
 7    city: String,
 8}
 9
10fn print_an_address() -> Result<()> {
11    // Some data structure.
12    let address = Address {
13        street: "10 Downing Street".to_owned(),
14        city: "London".to_owned(),
15    };
16
17    // Serialize it to a JSON string.
18    let j = serde_json::to_string(&address)?;
19
20    // Print, write to a file, or send to an HTTP server.
21    println!("{}", j);
22
23    Ok(())
24}

反序列化

serde_json Crate 提供了 serde_json::from_strserde_json::from_sliceserde_json::from_reader 三个函数将字符串、字节切片(&[u8])和 IO 输入(文件或者 TCP 流)解析为对应的 Rust 数据类型。

示例

 1use serde::{Deserialize, Serialize};
 2use serde_json::Result;
 3
 4#[derive(Serialize, Deserialize)]
 5struct Person {
 6    name: String,
 7    age: u8,
 8    phones: Vec<String>,
 9}
10
11fn typed_example() -> Result<()> {
12    // Some JSON input data as a &str. Maybe this comes from the user.
13    let data = r#"
14        {
15            "name": "John Doe",
16            "age": 43,
17            "phones": [
18                "+44 1234567",
19                "+44 2345678"
20            ]
21        }"#;
22
23    // Parse the string of data into a Person object. This is exactly the
24    // same function as the one that produced serde_json::Value above, but
25    // now we are asking it for a Person as output.
26    let p: Person = serde_json::from_str(data)?;
27
28    // Do things just like with any other Rust data structure.
29    println!("Please call {} at the number {}", p.name, p.phones[0]);
30
31    Ok(())
32}

如果一个 JSON 格式数据没有对应的 Rust 数据类型,那么可以将其解析为 serde_json::Value 类型。这是 serde_json Crate 提供的一个递归的枚举类型,它可以表示任何有效的 JSON 数据,其定义如下:

1enum Value {
2    Null,
3    Bool(bool),
4    Number(Number),
5    String(String),
6    Array(Vec<Value>),
7    Object(Map<String, Value>),
8}

将 JSON 格式数据解析为 serde_json::Value 类型,同样使用的是:serde_json::from_strserde_json::from_sliceserde_json::from_reader 三个函数。这三个函数返回值是泛型的,其返回值类型由指定给变量的类型决定。

示例

 1use serde_json::{Result, Value};
 2
 3fn untyped_example() -> Result<()> {
 4    // Some JSON input data as a &str. Maybe this comes from the user.
 5    let data = r#"
 6        {
 7            "name": "John Doe",
 8            "age": 43,
 9            "phones": [
10                "+44 1234567",
11                "+44 2345678"
12            ]
13        }"#;
14
15    // Parse the string of data into serde_json::Value.
16    let v: Value = serde_json::from_str(data)?;
17    // let v = serde_json::from_str::<Value>(data)?; // 或者可以这样写
18
19    // Access parts of the data by indexing with square brackets.
20    println!("Please call {} at the number {}", v["name"], v["phones"][0]);
21
22    Ok(())
23}

RON

Cargo.toml 文件中添加依赖:

1[dependencies]
2ron = "0.6.4"

RON(Rusty Object Notation) 是一种与 Rust 语法类似的一种序列化格式,它支持所有的 Serde 数据模型

RON 示例

 1Scene( // class name is optional
 2    materials: { // this is a map
 3        "metal": (
 4            reflectivity: 1.0,
 5        ),
 6        "plastic": (
 7            reflectivity: 0.5,
 8        ),
 9    },
10    entities: [ // this is an array
11        (
12            name: "hero",
13            material: "metal",
14        ),
15        (
16            name: "monster",
17            material: "plastic",
18        ),
19    ],
20)

RON 格式特点

  • 使用 (..) 表示结构体,使用 {..} 表示 Map,使用 [..] 表示数组。在使用 JSON 时是不能区分结构体和 Map 的。
  • 可以添加注释,在 JSON 中不能添加注释。
  • 与 Rust 语法一样,最后一个分项也加逗号。

ron::value::Value

与 serde_json Crate 类似,ron Crate 也使用递归枚举类型来表示 RON 格式数据。这个枚举类型是:ron::value::Value,它的定义如下:

 1pub enum Value {
 2    Bool(bool),
 3    Char(char),
 4    Map(Map),
 5    Number(Number),
 6    Option(Option<Box<Value>>),
 7    String(String),
 8    Seq(Vec<Value>),
 9    Unit,
10}

序列化和反序列化

ron Crate 提供的序列化函数:

  • ron::ser::to_string: Serializes value and returns it as string.
  • ron::ser::to_string_pretty: Serializes value in the recommended RON layout in a pretty way.
  • ron::ser::to_writer: Serializes value into writer.
  • ron::ser::to_writer_pretty: Serializes value into writer in a pretty way.

ron Crate 提供的反序列化函数:

  • ron::de::from_bytes: Building a deserializer and deserializing a value of type T from bytes.
  • ron::de::from_str: Reading data from a reader and feeding into a deserializer.
  • ron::de::from_reader: Building a deserializer and deserializing a value of type T from a string.

示例

 1use serde::{Deserialize, Serialize};
 2use std::fs::File;
 3
 4#[derive(Serialize, Deserialize, Debug)]
 5struct Point {
 6    x: i32,
 7    y: i32,
 8}
 9
10fn main() {
11    let point1 = Point { x: 1, y: 2 };
12    // Serializes point1 into a buffer
13    let mut buf = Vec::<u8>::new();
14    ron::ser::to_writer(&mut buf, &point1).unwrap();
15    println!("buf = {:?}", buf);
16
17    let point2: Point = ron::de::from_bytes(&buf).unwrap();
18    println!("point2: {:?}", point2);
19
20    let point3 = Point { x: 3, y: 4 };
21    // Serializes point3 and returns it as string
22    let point_str = ron::to_string(&point3).unwrap();
23    println!("point_str: {:?}", point_str);
24
25    let point4: Point = ron::from_str(&point_str).unwrap();
26    println!("point4: {:?}", point4);
27
28    let point5 = Point { x: 5, y: 6 };
29    let file = File::create("foo.txt").unwrap();
30    // Serializes point5 into foo.txt
31    ron::ser::to_writer(file, &point5).unwrap();
32
33    let file = File::open("foo.txt").unwrap();
34    let point6: Point = ron::de::from_reader(file).unwrap();
35    println!("point6: {:?}", point6);
36}

BSON

Cargo.toml 文件中添加依赖:

1[dependencies]
2bson = "1.1.0"

BSON(Binary JSON) 是一种二进制形式的类似 JSON 的格式(文档)。

BSON 示例

1// JSON equivalent
2{"hello": "world"}
3
4// BSON encoding
5\x16\x00\x00\x00                   // total document size
6\x02                               // 0x02 = type String
7hello\x00                          // field name
8\x06\x00\x00\x00world\x00          // field value
9\x00                               // 0x00 = type EOO ('end of object')

BSON 值

很多不同的数据类型可以表示为 BSON 值,Bson 枚举类型定义了可表示为 BSON 值的所有类型。

示例:创建 Bson 实例

 1// 直接创建 Bson 实例
 2let string = Bson::String("hello world".to_string());
 3let int = Bson::Int32(5);
 4let array = Bson::Array(vec![Bson::Int32(5), Bson::Boolean(false)]);
 5
 6// 使用 into() 创建 Bson 实例
 7let string: Bson = "hello world".into();
 8let int: Bson = 5i32.into();
 9
10// 使用 `bson!` 宏创建 Bson 实例
11let string = bson!("hello world");
12let int = bson!(5);
13let array = bson!([5, false]);

BSON documents

Bson 中的文档由一组有序的键值对组成,类似于 JSON 中的 Object。另外,还包含 Bson 值所占的空间大小。

示例一:创建文档实例

 1let mut bytes = hex::decode("0C0000001069000100000000").unwrap();
 2// 直接创建
 3let doc = Document::from_reader(&mut bytes.as_slice()).unwrap(); // { "i": 1 }
 4
 5// 通过 doc! 创建
 6let doc = doc! {
 7   "hello": "world",
 8   "int": 5,
 9   "subdoc": { "cat": true },
10};

示例二:访问文档成员

 1let doc = doc! {
 2   "string": "string",
 3   "bool": true,
 4   "i32": 5,
 5   "doc": { "x": true },
 6};
 7
 8// attempt get values as untyped Bson
 9let none = doc.get("asdfadsf"); // None
10let value = doc.get("string"); // Some(&Bson::String("string"))
11
12// attempt to get values with explicit typing
13let string = doc.get_str("string"); // Ok("string")
14let subdoc = doc.get_document("doc"); // Some(Document({ "x": true }))
15let error = doc.get_i64("i32"); // Err(...)

序列化和反序列化

bson Crate 提供的序列化函数:

  • bson::to_bson: Encode a T Serializable into a BSON Value.
  • bson::to_document: Encode a T Serializable into a BSON Document.

bson Crate 提供的反序列化函数:

  • bson::from_bson: Decode a BSON Value into a T Deserializable.
  • bson::from_document: Decode a BSON Document into a T Deserializable.

示例

 1use bson::*;
 2use serde::{Deserialize, Serialize};
 3
 4#[derive(Serialize, Deserialize)]
 5struct Person {
 6    name: String,
 7    age: i32,
 8    phones: Vec<String>,
 9}
10
11fn main() {
12    // Some BSON input data as a `Bson`.
13    let bson_data: Bson = bson!({
14        "name": "John Doe",
15        "age": 43,
16        "phones": [
17            "+44 1234567",
18            "+44 2345678"
19        ]
20    });
21
22    // Deserialize the Person struct from the BSON data.
23    let person: Person = bson::from_bson(bson_data).unwrap();
24
25    // Do things just like with any other Rust data structure.
26    println!("Redacting {}'s record.", person.name);
27
28    // Get a serialized version of the input data as a `Bson`.
29    let redacted_bson = bson::to_bson(&person).unwrap();
30    println!("{}", redacted_bson);
31}

相关资料

Serde

JSON

RON

BSON