[ 文章收集 ] Serializing (And Deserializing) Objects With Ruby

Source From Here
Preface
Serialization is one of those things you can easily do without until all of a sudden you really need it one day. That's pretty much how it went with me. I was happily using and learning Ruby for months before I ever ran into a situation where serializing a few objects really would have made my life easier. Even then I avoided looking into it, you can very easily convert the important data from an object into a string and write that out to a file. Then when you need to, you just read the file, parse the string and recreate the object, what could be simpler? Of course, it could be much simpler indeed, especially when you're dealing with a deep hierarchy of objects. I think being weaned on languages like Java, you come to expect operations like serialization to be non-trivial. Don't get me wrong, it is not really difficult in Java, but neither is it simple and if you want your serialized object to be human-readable, then you're into 3rd party library land and things can get easier or harder depending on your needs. Suffice to say, bad experiences in the past don't fill you with a lot of enthusiasm for the future.

When I started looking into serialization in Ruby, I fully expected to have to look into 3rd party solutions – surely the serialization mechanisms built into the language couldn't possibly, easily fit my needs. As usual, I was pleasantly surprised. Now, like that proverbial hammer, serialization seems to be useful all the time :). Anyway, I'll let you judge for yourself, let's take a look at the best and most common options you have, when it comes to serialization with Ruby.

Human-Readable Objects
Ruby has two object serialization mechanisms built right into the language. One is used to serialize into a human readable format, the other into a binary format. I will look into the binary one shortly, but for now let’s focus on human readable. Any object you create in Ruby can be serialized into YAML format, with pretty much no effort needed on your part. Let’s make some objects:

view plaincopy to clipboardprint?

1. require "yaml"  
2.   
3. class A  
4.   def initialize(string, number)  
5.     @string = string  
6.     @number = number  
7.   end  
8.   
9.   def to_s  
10.     "In A:\n   #{@string}, #{@number}\n"  
11.   end  
12. end  
13.   
14. class B  
15.   def initialize(number, a_object)  
16.     @number = number  
17.     @a_object = a_object  
18.   end  
19.   
20.   def to_s  
21.     "In B: #{@number} \n  #{@a_object.to_s}\n"  
22.   end  
23. end  
24.   
25. class C  
26.   def initialize(b_object, a_object)  
27.     @b_object = b_object  
28.     @a_object = a_object  
29.   end  
30.   
31.   def to_s  
32.     "In C:\n #{@a_object} #{@b_object}\n"  
33.   end  
34. end  
35.   
36. a = A.new("hello world", 5)  
37. b = B.new(7, a)  
38. c = C.new(b, a)  
39.   
40. puts c  

Since we created a to_s, method, we can see the string representation of our object tree:


To serialize our object tree we simply do the following:

view plaincopy to clipboardprint?

1. serialized_object = YAML::dump(c)  
2. puts serialized_object  

Our serialized object looks like this:


If we now want to get it back:

view plaincopy to clipboardprint?

1. puts YAML::load(serialized_object)  

This produces output which is exactly the same as what we had above, which means our object tree was reproduced correctly:


Of course you almost never want to serialize just one object, it is usually an array or a hash. In this case you have two options, either you serialize the whole array/hash in one go, or you serialize each value separately. The rule here is simple, if you always need to work with the whole set of data and never parts of it, just write out the whole array/hash, otherwise, iterate over it and write out each object. The reason you do this is almost always to share the data with someone else.

If you just write out the whole array/hash in one fell swoop then it is as simple as what we did above. When you do it one object at a time, it is a little more complicated, since we don't want to write it out to a whole bunch of files, but rather all of them to one file. It is a little more complicated since you want to be able to easily read your objects back in again which can be tricky as YAML serialization creates multiple lines per object. Here is a trick you can use, when you write the objects out, separate them with two newlines e.g.:

view plaincopy to clipboardprint?

1. File.open("/home/alan/tmp/blah.yaml", "w") do |file|  
2.   (1..10).each do |index|  
3.     file.puts YAML::dump(A.new("hello world", index))  
4.     file.puts ""  
5.   end  
6. end  

The file will look like this:


Then when you want to read all the objects back, simply set the input record separator to be two newlines e.g.:

view plaincopy to clipboardprint?

1. array = []  
2. $/="\n\n"  
3. File.open("/home/alan/tmp/blah.yaml", "r").each do |object|  
4.   array << YAML::load(object)  
5. end  
6.   
7. puts array  

The output is:


Which is exactly what we expect – handy.

A 3rd Party Alternative
Of course, if we don't want to resort to tricks like that, but still keep our serialized objects human-readable, we have another alternative which is basically as common as the Ruby built in serialization methods – JSON. The JSON support in Ruby is provided by a 3rd party library, all you need to do is:

# gem install json

or

# gem install json-pure

The second one is if you want a pure Ruby implementation (no native extensions).

The good thing about JSON, is the fact that it is even more human readable than YAML. It is also a "low-fat" alternative to XML and can be used to transport data over the wire by AJAX calls that require data from the server (that's the simple one sentence explanation :)). The other good news when it comes to serializing objects to JSON using Ruby is that if you save the object to a file, it saves it on one line, so we don't have to resort to tricks when saving multiple objects and reading them back again.

There is bad news of course, in that your objects won't automagically be converted to JSON, unless all you're using is hashes, arrays and primitives. You need to do a little bit of work to make sure your custom object is serializable. Let’s make one of the classes we introduced previously serializable using JSON.

view plaincopy to clipboardprint?

1. require "json"  
2.   
3. class A  
4.   def initialize(string, number)  
5.     @string = string  
6.     @number = number  
7.   end  
8.   
9.   def to_s  
10.     "In A:\n   #{@string}, #{@number}\n"  
11.   end  
12.   
13.   def to_json(*a)  
14.     {  
15.       "json_class"   => self.class.name,  
16.       "data"         => {"string" => @string, "number" => @number }  
17.     }.to_json(*a)  
18.   end  
19.   
20.   def self.json_create(o)  
21.     new(o["data"]["string"], o["data"]["number"])  
22.   end  
23. end  

Now you can simply do the following:

view plaincopy to clipboardprint?

1. a = A.new("hello world", 5)  
2. json_string = a.to_json  
3. puts json_string  
4. puts JSON.parse(json_string)  

Which produces output like this:


The first string is our serialized JSON string, and the second is the result of outputting our deserialized object, which gives the output that we expect. As you can see, we implement two methods:

* to_json – called on the object instance and allows us to convert an object into a JSON string.
* json_create – allows us to call JSON.parse passing in a JSON string which will convert the string into an instance of our object

You can also see that, when converting our object into a JSON string we need to make sure, that we end up with a hash and that contains the 'json_class' key. We also need to make sure that we only use hashes, arrays, primitives (i.e. integers, floats etc., not really primitives in Ruby but you get the picture) and strings.

So, JSON has some advantages and some disadvantages. I like it because it is widely supported so you can send data around and have it be recognised by other apps. I don't like it because you need to do work to make sure your objects are easily serializable, so if you don't need to send your data anywhere but simply want to share it locally, it is a bit of a pain.

Binary Serialization
The other serialization mechanism built into Ruby is binary serialization using Marshal. It is very similar to YAML and just as easy to use, the only difference is it's not human readable as it stores your objects in a binary format. You use Marshal exactly the same way you use YAML, but replace the word YAML with Marshal :)

view plaincopy to clipboardprint?

1. a = A.new("hello world", 5)  
2. puts a  
3. serialized_object = Marshal::dump(a)  
4. puts Marshal::load(serialized_object)  

As you can see, according to the output the objects before and after serialization are the same. You don't even need to require anything :). The thing to watch out for when outputting multiple Marshalled objects to the same file, is the record separator. Since you're writing binary data, it is not inconceivable that you may end up with a newline somewhere in a record accidentally, which will stuff everything up when you try to read the objects back in. So two rules of thumb to remember are:

* Don't use puts when outputting Marshalled objects to a file (use print instead), this way you avoid the extraneous newline from the puts
* Use a record separator other than newline, you can make anything unlikely up (if you scroll down a bit you will see that I used '—_—' as a separator)

The disadvantage of Marshal is the fact the its output it not human-readable. The advantage is its speed.

Which One To Choose?
It's simple, if you need to be able to read your serializable data then you have to go with one of the human-readable formats (YAML or JSON). I'd go with YAML purely because you don't need to do any work to get your custom objects to serialize properly, and the fact that it serializes each object as a multiline string is not such a big deal (as I showed above). The only times I would go with JSON (aside the whole wide support and sending it over the wire deal), is if you need to be able to easily edit your data by hand, or when you need human-readable data and you have a lot of data to deal with (see benchmarks below).

If you don't really need to be able to read your data, then always go with Marshal, especially if you have a lot of data.

Here is a situation I commonly have to deal with. I have a CSV file, or some other kind of data file, I want to read it, parse it and create an object per row or at least a hash per row, to make the data easier to deal with. What I like to do is read this CSV file, create my object and serialize them to a file at the same time using Marshal. This way I can operate on the whole data set or parts of the data set, by simply reading the serialized objects in, and it is orders of magnitude faster than reading the CSV file again. Let's do some benchmarks. I will create 500000 objects (a relatively small set of data) and serialize them all to a file using all three methods.

view plaincopy to clipboardprint?

1. require "benchmark"  
2.   
3. def benchmark_serialize(output_file)  
4.   Benchmark.realtime do  
5.     File.open(output_file, "w") do |file|  
6.       (1..500000).each do |index|  
7.         yield(file, A.new("hello world", index))  
8.       end  
9.     end  
10.   end  
11. end  
12.   
13. puts "YAML:"  
14. time = benchmark_serialize("/home/alan/tmp/yaml.dat") do |file, object|  
15.   file.puts YAML::dump(object)  
16.   file.puts ""  
17. end  
18. puts "Time: #{time} sec"  
19.   
20. puts "JSON:"  
21. time = benchmark_serialize("/home/alan/tmp/json.dat") do |file, object|  
22.   file.puts object.to_json  
23. end  
24. puts "Time: #{time} sec"  
25.   
26. puts "Marshal:"  
27. time = benchmark_serialize("/home/alan/tmp/marshal.dat") do |file, object|  
28.   file.print Marshal::dump(object)  
29.   file.print "---_---"  
30. end  
31. puts "Time: #{time} sec"  

What about deserializing all the objects:

view plaincopy to clipboardprint?

1. def benchmark_deserialize(input_file, array, input_separator)  
2.   $/=input_separator  
3.   Benchmark.realtime do  
4.     File.open(input_file, "r").each do |object|  
5.       array << yield(object)  
6.     end  
7.   end  
8. end  
9.   
10. array1 = []  
11. puts "YAML:"  
12. time = benchmark_deserialize("/home/alan/tmp/yaml.dat", array1, "\n\n") do |object|  
13.   YAML::load(object)  
14. end  
15. puts "Array size: #{array1.length}"  
16. puts "Time: #{time} sec"  
17.   
18. array2 = []  
19. puts "JSON:"  
20. time = benchmark_deserialize("/home/alan/tmp/json.dat", array2, "\n") do |object|  
21.   JSON.parse(object)  
22. end  
23. puts "Array size: #{array2.length}"  
24. puts "Time: #{time} sec"  
25.   
26. array3 = []  
27. puts "Marshal:"  
28. time = benchmark_deserialize("/home/alan/tmp/marshal.dat", array3, "---_---") do |object|  
29.   Marshal::load(object.chomp)  
30. end  
31. puts "Array size: #{array3.length}"  
32. puts "Time: #{time} sec"  

As you can see, it is significantly faster to serialize objects when you're using Marshal, although JSON is only about 2 times slower. YAML gets left in the dust. When deserializing, the differences are not as apparent, although Marshal is still the clear winner. The more data you have to deal with the more telling these results will be. So, for pure speed – choose Marshal. For speed and human readability – choose JSON (at the expense of having to add methods to custom objects). For human readability with relatively small sets of data – go with YAML.

Supplement
* Ruby 1.9 Core API - Marshal

The marshaling library converts collections of Ruby objects into a byte stream, allowing them to be stored outside the currently active script. This data may subsequently be read and the original objects reconstituted.

* Stackoverflow - How do I read/write binary files?
* Rubylearning.com - Object Serialization

view plaincopy to clipboardprint?

1. File.open('game') do |f|    
2.   @gc = Marshal.load(f)    
3. end