Source From Here
Preface
Serialization is one of those things you can easily do without until all of a sudden you really need it one day. That's pretty much how it went with me. I was happily using and learning Ruby for months before I ever ran into a situation where serializing a few objects really would have made my life easier. Even then I avoided looking into it, you can very easily convert the important data from an object into a string and write that out to a file. Then when you need to, you just read the file, parse the string and recreate the object, what could be simpler? Of course, it could be much simpler indeed, especially when you're dealing with a deep hierarchy of objects. I think being weaned on languages like Java, you come to expect operations like serialization to be non-trivial. Don't get me wrong, it is not really difficult in Java, but neither is it simple and if you want your serialized object to be human-readable, then you're into 3rd party library land and things can get easier or harder depending on your needs. Suffice to say, bad experiences in the past don't fill you with a lot of enthusiasm for the future.
When I started looking into serialization in Ruby, I fully expected to have to look into 3rd party solutions – surely the serialization mechanisms built into the language couldn't possibly, easily fit my needs. As usual, I was pleasantly surprised. Now, like that proverbial hammer, serialization seems to be useful all the time :). Anyway, I'll let you judge for yourself, let's take a look at the best and most common options you have, when it comes to serialization with Ruby.
Human-Readable Objects
Ruby has two object serialization mechanisms built right into the language. One is used to serialize into a human readable format, the other into a binary format. I will look into the binary one shortly, but for now let’s focus on human readable. Any object you create in Ruby can be serialized into YAML format, with pretty much no effort needed on your part. Let’s make some objects:
Since we created a to_s, method, we can see the string representation of our object tree:
To serialize our object tree we simply do the following:
Our serialized object looks like this:
If we now want to get it back:
This produces output which is exactly the same as what we had above, which means our object tree was reproduced correctly:
Of course you almost never want to serialize just one object, it is usually an array or a hash. In this case you have two options, either you serialize the whole array/hash in one go, or you serialize each value separately. The rule here is simple, if you always need to work with the whole set of data and never parts of it, just write out the whole array/hash, otherwise, iterate over it and write out each object. The reason you do this is almost always to share the data with someone else.
If you just write out the whole array/hash in one fell swoop then it is as simple as what we did above. When you do it one object at a time, it is a little more complicated, since we don't want to write it out to a whole bunch of files, but rather all of them to one file. It is a little more complicated since you want to be able to easily read your objects back in again which can be tricky as YAML serialization creates multiple lines per object. Here is a trick you can use, when you write the objects out, separate them with two newlines e.g.:
The file will look like this:
Then when you want to read all the objects back, simply set the input record separator to be two newlines e.g.:
The output is:
Which is exactly what we expect – handy.
A 3rd Party Alternative
Of course, if we don't want to resort to tricks like that, but still keep our serialized objects human-readable, we have another alternative which is basically as common as the Ruby built in serialization methods – JSON. The JSON support in Ruby is provided by a 3rd party library, all you need to do is:
or
The second one is if you want a pure Ruby implementation (no native extensions).
The good thing about JSON, is the fact that it is even more human readable than YAML. It is also a "low-fat" alternative to XML and can be used to transport data over the wire by AJAX calls that require data from the server (that's the simple one sentence explanation :)). The other good news when it comes to serializing objects to JSON using Ruby is that if you save the object to a file, it saves it on one line, so we don't have to resort to tricks when saving multiple objects and reading them back again.
There is bad news of course, in that your objects won't automagically be converted to JSON, unless all you're using is hashes, arrays and primitives. You need to do a little bit of work to make sure your custom object is serializable. Let’s make one of the classes we introduced previously serializable using JSON.
Now you can simply do the following:
Which produces output like this:
The first string is our serialized JSON string, and the second is the result of outputting our deserialized object, which gives the output that we expect. As you can see, we implement two methods:
You can also see that, when converting our object into a JSON string we need to make sure, that we end up with a hash and that contains the 'json_class' key. We also need to make sure that we only use hashes, arrays, primitives (i.e. integers, floats etc., not really primitives in Ruby but you get the picture) and strings.
So, JSON has some advantages and some disadvantages. I like it because it is widely supported so you can send data around and have it be recognised by other apps. I don't like it because you need to do work to make sure your objects are easily serializable, so if you don't need to send your data anywhere but simply want to share it locally, it is a bit of a pain.
Binary Serialization
The other serialization mechanism built into Ruby is binary serialization using Marshal. It is very similar to YAML and just as easy to use, the only difference is it's not human readable as it stores your objects in a binary format. You use Marshal exactly the same way you use YAML, but replace the word YAML with Marshal :)
As you can see, according to the output the objects before and after serialization are the same. You don't even need to require anything :). The thing to watch out for when outputting multiple Marshalled objects to the same file, is the record separator. Since you're writing binary data, it is not inconceivable that you may end up with a newline somewhere in a record accidentally, which will stuff everything up when you try to read the objects back in. So two rules of thumb to remember are:
The disadvantage of Marshal is the fact the its output it not human-readable. The advantage is its speed.
Which One To Choose?
It's simple, if you need to be able to read your serializable data then you have to go with one of the human-readable formats (YAML or JSON). I'd go with YAML purely because you don't need to do any work to get your custom objects to serialize properly, and the fact that it serializes each object as a multiline string is not such a big deal (as I showed above). The only times I would go with JSON (aside the whole wide support and sending it over the wire deal), is if you need to be able to easily edit your data by hand, or when you need human-readable data and you have a lot of data to deal with (see benchmarks below).
If you don't really need to be able to read your data, then always go with Marshal, especially if you have a lot of data.
Here is a situation I commonly have to deal with. I have a CSV file, or some other kind of data file, I want to read it, parse it and create an object per row or at least a hash per row, to make the data easier to deal with. What I like to do is read this CSV file, create my object and serialize them to a file at the same time using Marshal. This way I can operate on the whole data set or parts of the data set, by simply reading the serialized objects in, and it is orders of magnitude faster than reading the CSV file again. Let's do some benchmarks. I will create 500000 objects (a relatively small set of data) and serialize them all to a file using all three methods.
What about deserializing all the objects:
As you can see, it is significantly faster to serialize objects when you're using Marshal, although JSON is only about 2 times slower. YAML gets left in the dust. When deserializing, the differences are not as apparent, although Marshal is still the clear winner. The more data you have to deal with the more telling these results will be. So, for pure speed – choose Marshal. For speed and human readability – choose JSON (at the expense of having to add methods to custom objects). For human readability with relatively small sets of data – go with YAML.
Supplement
* Ruby 1.9 Core API - Marshal
* Stackoverflow - How do I read/write binary files?
* Rubylearning.com - Object Serialization
Preface
Serialization is one of those things you can easily do without until all of a sudden you really need it one day. That's pretty much how it went with me. I was happily using and learning Ruby for months before I ever ran into a situation where serializing a few objects really would have made my life easier. Even then I avoided looking into it, you can very easily convert the important data from an object into a string and write that out to a file. Then when you need to, you just read the file, parse the string and recreate the object, what could be simpler? Of course, it could be much simpler indeed, especially when you're dealing with a deep hierarchy of objects. I think being weaned on languages like Java, you come to expect operations like serialization to be non-trivial. Don't get me wrong, it is not really difficult in Java, but neither is it simple and if you want your serialized object to be human-readable, then you're into 3rd party library land and things can get easier or harder depending on your needs. Suffice to say, bad experiences in the past don't fill you with a lot of enthusiasm for the future.
When I started looking into serialization in Ruby, I fully expected to have to look into 3rd party solutions – surely the serialization mechanisms built into the language couldn't possibly, easily fit my needs. As usual, I was pleasantly surprised. Now, like that proverbial hammer, serialization seems to be useful all the time :). Anyway, I'll let you judge for yourself, let's take a look at the best and most common options you have, when it comes to serialization with Ruby.
Human-Readable Objects
Ruby has two object serialization mechanisms built right into the language. One is used to serialize into a human readable format, the other into a binary format. I will look into the binary one shortly, but for now let’s focus on human readable. Any object you create in Ruby can be serialized into YAML format, with pretty much no effort needed on your part. Let’s make some objects:
- require "yaml"
- class A
- def initialize(string, number)
- @string = string
- @number = number
- end
- def to_s
- "In A:\n #{@string}, #{@number}\n"
- end
- end
- class B
- def initialize(number, a_object)
- @number = number
- @a_object = a_object
- end
- def to_s
- "In B: #{@number} \n #{@a_object.to_s}\n"
- end
- end
- class C
- def initialize(b_object, a_object)
- @b_object = b_object
- @a_object = a_object
- end
- def to_s
- "In C:\n #{@a_object} #{@b_object}\n"
- end
- end
- a = A.new("hello world", 5)
- b = B.new(7, a)
- c = C.new(b, a)
- puts c
To serialize our object tree we simply do the following:
- serialized_object = YAML::dump(c)
- puts serialized_object
If we now want to get it back:
- puts YAML::load(serialized_object)
Of course you almost never want to serialize just one object, it is usually an array or a hash. In this case you have two options, either you serialize the whole array/hash in one go, or you serialize each value separately. The rule here is simple, if you always need to work with the whole set of data and never parts of it, just write out the whole array/hash, otherwise, iterate over it and write out each object. The reason you do this is almost always to share the data with someone else.
If you just write out the whole array/hash in one fell swoop then it is as simple as what we did above. When you do it one object at a time, it is a little more complicated, since we don't want to write it out to a whole bunch of files, but rather all of them to one file. It is a little more complicated since you want to be able to easily read your objects back in again which can be tricky as YAML serialization creates multiple lines per object. Here is a trick you can use, when you write the objects out, separate them with two newlines e.g.:
- File.open("/home/alan/tmp/blah.yaml", "w") do |file|
- (1..10).each do |index|
- file.puts YAML::dump(A.new("hello world", index))
- file.puts ""
- end
- end
Then when you want to read all the objects back, simply set the input record separator to be two newlines e.g.:
- array = []
- $/="\n\n"
- File.open("/home/alan/tmp/blah.yaml", "r").each do |object|
- array << YAML::load(object)
- end
- puts array
Which is exactly what we expect – handy.
A 3rd Party Alternative
Of course, if we don't want to resort to tricks like that, but still keep our serialized objects human-readable, we have another alternative which is basically as common as the Ruby built in serialization methods – JSON. The JSON support in Ruby is provided by a 3rd party library, all you need to do is:
or
The second one is if you want a pure Ruby implementation (no native extensions).
The good thing about JSON, is the fact that it is even more human readable than YAML. It is also a "low-fat" alternative to XML and can be used to transport data over the wire by AJAX calls that require data from the server (that's the simple one sentence explanation :)). The other good news when it comes to serializing objects to JSON using Ruby is that if you save the object to a file, it saves it on one line, so we don't have to resort to tricks when saving multiple objects and reading them back again.
There is bad news of course, in that your objects won't automagically be converted to JSON, unless all you're using is hashes, arrays and primitives. You need to do a little bit of work to make sure your custom object is serializable. Let’s make one of the classes we introduced previously serializable using JSON.
- require "json"
- class A
- def initialize(string, number)
- @string = string
- @number = number
- end
- def to_s
- "In A:\n #{@string}, #{@number}\n"
- end
- def to_json(*a)
- {
- "json_class" => self.class.name,
- "data" => {"string" => @string, "number" => @number }
- }.to_json(*a)
- end
- def self.json_create(o)
- new(o["data"]["string"], o["data"]["number"])
- end
- end
- a = A.new("hello world", 5)
- json_string = a.to_json
- puts json_string
- puts JSON.parse(json_string)
The first string is our serialized JSON string, and the second is the result of outputting our deserialized object, which gives the output that we expect. As you can see, we implement two methods:
You can also see that, when converting our object into a JSON string we need to make sure, that we end up with a hash and that contains the 'json_class' key. We also need to make sure that we only use hashes, arrays, primitives (i.e. integers, floats etc., not really primitives in Ruby but you get the picture) and strings.
So, JSON has some advantages and some disadvantages. I like it because it is widely supported so you can send data around and have it be recognised by other apps. I don't like it because you need to do work to make sure your objects are easily serializable, so if you don't need to send your data anywhere but simply want to share it locally, it is a bit of a pain.
Binary Serialization
The other serialization mechanism built into Ruby is binary serialization using Marshal. It is very similar to YAML and just as easy to use, the only difference is it's not human readable as it stores your objects in a binary format. You use Marshal exactly the same way you use YAML, but replace the word YAML with Marshal :)
- a = A.new("hello world", 5)
- puts a
- serialized_object = Marshal::dump(a)
- puts Marshal::load(serialized_object)
As you can see, according to the output the objects before and after serialization are the same. You don't even need to require anything :). The thing to watch out for when outputting multiple Marshalled objects to the same file, is the record separator. Since you're writing binary data, it is not inconceivable that you may end up with a newline somewhere in a record accidentally, which will stuff everything up when you try to read the objects back in. So two rules of thumb to remember are:
The disadvantage of Marshal is the fact the its output it not human-readable. The advantage is its speed.
Which One To Choose?
It's simple, if you need to be able to read your serializable data then you have to go with one of the human-readable formats (YAML or JSON). I'd go with YAML purely because you don't need to do any work to get your custom objects to serialize properly, and the fact that it serializes each object as a multiline string is not such a big deal (as I showed above). The only times I would go with JSON (aside the whole wide support and sending it over the wire deal), is if you need to be able to easily edit your data by hand, or when you need human-readable data and you have a lot of data to deal with (see benchmarks below).
If you don't really need to be able to read your data, then always go with Marshal, especially if you have a lot of data.
Here is a situation I commonly have to deal with. I have a CSV file, or some other kind of data file, I want to read it, parse it and create an object per row or at least a hash per row, to make the data easier to deal with. What I like to do is read this CSV file, create my object and serialize them to a file at the same time using Marshal. This way I can operate on the whole data set or parts of the data set, by simply reading the serialized objects in, and it is orders of magnitude faster than reading the CSV file again. Let's do some benchmarks. I will create 500000 objects (a relatively small set of data) and serialize them all to a file using all three methods.
- require "benchmark"
- def benchmark_serialize(output_file)
- Benchmark.realtime do
- File.open(output_file, "w") do |file|
- (1..500000).each do |index|
- yield(file, A.new("hello world", index))
- end
- end
- end
- end
- puts "YAML:"
- time = benchmark_serialize("/home/alan/tmp/yaml.dat") do |file, object|
- file.puts YAML::dump(object)
- file.puts ""
- end
- puts "Time: #{time} sec"
- puts "JSON:"
- time = benchmark_serialize("/home/alan/tmp/json.dat") do |file, object|
- file.puts object.to_json
- end
- puts "Time: #{time} sec"
- puts "Marshal:"
- time = benchmark_serialize("/home/alan/tmp/marshal.dat") do |file, object|
- file.print Marshal::dump(object)
- file.print "---_---"
- end
- puts "Time: #{time} sec"
What about deserializing all the objects:
- def benchmark_deserialize(input_file, array, input_separator)
- $/=input_separator
- Benchmark.realtime do
- File.open(input_file, "r").each do |object|
- array << yield(object)
- end
- end
- end
- array1 = []
- puts "YAML:"
- time = benchmark_deserialize("/home/alan/tmp/yaml.dat", array1, "\n\n") do |object|
- YAML::load(object)
- end
- puts "Array size: #{array1.length}"
- puts "Time: #{time} sec"
- array2 = []
- puts "JSON:"
- time = benchmark_deserialize("/home/alan/tmp/json.dat", array2, "\n") do |object|
- JSON.parse(object)
- end
- puts "Array size: #{array2.length}"
- puts "Time: #{time} sec"
- array3 = []
- puts "Marshal:"
- time = benchmark_deserialize("/home/alan/tmp/marshal.dat", array3, "---_---") do |object|
- Marshal::load(object.chomp)
- end
- puts "Array size: #{array3.length}"
- puts "Time: #{time} sec"
As you can see, it is significantly faster to serialize objects when you're using Marshal, although JSON is only about 2 times slower. YAML gets left in the dust. When deserializing, the differences are not as apparent, although Marshal is still the clear winner. The more data you have to deal with the more telling these results will be. So, for pure speed – choose Marshal. For speed and human readability – choose JSON (at the expense of having to add methods to custom objects). For human readability with relatively small sets of data – go with YAML.
Supplement
* Ruby 1.9 Core API - Marshal
* Stackoverflow - How do I read/write binary files?
* Rubylearning.com - Object Serialization
沒有留言:
張貼留言