class PStore - pstore: Ruby Standard Library Documentation

# class PStore

PStore implements a file based persistence mechanism based on a Hash. User code can store hierarchies of Ruby objects (values) into the data store by name (keys). An object hierarchy may be just a single object. User code may later read values back from the data store or even update data, as needed.

The transactional behavior ensures that any changes succeed or fail together. This can be used to ensure that the data store is not left in a transitory state, where some values were updated but others were not.

Behind the scenes, Ruby objects are stored to the data store file with Marshal. That carries the usual limitations. Proc objects cannot be marshalled, for example.

There are three important concepts here (details at the links):

- Store: a store is an instance of PStore.

- Entries: the store is hash-like; each entry is the key for a stored object.

- Transactions: each transaction is a collection of prospective changes to the store; a transaction is defined in the block given with a call to `PStore#transaction`.

## About the Examples¶ ↑

Examples on this page need a store that has known properties. They can get a new (and populated) store by calling thus:

    example_store do |store|
      # Example code using store goes here.
    end

All we really need to know about `example_store` is that it yields a fresh store with a known population of entries; its implementation:

    require 'pstore'
    require 'tempfile'
    # Yield a pristine store for use in examples.
    def example_store
      # Create the store in a temporary file.
      Tempfile.create do |file|
        store = PStore.new(file)
        # Populate the store.
        store.transaction do
          store[:foo] = 0
          store[:bar] = 1
          store[:baz] = 2
        end
        yield store
      end
    end

## The Store¶ ↑

The contents of the store are maintained in a file whose path is specified when the store is created (see `PStore.new`). The objects are stored and retrieved using module Marshal, which means that certain objects cannot be added to the store; see Marshal::dump.

## Entries¶ ↑

A store may have any number of entries. Each entry has a key and a value, just as in a hash:

- Key: as in a hash, the key can be (almost) any object; see Hash. You may find it convenient to keep it simple by using only symbols or strings as keys.

- Value: the value may be any object that can be marshalled by Marshal (see Marshal::dump) and in fact may be a collection (e.g., an array, a hash, a set, a range, etc). That collection may in turn contain nested objects, including collections, to any depth; those objects must also be Marshal-able. See Hierarchical Values.

## Transactions¶ ↑

### The Transaction Block¶ ↑

The block given with a call to method `transaction`# contains a transaction, which consists of calls to PStore methods that read from or write to the store (that is, all PStore methods except `transaction` itself, `path`, and Pstore.new):

    example_store do |store|
      store.transaction do
        store.keys # => [:foo, :bar, :baz]
        store[:bat] = 3
        store.keys # => [:foo, :bar, :baz, :bat]
      end
    end

Execution of the transaction is deferred until the block exits, and is executed atomically (all-or-nothing): either all transaction calls are executed, or none are. This maintains the integrity of the store.

Other code in the block (including even calls to `path` and `PStore.new`) is executed immediately, not deferred.

The transaction block:

- May not contain a nested call to `transaction`.

- Is the only context where methods that read from or write to the store are allowed.

As seen above, changes in a transaction are made automatically when the block exits. The block may be exited early by calling method `commit` or `abort`.

- Method `commit` triggers the update to the store and exits the block:

      example_store do |store|
        store.transaction do
          store.keys # => [:foo, :bar, :baz]
          store[:bat] = 3
          store.commit
          fail 'Cannot get here'
        end
        store.transaction do
          # Update was completed.
          store.keys # => [:foo, :bar, :baz, :bat]
        end
      end

- Method `abort` discards the update to the store and exits the block:

      example_store do |store|
        store.transaction do
          store.keys # => [:foo, :bar, :baz]
          store[:bat] = 3
          store.abort
          fail 'Cannot get here'
        end
        store.transaction do
          # Update was not completed.
          store.keys # => [:foo, :bar, :baz]
        end
      end

### Read-Only Transactions¶ ↑

By default, a transaction allows both reading from and writing to the store:

    store.transaction do
      # Read-write transaction.
      # Any code except a call to #transaction is allowed here.
    end

If argument `read_only` is passed as `true`, only reading is allowed:

    store.transaction(true) do
      # Read-only transaction:
      # Calls to #transaction, #[]=, and #delete are not allowed here.
    end

## Hierarchical Values¶ ↑

The value for an entry may be a simple object (as seen above). It may also be a hierarchy of objects nested to any depth:

    deep_store = PStore.new('deep.store')
    deep_store.transaction do
      array_of_hashes = [{}, {}, {}]
      deep_store[:array_of_hashes] = array_of_hashes
      deep_store[:array_of_hashes] # => [{}, {}, {}]
      hash_of_arrays = {foo: [], bar: [], baz: []}
      deep_store[:hash_of_arrays] = hash_of_arrays
      deep_store[:hash_of_arrays]  # => {:foo=>[], :bar=>[], :baz=>[]}
      deep_store[:hash_of_arrays][:foo].push(:bat)
      deep_store[:hash_of_arrays]  # => {:foo=>[:bat], :bar=>[], :baz=>[]}
    end

And recall that you can use dig methods in a returned hierarchy of objects.

## Working with the Store¶ ↑

### Creating a Store¶ ↑

Use method `PStore.new` to create a store. The new store creates or opens its containing file:

    store = PStore.new('t.store')

### Modifying the Store¶ ↑

Use method `[]=` to update or create an entry:

    example_store do |store|
      store.transaction do
        store[:foo] = 1 # Update.
        store[:bam] = 1 # Create.
      end
    end

Use method `delete` to remove an entry:

    example_store do |store|
      store.transaction do
        store.delete(:foo)
        store[:foo] # => nil
      end
    end

### Retrieving Values¶ ↑

Use method `fetch` (allows default) or `[]` (defaults to `nil`) to retrieve an entry:

    example_store do |store|
      store.transaction do
        store[:foo]             # => 0
        store[:nope]            # => nil
        store.fetch(:baz)       # => 2
        store.fetch(:nope, nil) # => nil
        store.fetch(:nope)      # Raises exception.
      end
    end

### Querying the Store¶ ↑

Use method `key?` to determine whether a given key exists:

    example_store do |store|
      store.transaction do
        store.key?(:foo) # => true
      end
    end

Use method `keys` to retrieve keys:

    example_store do |store|
      store.transaction do
        store.keys # => [:foo, :bar, :baz]
      end
    end

Use method `path` to retrieve the path to the store’s underlying file; this method may be called from outside a transaction block:

    store = PStore.new('t.store')
    store.path # => "t.store"

## Transaction Safety¶ ↑

For transaction safety, see:

- Optional argument `thread_safe` at method `PStore.new`.

- Attribute `ultra_safe`.

Needless to say, if you’re storing valuable data with PStore, then you should backup the PStore file from time to time.

## An Example Store¶ ↑

    require "pstore"

    # A mock wiki object.
    class WikiPage

      attr_reader :page_name

      def initialize(page_name, author, contents)
        @page_name = page_name
        @revisions = Array.new
        add_revision(author, contents)
      end

      def add_revision(author, contents)
        @revisions << {created: Time.now,
                       author: author,
                       contents: contents}
      end

      def wiki_page_references
        [@page_name] + @revisions.last[:contents].scan(/\b(?:[A-Z]+[a-z]+){2,}/)
      end

    end

    # Create a new wiki page.
    home_page = WikiPage.new("HomePage", "James Edward Gray II",
                             "A page about the JoysOfDocumentation..." )

    wiki = PStore.new("wiki_pages.pstore")
    # Update page data and the index together, or not at all.
    wiki.transaction do
      # Store page.
      wiki[home_page.page_name] = home_page
      # Create page index.
      wiki[:wiki_index] ||= Array.new
      # Update wiki index.
      wiki[:wiki_index].push(*home_page.wiki_page_references)
    end

    # Read wiki data, setting argument read_only to true.
    wiki.transaction(true) do
      wiki.keys.each do |key|
        puts key
        puts wiki[key]
      end
    end

### Constants

CHECKSUM_ALGO

  Constant for relieving Ruby’s garbage collector.

EMPTY_MARSHAL_CHECKSUM
EMPTY_MARSHAL_DATA
EMPTY_STRING
RDWR_ACCESS
RD_ACCESS
VERSION
WR_ACCESS

### Attributes

ultra_safe[RW]

Whether PStore should do its best to prevent file corruptions, even when an unlikely error (such as memory-error or filesystem error) occurs:

- `true`: changes are posted by creating a temporary file, writing the updated data to it, then renaming the file to the given `path`. File integrity is maintained. Note: has effect only if the filesystem has atomic file rename (as do POSIX platforms Linux, MacOS, FreeBSD and others).

- `false` (the default): changes are posted by rewinding the open file and writing the updated data. File integrity is maintained if the filesystem raises no unexpected I/O error; if such an error occurs during a write to the store, the file may become corrupted.

### Public Class Methods

new(file, thread_safe = false) click to toggle source

Returns a new PStore object.

Argument `file` is the path to the file in which objects are to be stored; if the file exists, it should be one that was written by PStore.

    path = 't.store'
    store = PStore.new(path)

A PStore object is reentrant. If argument `thread_safe` is given as `true`, the object is also thread-safe (at the cost of a small performance penalty):

    store = PStore.new(path, true)

    # File pstore.rb, line 372
    def initialize(file, thread_safe = false)
      dir = File::dirname(file)
      unless File::directory? dir
        raise PStore::Error, format("directory %s does not exist", dir)
      end
      if File::exist? file and not File::readable? file
        raise PStore::Error, format("file %s not readable", file)
      end
      @filename = file
      @abort = false
      @ultra_safe = false
      @thread_safe = thread_safe
      @lock = Thread::Mutex.new
    end

### Public Instance Methods

[](key) click to toggle source

Returns the value for the given `key` if the key exists. `nil` otherwise; if not `nil`, the returned value is an object or a hierarchy of objects:

    example_store do |store|
      store.transaction do
        store[:foo]  # => 0
        store[:nope] # => nil
      end
    end

Returns `nil` if there is no such key.

See also Hierarchical Values.

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 417
    def [](key)
      in_transaction
      @table[key]
    end

[]=(key, value) click to toggle source

Creates or replaces the value for the given `key`:

    example_store do |store|
      temp.transaction do
        temp[:bat] = 3
      end
    end

See also Hierarchical Values.

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 459
    def []=(key, value)
      in_transaction_wr
      @table[key] = value
    end

abort() click to toggle source

Exits the current transaction block, discarding any changes specified in the transaction block.

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 535
    def abort
      in_transaction
      @abort = true
      throw :pstore_abort_transaction
    end

commit() click to toggle source

Exits the current transaction block, committing any changes specified in the transaction block.

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 524
    def commit
      in_transaction
      @abort = false
      throw :pstore_abort_transaction
    end

delete(key) click to toggle source

Removes and returns the value at `key` if it exists:

    example_store do |store|
      store.transaction do
        store[:bat] = 3
        store.delete(:bat)
      end
    end

Returns `nil` if there is no such key.

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 476
    def delete(key)
      in_transaction_wr
      @table.delete key
    end

fetch(key, default=PStore::Error) click to toggle source

Like `[]`, except that it accepts a default value for the store. If the `key` does not exist:

- Raises an exception if `default` is `PStore::Error`.

- Returns the value of `default` otherwise:

      example_store do |store|
        store.transaction do
          store.fetch(:nope, nil) # => nil
          store.fetch(:nope)      # Raises an exception.
        end
      end

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 436
    def fetch(key, default=PStore::Error)
      in_transaction
      unless @table.key? key
        if default == PStore::Error
          raise PStore::Error, format("undefined key '%s'", key)
        else
          return default
        end
      end
      @table[key]
    end

key?(key) click to toggle source

Returns `true` if `key` exists, `false` otherwise:

    example_store do |store|
      store.transaction do
        store.key?(:foo) # => true
      end
    end

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 505
    def key?(key)
      in_transaction
      @table.key? key
    end

Also aliased as: root?
keys() click to toggle source

Returns an array of the existing keys:

    example_store do |store|
      store.transaction do
        store.keys # => [:foo, :bar, :baz]
      end
    end

Raises an exception if called outside a transaction block.

    # File pstore.rb, line 490
    def keys
      in_transaction
      @table.keys
    end

Also aliased as: roots
path() click to toggle source

Returns the string file path used to create the store:

    store.path # => "flat.store"

    # File pstore.rb, line 515
    def path
      @filename
    end

root?(key)
Alias for: key?
roots()
Alias for: keys
transaction(read_only = false) { |pstore| ... } click to toggle source

Opens a transaction block for the store. See Transactions.

With argument `read_only` as `false`, the block may both read from and write to the store.

With argument `read_only` as `true`, the block may not include calls to `transaction`, `[]=`, or `delete`.

Raises an exception if called within a transaction block.

    # File pstore.rb, line 551
    def transaction(read_only = false)  # :yields:  pstore
      value = nil
      if !@thread_safe
        raise PStore::Error, "nested transaction" unless @lock.try_lock
      else
        begin
          @lock.lock
        rescue ThreadError
          raise PStore::Error, "nested transaction"
        end
      end
      begin
        @rdonly = read_only
        @abort = false
        file = open_and_lock_file(@filename, read_only)
        if file
          begin
            @table, checksum, original_data_size = load_data(file, read_only)

            catch(:pstore_abort_transaction) do
              value = yield(self)
            end

            if !@abort && !read_only
              save_data(checksum, original_data_size, file)
            end
          ensure
            file.close
          end
        else
          # This can only occur if read_only == true.
          @table = {}
          catch(:pstore_abort_transaction) do
            value = yield(self)
          end
        end
      ensure
        @lock.unlock
      end
      value
    end

### Private Instance Methods

empty_marshal_checksum() click to toggle source

    # File pstore.rb, line 728
    def empty_marshal_checksum
      EMPTY_MARSHAL_CHECKSUM
    end

empty_marshal_data() click to toggle source

    # File pstore.rb, line 725
    def empty_marshal_data
      EMPTY_MARSHAL_DATA
    end

in_transaction() click to toggle source

Raises `PStore::Error` if the calling code is not in a `PStore#transaction`.

    # File pstore.rb, line 388
    def in_transaction
      raise PStore::Error, "not in transaction" unless @lock.locked?
    end

in_transaction_wr() click to toggle source

Raises `PStore::Error` if the calling code is not in a `PStore#transaction` or if the code is in a read-only `PStore#transaction`.

    # File pstore.rb, line 395
    def in_transaction_wr
      in_transaction
      raise PStore::Error, "in read-only transaction" if @rdonly
    end

load_data(file, read_only) click to toggle source

Load the given `PStore` file. If `read_only` is true, the unmarshalled Hash will be returned. If `read_only` is false, a 3-tuple will be returned: the unmarshalled Hash, a checksum of the data, and the size of the data.

    # File pstore.rb, line 639
    def load_data(file, read_only)
      if read_only
        begin
          table = load(file)
          raise Error, "PStore file seems to be corrupted." unless table.is_a?(Hash)
        rescue EOFError
          # This seems to be a newly-created file.
          table = {}
        end
        table
      else
        data = file.read
        if data.empty?
          # This seems to be a newly-created file.
          table = {}
          checksum = empty_marshal_checksum
          size = empty_marshal_data.bytesize
        else
          table = load(data)
          checksum = CHECKSUM_ALGO.digest(data)
          size = data.bytesize
          raise Error, "PStore file seems to be corrupted." unless table.is_a?(Hash)
        end
        data.replace(EMPTY_STRING)
        [table, checksum, size]
      end
    end

on_windows?() click to toggle source

    # File pstore.rb, line 667
    def on_windows?
      is_windows = RUBY_PLATFORM =~ /mswin|mingw|bccwin|wince/
      self.class.__send__(:define_method, :on_windows?) do
        is_windows
      end
      is_windows
    end

open_and_lock_file(filename, read_only) click to toggle source

Open the specified filename (either in read-only mode or in read-write mode) and lock it for reading or writing.

The opened File object will be returned. If read_only is true, and the file does not exist, then nil will be returned.

All exceptions are propagated.

    # File pstore.rb, line 614
    def open_and_lock_file(filename, read_only)
      if read_only
        begin
          file = File.new(filename, **RD_ACCESS)
          begin
            file.flock(File::LOCK_SH)
            return file
          rescue
            file.close
            raise
          end
        rescue Errno::ENOENT
          return nil
        end
      else
        file = File.new(filename, **RDWR_ACCESS)
        file.flock(File::LOCK_EX)
        return file
      end
    end

save_data(original_checksum, original_file_size, file) click to toggle source

    # File pstore.rb, line 675
    def save_data(original_checksum, original_file_size, file)
      new_data = dump(@table)

      if new_data.bytesize != original_file_size || CHECKSUM_ALGO.digest(new_data) != original_checksum
        if @ultra_safe && !on_windows?
          # Windows doesn't support atomic file renames.
          save_data_with_atomic_file_rename_strategy(new_data, file)
        else
          save_data_with_fast_strategy(new_data, file)
        end
      end

      new_data.replace(EMPTY_STRING)
    end

save_data_with_atomic_file_rename_strategy(data, file) click to toggle source

    # File pstore.rb, line 690
    def save_data_with_atomic_file_rename_strategy(data, file)
      temp_filename = "#{@filename}.tmp.#{Process.pid}.#{rand 1000000}"
      temp_file = File.new(temp_filename, **WR_ACCESS)
      begin
        temp_file.flock(File::LOCK_EX)
        temp_file.write(data)
        temp_file.flush
        File.rename(temp_filename, @filename)
      rescue
        File.unlink(temp_file) rescue nil
        raise
      ensure
        temp_file.close
      end
    end

save_data_with_fast_strategy(data, file) click to toggle source

    # File pstore.rb, line 706
    def save_data_with_fast_strategy(data, file)
      file.rewind
      file.write(data)
      file.truncate(data.bytesize)
    end