A ‘cabinet’, ie a Tokyo Cabinet [abstract] database.
Follows the abstract API described at :
http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi
An usage example :
db = Rufus::Tokyo::Cabinet.new('test_data.tch')
db['pillow'] = 'Shonagon'
db.size # => 1
db['pillow'] # => 'Shonagon'
db.delete('pillow') # => 'Shonagon'
db.size # => 0
db.close
Methods
public class
public instance
- []=
- add_double
- add_int
- adddouble
- addint
- clear
- close
- compact_copy
- copy
- defrag
- delete
- delete_keys_with_prefix
- incr
- keys
- ldelete
- lget
- lib
- lput
- merge!
- path
- putkeep
- size
- sync
- tranabort
- tranbegin
- trancommit
- weight
protected instance
Included modules
Public class methods
Creates/opens the cabinet, raises an exception in case of creation/opening failure.
This method accepts a ‘name’ parameter and an optional ‘params’ hash parameter.
‘name’ follows the syntax described at
http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi
under tcadbopen(). For example :
db = Rufus::Tokyo::Cabinet.new('casket.tch#bnum=100000#opts=ld')
will open (eventually create) a hash database backed in the file ‘casket.tch’ with a bucket number of 100000 and the ‘large’ and ‘deflate’ options (opts) turned on.
Note that there is an open method similar to File#open for openening a db and closing it when it’s no longer needed :
Rufus::Tokyo::Cabinet.new('data.tch') do |db|
db['key'] = value
end
database name
From tokyocabinet.sourceforge.net/spex-en.html#tcadbapi :
‘If it is “*”, the database will be an on-memory hash database. If it is
"+", the database will be an on-memory tree database. If its suffix is ".tch", the database will be a hash database. If its suffix is ".tcb", the database will be a B+ tree database. If its suffix is ".tcf", the database will be a fixed-length database. If its suffix is ".tct", the database will be a table database.'
You’re supposed to give a path to the database file you want to use and Cabinet expects you to give the proper prefix.
db = Rufus::Tokyo::Cabinet.new('data.tch') # hash database
db = Rufus::Tokyo::Cabinet.new('data.tcb') # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data.tcf') # fixed-length db
will result with the same file names :
db = Rufus::Tokyo::Cabinet.new('data', :type => :hash) # hash database
db = Rufus::Tokyo::Cabinet.new('data', :type => :btree) # B+ tree db
db = Rufus::Tokyo::Cabinet.new('data', :type => :fixed) # fixed-length db
You can open an in-memory hash and an in-memory B+ tree with :
h = Rufus::Tokyo::Cabinet.new(:mem_hash) # or
h = Rufus::Tokyo::Cabinet.new('*')
t = Rufus::Tokyo::Cabinet.new(:mem_tree) # or
t = Rufus::Tokyo::Cabinet.new('+')
parameters
There are two ways to pass parameters at the opening of a db :
db = Rufus::Tokyo::Cabinet.new('data.tch#opts=ld#mode=w') # or
db = Rufus::Tokyo::Cabinet.new('data.tch', :opts => 'ld', :mode => 'w')
most verbose :
db = Rufus::Tokyo::Cabinet.new( 'data', :type => :hash, :opts => 'ld', :mode => 'w')
mode
* :mode a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
'e' non locking, 'f' non blocking lock), default is 'wc'
other parameters
‘On-memory hash database supports “bnum”, “capnum”, and “capsiz”.
On-memory tree database supports "capnum" and "capsiz".
Hash database supports "mode", "bnum", "apow", "fpow", "opts",
"rcnum", and "xmsiz".
B+ tree database supports "mode", "lmemb", "nmemb", "bnum", "apow",
"fpow", "opts", "lcnum", "ncnum", and "xmsiz".
Fixed-length database supports "mode", "width", and "limsiz"'
* :opts a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
(usually empty or something like 'ld' or 'lb')
* :bnum number of elements of the bucket array
* :apow size of record alignment by power of 2 (defaults to 4)
* :fpow maximum number of elements of the free block pool by
power of 2 (defaults to 10)
* :mutex when set to true, makes sure only 1 thread at a time
accesses the table (well, Ruby, global thread lock, ...)
* :rcnum specifies the maximum number of records to be cached.
If it is not more than 0, the record cache is disabled.
It is disabled by default.
* :lcnum specifies the maximum number of leaf nodes to be cached.
If it is not more than 0, the default value is specified.
The default value is 2048.
* :ncnum specifies the maximum number of non-leaf nodes to be
cached. If it is not more than 0, the default value is
specified. The default value is 512.
* :xmsiz specifies the size of the extra mapped memory. If it is
not more than 0, the extra mapped memory is disabled.
The default size is 67108864.
* :capnum specifies the capacity number of records.
* :capsiz specifies the capacity size of using memory.
* :dfunit unit step number. If it is not more than 0,
the auto defragmentation is disabled. (Since TC 1.4.21)
NOTE :
On reopening a file, Cabinet will tend to stick to the parameters as set when the file was opened. To change that, have a look at the man pages of the various command line tools coming with Tokyo Cabinet.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 179 179: def initialize (name, params={}) 180: 181: @db = lib.tcadbnew 182: 183: name = '*' if name == :mem_hash # in memory hash database 184: name = '+' if name == :mem_tree # in memory B+ tree database 185: 186: if type = params.delete(:type) 187: name += { :hash => '.tch', :btree => '.tcb', :fixed => '.tcf' }[type] 188: end 189: 190: @path = name 191: 192: name = name + params.collect { |k, v| "##{k}=#{v}" }.join('') 193: 194: (lib.tcadbopen(@db, name) == 1) || raise( 195: TokyoError.new("failed to open/create db '#{name}' #{params.inspect}")) 196: 197: self.default = params[:default] 198: @default_proc ||= params[:default_proc] 199: end
Returns a new in-memory hash. Accepts the same optional params hash as new().
# File lib/rufus/tokyo/cabinet/abstract.rb, line 223 223: def self.new_hash (params={}) 224: 225: self.new(:hash, params) 226: end
Returns a new in-memory B+ tree. Accepts the same optional params hash as new().
# File lib/rufus/tokyo/cabinet/abstract.rb, line 231 231: def self.new_tree (params={}) 232: 233: self.new(:tree, params) 234: end
Same args as initialize, but can take a block form that will close the db when done. Similar to File.open
# File lib/rufus/tokyo/cabinet/abstract.rb, line 204 204: def self.open (name, params={}) 205: 206: db = self.new(name, params) 207: 208: if block_given? 209: yield db 210: nil 211: else 212: db 213: end 214: 215: ensure 216: 217: db.close if block_given? && db 218: end
Public instance methods
No comment
# File lib/rufus/tokyo/cabinet/abstract.rb, line 252 252: def []= (k, v) 253: 254: k = k.to_s; v = v.to_s 255: 256: lib.abs_put(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) 257: end
Removes all the records in the cabinet (use with care)
Returns self (like Ruby’s Hash does).
# File lib/rufus/tokyo/cabinet/abstract.rb, line 303 303: def clear 304: 305: lib.abs_vanish(@db) 306: 307: self 308: end
Closes the cabinet (and frees the datastructure allocated for it), returns true in case of success.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 320 320: def close 321: 322: result = lib.abs_close(@db) 323: lib.abs_del(@db) 324: 325: (result == 1) 326: end
Copies the current cabinet to a new file.
Does it by copying each entry afresh to the target file. Spares some space, hence the ‘compact’ label...
# File lib/rufus/tokyo/cabinet/abstract.rb, line 342 342: def compact_copy (target_path) 343: 344: @other_db = Cabinet.new(target_path) 345: self.each { |k, v| @other_db[k] = v } 346: @other_db.close 347: end
Copies the current cabinet to a new file.
Returns true if it was successful.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 332 332: def copy (target_path) 333: 334: (lib.abs_copy(@db, target_path) == 1) 335: end
Triggers a defrag run (TC >= 1.4.21 only)
# File lib/rufus/tokyo/cabinet/abstract.rb, line 482 482: def defrag 483: 484: raise(NotImplementedError.new( 485: "method defrag is supported since Tokyo Cabinet 1.4.21. " + 486: "your TC version doesn't support it" 487: )) unless lib.respond_to?(:tctdbsetdfunit) 488: 489: call_misc('defrag', Rufus::Tokyo::List.new) 490: end
Removes a record from the cabinet, returns the value if successful else nil.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 283 283: def delete (k) 284: 285: k = k.to_s 286: 287: v = self[k] 288: 289: (lib.abs_out(@db, k, Rufus::Tokyo.blen(k)) == 1) ? v : nil 290: end
Deletes all the entries whose keys begin with the given prefix
# File lib/rufus/tokyo/cabinet/abstract.rb, line 411 411: def delete_keys_with_prefix (prefix) 412: 413: call_misc( 414: 'outlist', lib.abs_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1)) 415: # -1 for no limits 416: 417: nil 418: end
Increments the value stored under the given key with the given increment (defaults to 1 (integer)).
Accepts an integer or a double value.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 460 460: def incr (key, inc=1) 461: 462: key = key.to_s 463: 464: v = inc.is_a?(Fixnum) ? 465: lib.addint(@db, key, Rufus::Tokyo.blen(key), inc) : 466: lib.adddouble(@db, key, Rufus::Tokyo.blen(key), inc) 467: 468: raise(TokyoError.new( 469: "incr failed, there is probably already a string value set " + 470: "for the key '#{key}'" 471: )) if v == Rufus::Tokyo::INT_MIN || (v.respond_to?(:nan?) && v.nan?) 472: 473: v 474: end
Returns an array with all the keys in the databse
With no options given, this method will return all the keys (strings) in a Ruby array.
:prefix --> returns only the keys who match a given string prefix :limit --> returns a limited number of keys :native --> returns an instance of Rufus::Tokyo::List instead of a Ruby Hash, you have to call #free on that List when done with it ! Else you're exposing yourself to a memory leak.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 370 370: def keys (options={}) 371: 372: outlen = nil 373: 374: if pre = options[:prefix] 375: 376: l = lib.abs_fwmkeys( 377: @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1) 378: 379: l = Rufus::Tokyo::List.new(l) 380: 381: options[:native] ? l : l.release 382: 383: else 384: 385: limit = options[:limit] || -1 386: limit = nil if limit < 1 387: 388: l = options[:native] ? Rufus::Tokyo::List.new : [] 389: 390: lib.abs_iterinit(@db) 391: 392: outlen = FFI::MemoryPointer.new(:int) 393: 394: loop do 395: break if limit and l.size >= limit 396: out = lib.abs_iternext(@db, outlen) 397: break if out.address == 0 398: l << out.get_bytes(0, outlen.get_int(0)) 399: end 400: 401: l 402: end 403: 404: ensure 405: 406: outlen.free if outlen 407: end
Given a list of keys, deletes all the matching entries (in one sweep).
# File lib/rufus/tokyo/cabinet/abstract.rb, line 448 448: def ldelete (keys) 449: 450: keys = keys.collect { |k| k.to_s } 451: 452: call_misc('outlist', Rufus::Tokyo::List.new(keys)) 453: end
Given a list of keys, returns a Hash { key => value } of the matching entries (in one sweep).
# File lib/rufus/tokyo/cabinet/abstract.rb, line 423 423: def lget (keys) 424: 425: keys = keys.collect { |k| k.to_s } 426: 427: Hash[*call_misc('getlist', Rufus::Tokyo::List.new(keys))] 428: end
Using the cabinet lib
# File lib/rufus/tokyo/cabinet/abstract.rb, line 238 238: def lib 239: 240: CabinetLib 241: end
# File lib/rufus/tokyo/cabinet/abstract.rb, line 432 432: def merge! (hash) 433: 434: call_misc( 435: 'putlist', 436: hash.inject(Rufus::Tokyo::List.new) { |l, (k, v)| 437: l << k.to_s 438: l << v.to_s 439: l 440: }) 441: 442: self 443: end
Returns the path to this database.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 245 245: def path 246: 247: @path 248: end
Like put but doesn’t overwrite the value if already set. Returns true only if there no previous entry for k.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 262 262: def putkeep (k, v) 263: 264: k = k.to_s; v = v.to_s 265: 266: (lib.abs_putkeep( 267: @db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) == 1) 268: end
Returns the number of records in the ‘cabinet’
# File lib/rufus/tokyo/cabinet/abstract.rb, line 294 294: def size 295: 296: lib.abs_rnum(@db) 297: end
“synchronize updated contents of an abstract database object with the file and the device“
# File lib/rufus/tokyo/cabinet/abstract.rb, line 352 352: def sync 353: 354: (lib.abs_sync(@db) == 1) 355: end
Warning : this method is low-level, you probably only need to use transaction and a block.
Direct call for ‘transaction abort’.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 521 521: def tranabort 522: 523: #check_transaction_support 524: 525: libcall(:tcadbtranabort) 526: end
Warning : this method is low-level, you probably only need to use transaction and a block.
Direct call for ‘transaction begin’.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 497 497: def tranbegin 498: 499: #check_transaction_support 500: 501: libcall(:tcadbtranbegin) 502: end
Warning : this method is low-level, you probably only need to use transaction and a block.
Direct call for ‘transaction commit’.
# File lib/rufus/tokyo/cabinet/abstract.rb, line 509 509: def trancommit 510: 511: #check_transaction_support 512: 513: libcall(:tcadbtrancommit) 514: end
Returns the ‘weight’ of the db (in bytes)
# File lib/rufus/tokyo/cabinet/abstract.rb, line 312 312: def weight 313: 314: lib.abs_size(@db) 315: end
Protected instance methods
Wrapping tcadbmisc or tcrdbmisc (and taking care of freeing the list_pointer)
# File lib/rufus/tokyo/cabinet/abstract.rb, line 540 540: def call_misc (function, list_pointer) 541: 542: list_pointer = list_pointer.pointer \ 543: if list_pointer.is_a?(Rufus::Tokyo::List) 544: 545: begin 546: l = do_call_misc(function, list_pointer) 547: raise "function '#{function}' failed" unless l 548: Rufus::Tokyo::List.new(l).release 549: ensure 550: Rufus::Tokyo::List.free(list_pointer) 551: end 552: end
Calls the tcadbmisc function
# File lib/rufus/tokyo/cabinet/abstract.rb, line 556 556: def do_call_misc (function, list_pointer) 557: 558: lib.tcadbmisc(@db, function, list_pointer) 559: end
(The actual #[] method is provided by HashMethods
# File lib/rufus/tokyo/cabinet/abstract.rb, line 272 272: def get (k) 273: 274: k = k.to_s 275: 276: outlen_op(:abs_get, k, Rufus::Tokyo.blen(k)) 277: end
# File lib/rufus/tokyo/cabinet/abstract.rb, line 561 561: def libcall (lib_method, *args) 562: 563: (eval(%{ lib.#{lib_method}(@db, *args) }) == 1) or \ 564: raise TokyoError.new("call to #{lib_method} failed") 565: end