Class: Rufus::Tokyo::Cabinet

A ‘cabinet’, ie a Tokyo Cabinet [abstract] database.

Follows the abstract API described at :

  http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

An usage example :

  db = Rufus::Tokyo::Cabinet.new('test_data.tch')
  db['pillow'] = 'Shonagon'

  db.size # => 1
  db['pillow'] # => 'Shonagon'

  db.delete('pillow') # => 'Shonagon'
  db.size # => 0

  db.close

Constructor Summary

public initialize(name, params = {})

Creates/opens the cabinet, raises an exception in case of creation/opening failure.

This method accepts a ‘name’ parameter and an optional ‘params’ hash parameter.

‘name’ follows the syntax described at

  http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

under tcadbopen(). For example :

  db = Rufus::Tokyo::Cabinet.new('casket.tch#bnum=100000#opts=ld')

will open (eventually create) a hash database backed in the file ‘casket.tch’ with a bucket number of 100000 and the ‘large’ and ‘deflate’ options (opts) turned on.

Note that there is an #open method similar to File#open for openening a db and closing it when it’s no longer needed :

  Rufus::Tokyo::Cabinet.new('data.tch') do |db|
    db['key'] = value
  end

database name

From http://tokyocabinet.sourceforge.net/spex-en.html#tcadbapi :

‘If it is "*", the database will be an on-memory hash database. If it is

 "<tt>", the database will be an on-memory tree database. If its suffix is
 ".tch", the database will be a hash database. If its suffix is ".tcb",
 the database will be a B</tt> tree database. If its suffix is ".tcf", the
 database will be a fixed-length database. If its suffix is ".tct", the
 database will be a table database.'

You’re supposed to give a path to the database file you want to use and Cabinet expects you to give the proper prefix.

  db = Rufus::Tokyo::Cabinet.new('data.tch') # hash database
  db = Rufus::Tokyo::Cabinet.new('data.tcb') # B+ tree db
  db = Rufus::Tokyo::Cabinet.new('data.tcf') # fixed-length db

will result with the same file names :

  db = Rufus::Tokyo::Cabinet.new('data', :type => :hash) # hash database
  db = Rufus::Tokyo::Cabinet.new('data', :type => :btree) # B+ tree db
  db = Rufus::Tokyo::Cabinet.new('data', :type => :fixed) # fixed-length db

You can open an in-memory hash and an in-memory B+ tree with :

  h = Rufus::Tokyo::Cabinet.new(:mem_hash) # or
  h = Rufus::Tokyo::Cabinet.new('*')

  t = Rufus::Tokyo::Cabinet.new(:mem_tree) # or
  t = Rufus::Tokyo::Cabinet.new('+')

parameters

There are two ways to pass parameters at the opening of a db :

  db = Rufus::Tokyo::Cabinet.new('data.tch#opts=ld#mode=w') # or
  db = Rufus::Tokyo::Cabinet.new('data.tch', :opts => 'ld', :mode => 'w')

most verbose :

  db = Rufus::Tokyo::Cabinet.new(
    'data', :type => :hash, :opts => 'ld', :mode => 'w')

mode

  * :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
             'e' non locking, 'f' non blocking lock), default is 'wc'

other parameters

‘On-memory hash database supports "bnum", "capnum", and "capsiz".

 On-memory tree database supports "capnum" and "capsiz".
 Hash database supports "mode", "bnum", "apow", "fpow", "opts",
 "rcnum", and "xmsiz".
 B+ tree database supports "mode", "lmemb", "nmemb", "bnum", "apow",
 "fpow", "opts", "lcnum", "ncnum", and "xmsiz".
 Fixed-length database supports "mode", "width", and "limsiz"'

  * :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
             (usually empty or something like 'ld' or 'lb')

  * :bnum    number of elements of the bucket array
  * :apow    size of record alignment by power of 2 (defaults to 4)
  * :fpow    maximum number of elements of the free block pool by
             power of 2 (defaults to 10)
  * :mutex   when set to true, makes sure only 1 thread at a time
             accesses the table (well, Ruby, global thread lock, ...)

  * :rcnum   specifies the maximum number of records to be cached.
             If it is not more than 0, the record cache is disabled.
             It is disabled by default.
  * :lcnum   specifies the maximum number of leaf nodes to be cached.
             If it is not more than 0, the default value is specified.
             The default value is 2048.
  * :ncnum   specifies the maximum number of non-leaf nodes to be
             cached. If it is not more than 0, the default value is
             specified. The default value is 512.

  * :xmsiz   specifies the size of the extra mapped memory. If it is
             not more than 0, the extra mapped memory is disabled.
             The default size is 67108864.

  * :capnum  specifies the capacity number of records.
  * :capsiz  specifies the capacity size of using memory.

  * :dfunit  unit step number. If it is not more than 0,
             the auto defragmentation is disabled. (Since TC 1.4.21)

NOTE :

On reopening a file, Cabinet will tend to stick to the parameters as set when the file was opened. To change that, have a look at the man pages of the various command line tools coming with Tokyo Cabinet.

[View source]


179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 179

def initialize (name, params={})

  @db = lib.tcadbnew

  name = '*' if name == :mem_hash # in memory hash database
  name = '+' if name == :mem_tree # in memory B+ tree database

  if type = params.delete(:type)
    name += { :hash => '.tch', :btree => '.tcb', :fixed => '.tcf' }[type]
  end

  @path = name

  name = name + params.collect { |k, v| "##{k}=#{v}" }.join('')

  (lib.tcadbopen(@db, name) == 1) || raise(
    TokyoError.new("failed to open/create db '#{name}' #{params.inspect}"))

  self.default = params[:default]
  @default_proc ||= params[:default_proc]
end

Public Visibility

Public Class Method Summary

new_hash(params = {})

Returns a new in-memory hash.

new_tree(params = {})

Returns a new in-memory B+ tree.

open(name, params = {})

Same args as initialize, but can take a block form that will close the db when done.

Public Instance Method Summary

#[]=(k, v)

No comment.

#clear

Removes all the records in the cabinet (use with care).

#close

Closes the cabinet (and frees the datastructure allocated for it), returns true in case of success.

#compact_copy(target_path)

Copies the current cabinet to a new file.

#copy(target_path)

Copies the current cabinet to a new file.

#defrag

Triggers a defrag run (TC >= 1.

#delete(k)

Removes a record from the cabinet, returns the value if successful else nil.

#delete_keys_with_prefix(prefix)

Deletes all the entries whose keys begin with the given prefix.

#get4(k) #getdup

This is a B+ Tree method only, returns all the values for a given key.

#incr(key, inc = 1) #add_double #addint #adddouble #add_int

Increments the value stored under the given key with the given increment (defaults to 1 (integer)).

#keys(options = {})

Returns an array with all the keys in the databse.

#ldelete(keys)

Given a list of keys, deletes all the matching entries (in one sweep).

#lget(keys) #mget

Given a list of keys, returns a Hash { key => value } of the matching entries (in one sweep).

#lib

Using the cabinet lib.

#merge!(hash) #lput

Merges the given hash into this Cabinet (or Tyrant) and returns self.

#path

Returns the path to this database.

#putcat(k, v)

Appends the given string at the end of the current string value for key k.

#putdup(k, v)

--.

#putkeep(k, v)

Like #put but doesn’t overwrite the value if already set.

#size

Returns the number of records in the ‘cabinet’.

#sync

"synchronize updated contents of an abstract database object with the file and the device".

#tranabort

Warning : this method is low-level, you probably only need to use #transaction and a block.

#tranbegin

Warning : this method is low-level, you probably only need to use #transaction and a block.

#trancommit

Warning : this method is low-level, you probably only need to use #transaction and a block.

#weight

Returns the ‘weight’ of the db (in bytes).

Public Instance Methods Included from Rufus::Tokyo::HashMethods

[], default, default=, each, merge, to_a, to_h, values

Public Instance Methods Included from Rufus::Tokyo::Transactions

abort, transaction

Public Instance Methods Included from Rufus::Tokyo::Outlen

outlen_op

Public Class Method Details

new_hash

public new_hash(params = {})

Returns a new in-memory hash. Accepts the same optional params hash as new().

[View source]


223
224
225
226
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 223

def self.new_hash (params={})

  self.new(:hash, params)
end

new_tree

public new_tree(params = {})

Returns a new in-memory B+ tree. Accepts the same optional params hash as new().

[View source]


231
232
233
234
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 231

def self.new_tree (params={})

  self.new(:tree, params)
end

open

public open(name, params = {})

Same args as initialize, but can take a block form that will close the db when done. Similar to File.open

[View source]


204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 204

def self.open (name, params={})

  db = self.new(name, params)

  if block_given?
    yield db
    nil
  else
    db
  end

ensure

  db.close if block_given? && db
end

Public Instance Method Details

[]=

public []=(k, v)

No comment

[View source]


252
253
254
255
256
257
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 252

def []= (k, v)

  k = k.to_s; v = v.to_s

  lib.abs_put(@db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

clear

public clear

Removes all the records in the cabinet (use with care)

Returns self (like Ruby’s Hash does).

[View source]


316
317
318
319
320
321
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 316

def clear

  lib.abs_vanish(@db)

  self
end

close

public close

Closes the cabinet (and frees the datastructure allocated for it), returns true in case of success.

[View source]


333
334
335
336
337
338
339
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 333

def close

  result = lib.abs_close(@db)
  lib.abs_del(@db)

  (result == 1)
end

compact_copy

public compact_copy(target_path)

Copies the current cabinet to a new file.

Does it by copying each entry afresh to the target file. Spares some space, hence the ‘compact’ label…

[View source]


355
356
357
358
359
360
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 355

def compact_copy (target_path)

  @other_db = Cabinet.new(target_path)
  self.each { |k, v| @other_db[k] = v }
  @other_db.close
end

copy

public copy(target_path)

Copies the current cabinet to a new file.

Returns true if it was successful.

[View source]


345
346
347
348
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 345

def copy (target_path)

  (lib.abs_copy(@db, target_path) == 1)
end

defrag

public defrag

Triggers a defrag run (TC >= 1.4.21 only)

Meta Tags

Raises:

[NotImplementedError]
[View source]


497
498
499
500
501
502
503
504
505
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 497

def defrag

  raise(NotImplementedError.new(
    "method defrag is supported since Tokyo Cabinet 1.4.21. " +
    "your TC version doesn't support it"
  )) unless lib.respond_to?(:tctdbsetdfunit)

  call_misc('defrag', Rufus::Tokyo::List.new)
end

delete

public delete(k)

Removes a record from the cabinet, returns the value if successful else nil.

[View source]


296
297
298
299
300
301
302
303
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 296

def delete (k)

  k = k.to_s

  v = self[k]

  (lib.abs_out(@db, k, Rufus::Tokyo.blen(k)) == 1) ? v : nil
end

delete_keys_with_prefix

public delete_keys_with_prefix(prefix)

Deletes all the entries whose keys begin with the given prefix

[View source]


424
425
426
427
428
429
430
431
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 424

def delete_keys_with_prefix (prefix)

  call_misc(
    'outlist', lib.abs_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1))
      # -1 for no limits

  nil
end

get4

public get4(k)

Also known as: getdup

This is a B+ Tree method only, returns all the values for a given key.

[View source]


561
562
563
564
565
566
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 561

def get4 (k)

  l = lib.tcbdbget4(as_btree, k, Rufus::Tokyo.blen(k))

  Rufus::Tokyo::List.new(l).release
end

incr

public incr(key, inc = 1)

Also known as: add_double addint adddouble add_int

Increments the value stored under the given key with the given increment (defaults to 1 (integer)).

Accepts an integer or a double value.

Meta Tags

Raises:

[TokyoError]
[View source]


475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 475

def incr (key, inc=1)

  key = key.to_s

  v = inc.is_a?(Fixnum) ?
    lib.addint(@db, key, Rufus::Tokyo.blen(key), inc) :
    lib.adddouble(@db, key, Rufus::Tokyo.blen(key), inc)

  raise(TokyoError.new(
    "incr failed, there is probably already a string value set " +
    "for the key '#{key}'"
  )) if v == Rufus::Tokyo::INT_MIN || (v.respond_to?(:nan?) && v.nan?)

  v
end

keys

public keys(options = {})

Returns an array with all the keys in the databse

With no options given, this method will return all the keys (strings) in a Ruby array.

  :prefix --> returns only the keys who match a given string prefix

  :limit --> returns a limited number of keys

  :native --> returns an instance of Rufus::Tokyo::List instead of
    a Ruby Hash, you have to call #free on that List when done with it !
    Else you're exposing yourself to a memory leak.
[View source]


383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 383

def keys (options={})

  outlen = nil

  if pre = options[:prefix]

    l = lib.abs_fwmkeys(
      @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

    l = Rufus::Tokyo::List.new(l)

    options[:native] ? l : l.release

  else

    limit = options[:limit] || -1
    limit = nil if limit < 1

    l = options[:native] ? Rufus::Tokyo::List.new : []

    lib.abs_iterinit(@db)

    outlen = FFI::MemoryPointer.new(:int)

    loop do
      break if limit and l.size >= limit
      out = lib.abs_iternext(@db, outlen)
      break if out.address == 0
      l << out.get_bytes(0, outlen.get_int(0))
    end

    l
  end

ensure

  outlen.free if outlen
end

ldelete

public ldelete(keys)

Given a list of keys, deletes all the matching entries (in one sweep).

[View source]


463
464
465
466
467
468
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 463

def ldelete (keys)

  keys = keys.collect { |k| k.to_s }

  call_misc('outlist', Rufus::Tokyo::List.new(keys))
end

lget

public lget(keys)

Also known as: mget

Given a list of keys, returns a Hash { key => value } of the matching entries (in one sweep).

[View source]


436
437
438
439
440
441
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 436

def lget (keys)

  keys = keys.collect { |k| k.to_s }

  Hash[*call_misc('getlist', Rufus::Tokyo::List.new(keys))]
end

lib

public lib

Using the cabinet lib

[View source]


238
239
240
241
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 238

def lib

  CabinetLib
end

merge!

public merge!(hash)

Also known as: lput

Merges the given hash into this Cabinet (or Tyrant) and returns self.

[View source]


447
448
449
450
451
452
453
454
455
456
457
458
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 447

def merge! (hash)

  call_misc(
    'putlist',
    hash.inject(Rufus::Tokyo::List.new) { |l, (k, v)|
      l << k.to_s
      l << v.to_s
      l
    })

  self
end

path

public path

Returns the path to this database.

[View source]


245
246
247
248
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 245

def path

  @path
end

putcat

public putcat(k, v)

Appends the given string at the end of the current string value for key k. If there is no record for key k, a new record will be created.

Returns true if successful.

[View source]


275
276
277
278
279
280
281
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 275

def putcat (k, v)

  k = k.to_s; v = v.to_s

  (lib.abs_putcat(
    @db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) == 1)
end

putdup

public putdup(k, v)

BTREE methods

++ This is a B+ Tree method only, puts a value for a key who has

potentially
multiple values.
[View source]


552
553
554
555
556
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 552

def putdup (k, v)

  lib.tcbdbputdup(
    as_btree, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v))
end

putkeep

public putkeep(k, v)

Like #put but doesn’t overwrite the value if already set. Returns true only if there no previous entry for k.

[View source]


262
263
264
265
266
267
268
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 262

def putkeep (k, v)

  k = k.to_s; v = v.to_s

  (lib.abs_putkeep(
    @db, k, Rufus::Tokyo.blen(k), v, Rufus::Tokyo.blen(v)) == 1)
end

size

public size

Returns the number of records in the ‘cabinet’

[View source]


307
308
309
310
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 307

def size

  lib.abs_rnum(@db)
end

sync

public sync

"synchronize updated contents of an abstract database object with the file and the device"

[View source]


365
366
367
368
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 365

def sync

  (lib.abs_sync(@db) == 1)
end

tranabort

public tranabort

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.

[View source]


536
537
538
539
540
541
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 536

def tranabort

  #check_transaction_support

  libcall(:tcadbtranabort)
end

tranbegin

public tranbegin

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.

[View source]


512
513
514
515
516
517
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 512

def tranbegin

  #check_transaction_support

  libcall(:tcadbtranbegin)
end

trancommit

public trancommit

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.

[View source]


524
525
526
527
528
529
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 524

def trancommit

  #check_transaction_support

  libcall(:tcadbtrancommit)
end

weight

public weight

Returns the ‘weight’ of the db (in bytes)

[View source]


325
326
327
328
# File 'lib/rufus/tokyo/cabinet/abstract.rb', line 325

def weight

  lib.abs_size(@db)
end
Generated on Friday, September 18 2009 at 10:35:40 AM by YARD 0.2.3.5 (ruby-1.8.7).