Class: Rufus::Tokyo::Table

A ‘table’ a table database.

  http://alpha.mixi.co.jp/blog/?p=290
  http://tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

A short example :

  require 'rubygems'
  require 'rufus/tokyo/cabinet/table'

  t = Rufus::Tokyo::Table.new('table.tdb', :create, :write)
    # '.tdb' suffix is a must

  t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
  t['pk1'] = { 'name' => 'bob', 'age' => '18' }
  t['pk2'] = { 'name' => 'charly', 'age' => '45' }
  t['pk3'] = { 'name' => 'doug', 'age' => '77' }
  t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

  p t.query { |q|
    q.add_condition 'age', :numge, '32'
    q.order_by 'age'
    q.limit 2
  }
    # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
    #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

  t.close

Constants

INDEX_TYPES
{ :lexical => 0, :decimal => 1, :token => 2, :qgram => 3, :opt => 9998, :optimized => 9998, :void => 9999, :remove => 9999, :keep => 1 << 24 }
META_TYPES
{ :union => 0, :intersection => 1, :difference => 2, :diff => 2 }

Constructor Summary

public initialize(path, params = {})

this class has tranbegin/trancommit/tranabort so let’s include the transaction mixin Creates a Table instance (creates or opens it depending on the args)

For example,

  t = Rufus::Tokyo::Table.new('table.tdb')
    # '.tdb' suffix is a must

will create the table.tdb (or simply open it if already present) and make sure we have write access to it.

parameters

Parameters can be set in the path or via the optional params hash (like in Rufus::Tokyo::Cabinet)

  * :mode    a set of chars ('r'ead, 'w'rite, 'c'reate, 't'runcate,
             'e' non locking, 'f' non blocking lock), default is 'wc'
  * :opts    a set of chars ('l'arge, 'd'eflate, 'b'zip2, 't'cbs)
             (usually empty or something  'ld' or 'lb')

  * :bnum    number of elements of the bucket array
  * :apow    size of record alignment by power of 2 (defaults to 4)
  * :fpow    maximum number of elements of the free block pool by
             power of 2 (defaults to 10)
  * :mutex   when set to true, makes sure only 1 thread at a time
             accesses the table (well, Ruby, global thread lock, ...)

  * :rcnum   specifies the maximum number of records to be cached.
             If it is not more than 0, the record cache is disabled.
             It is disabled by default.
  * :lcnum   specifies the maximum number of leaf nodes to be cached.
             If it is not more than 0, the default value is specified.
             The default value is 2048.
  * :ncnum   specifies the maximum number of non-leaf nodes to be
             cached. If it is not more than 0, the default value is
             specified. The default value is 512.

  * :xmsiz   specifies the size of the extra mapped memory. If it is
             not more than 0, the extra mapped memory is disabled.
             The default size is 67108864.

  * :dfunit  unit step number. If it is not more than 0,
             the auto defragmentation is disabled. (Since TC 1.4.21)

Some examples :

  t = Rufus::Tokyo::Table.new('table.tdb')
  t = Rufus::Tokyo::Table.new('table.tdb#mode=r')
  t = Rufus::Tokyo::Table.new('table.tdb', :mode => 'r')
  t = Rufus::Tokyo::Table.new('table.tdb#opts=ld#mode=r')
  t = Rufus::Tokyo::Table.new('table.tdb', :opts => 'ld', :mode => 'r')
[View source]


125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
# File 'lib/rufus/tokyo/cabinet/table.rb', line 125

def initialize (path, params={})

  conf = determine_conf(path, params, :table)

  @db = lib.tctdbnew

  #
  # tune table

  libcall(:tctdbsetmutex) if conf[:mutex]

  libcall(:tctdbtune, conf[:bnum], conf[:apow], conf[:fpow], conf[:opts])

  # TODO : set indexes here... well, there is already #set_index
  #conf[:indexes]...

  libcall(:tctdbsetcache, conf[:rcnum], conf[:lcnum], conf[:ncnum])

  libcall(:tctdbsetxmsiz, conf[:xmsiz])

  libcall(:tctdbsetdfunit, conf[:dfunit]) \
    if lib.respond_to?(:tctdbsetdfunit) # TC >= 1.4.21

  #
  # open table

  @path = conf[:path]

  libcall(:tctdbopen, @path, conf[:mode])
end

Public Visibility

Public Instance Method Summary

#[]=(pk, h_or_a)

Inserts a record in the table db.

#clear

Removes all records in this table database.

#close

Closes the table (and frees the datastructure allocated for it), returns true in case of success.

#delete(k)

Removes an entry in the table.

#delete_keys_with_prefix(prefix)

Deletes all the entries whose key begin with the given prefix.

#difference(*queries)

Returns the difference of the listed queries.

#do_query(&block)

Prepares and runs a query, returns a ResultSet instance (takes care of freeing the query structure).

#generate_unique_id #genuid

Generates a unique id (in the context of this Table instance).

#intersection(*queries)

Returns the intersection of the listed queries.

#keys(options = {})

Returns an array of all the primary keys in the table.

#lget(keys) #mget

No ‘misc’ methods for the table library, so this lget is equivalent to calling get for each key.

#lib

Using the cabinet lib.

#path

Returns the path to the table.

#pointer

Returns the actual pointer to the Tokyo Cabinet table.

#prepare_query(&block)

Prepares a query instance (block is optional).

#query(&block)

Prepares and runs a query, returns an array of hashes (all Ruby) (takes care of freeing the query and the result set structures).

#query_delete(&block)

Prepares a query and then runs it and deletes all the results.

#search(type, *queries)

A #search a la ruby-tokyotyrant (http://github.

#set_index(column_name, *types)

Sets an index on a column of the table.

#size

Returns the number of records in this table db.

#tranabort

Warning : this method is low-level, you probably only need to use #transaction and a block.

#tranbegin

Warning : this method is low-level, you probably only need to use #transaction and a block.

#trancommit

Warning : this method is low-level, you probably only need to use #transaction and a block.

#union(*queries)

Returns the union of the listed queries.

Public Instance Methods Included from Rufus::Tokyo::HashMethods

[], default, default=, each, merge, merge!, to_a, to_h, values

Public Instance Methods Included from Rufus::Tokyo::Transactions

abort, transaction

Public Instance Method Details

[]=

public []=(pk, h_or_a)

Inserts a record in the table db

  table['pk0'] = [ 'name', 'fred', 'age', '45' ]
  table['pk1'] = { 'name' => 'jeff', 'age' => '46' }

Accepts both a hash or an array (expects the array to be of the form [ key, value, key, value, … ] else it will raise an ArgumentError)

Raises an error in case of failure.

[View source]


240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
# File 'lib/rufus/tokyo/cabinet/table.rb', line 240

def []= (pk, h_or_a)

  pk = pk.to_s
  h_or_a = Rufus::Tokyo.h_or_a_to_s(h_or_a)

  m = Rufus::Tokyo::Map[h_or_a]

  r = lib.tab_put(@db, pk, Rufus::Tokyo.blen(pk), m.pointer)

  m.free

  (r == 1) || raise_error # raising potential error after freeing map

  h_or_a
end

clear

public clear

Removes all records in this table database

[View source]


274
275
276
277
# File 'lib/rufus/tokyo/cabinet/table.rb', line 274

def clear

  libcall(:tab_vanish)
end

close

public close

Closes the table (and frees the datastructure allocated for it), returns true in case of success.

[View source]


173
174
175
176
177
178
179
# File 'lib/rufus/tokyo/cabinet/table.rb', line 173

def close

  result = lib.tab_close(@db)
  lib.tab_del(@db)

  (result == 1)
end

delete

public delete(k)

Removes an entry in the table

(might raise an error if the delete itself failed, but returns nil if there was no entry for the given key)

[View source]


261
262
263
264
265
266
267
268
269
270
# File 'lib/rufus/tokyo/cabinet/table.rb', line 261

def delete (k)

  k = k.to_s

  v = self[k]
  return nil unless v
  libcall(:tab_out, k, Rufus::Tokyo.blen(k))

  v
end

delete_keys_with_prefix

public delete_keys_with_prefix(prefix)

Deletes all the entries whose key begin with the given prefix.

[View source]


333
334
335
336
337
338
339
340
341
342
343
344
345
346
# File 'lib/rufus/tokyo/cabinet/table.rb', line 333

def delete_keys_with_prefix (prefix)

  # TODO : use ...searchout

  ks = lib.tab_fwmkeys(@db, prefix, Rufus::Tokyo.blen(prefix), -1)
    # -1 for no limit

  begin
    ks = Rufus::Tokyo::List.new(ks)
    ks.each { |k| self.delete(k) }
  ensure
    ks && ks.free
  end
end

difference

public difference(*queries)

Returns the difference of the listed queries

  r = table.intersection(
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'es'
    },
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'li'
    }
  )

will return a hash { primary_key => record } of the values matching the first query OR the second but not both.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

[View source]


512
513
514
515
# File 'lib/rufus/tokyo/cabinet/table.rb', line 512

def difference (*queries)

  search(:difference, *queries)
end

do_query

public do_query(&block)

Prepares and runs a query, returns a ResultSet instance (takes care of freeing the query structure)

[View source]


379
380
381
382
383
384
385
386
387
388
# File 'lib/rufus/tokyo/cabinet/table.rb', line 379

def do_query (&block)

  q = prepare_query(&block)
  rs = q.run

  return rs

ensure
  q && q.free
end

generate_unique_id

public generate_unique_id

Also known as: genuid

Generates a unique id (in the context of this Table instance)

[View source]


183
184
185
186
# File 'lib/rufus/tokyo/cabinet/table.rb', line 183

def generate_unique_id

  lib.tab_genuid(@db)
end

intersection

public intersection(*queries)

Returns the intersection of the listed queries

  r = table.intersection(
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'es'
    },
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'li'
    }
  )

will return a hash { primary_key => record } of the values matching the first query AND the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

[View source]


490
491
492
493
# File 'lib/rufus/tokyo/cabinet/table.rb', line 490

def intersection (*queries)

  search(:intersection, *queries)
end

keys

public keys(options = {})

Returns an array of all the primary keys in the table

With no options given, this method will return all the keys (strings) in a Ruby array.

  :prefix --> returns only the keys who match a given string prefix

  :limit --> returns a limited number of keys

  :native --> returns an instance of Rufus::Tokyo::List instead of
    a Ruby Hash, you have to call #free on that List when done with it !
    Else you're exposing yourself to a memory leak.
[View source]


292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
# File 'lib/rufus/tokyo/cabinet/table.rb', line 292

def keys (options={})

  outlen = nil

  if pre = options[:prefix]

    l = lib.tab_fwmkeys(
      @db, pre, Rufus::Tokyo.blen(pre), options[:limit] || -1)

    l = Rufus::Tokyo::List.new(l)

    options[:native] ? l : l.release

  else

    limit = options[:limit] || -1
    limit = nil if limit < 1

    l = options[:native] ? Rufus::Tokyo::List.new : []

    lib.tab_iterinit(@db)

    outlen = FFI::MemoryPointer.new(:int)

    loop do
      break if limit and l.size >= limit
      out = lib.tab_iternext(@db, outlen)
      break if out.address == 0
      l << out.get_bytes(0, outlen.get_int(0))
    end

    l
  end

ensure

  outlen && outlen.free
end

lget

public lget(keys)

Also known as: mget

No ‘misc’ methods for the table library, so this lget is equivalent to calling get for each key. Hoping later versions of TC will provide a mget method.

[View source]


352
353
354
355
# File 'lib/rufus/tokyo/cabinet/table.rb', line 352

def lget (keys)

  keys.inject({}) { |h, k| k = k.to_s; v = self[k]; h[k] = v if v; h }
end

lib

public lib

Using the cabinet lib

[View source]


158
159
160
161
# File 'lib/rufus/tokyo/cabinet/table.rb', line 158

def lib

  CabinetLib
end

path

public path

Returns the path to the table.

[View source]


165
166
167
168
# File 'lib/rufus/tokyo/cabinet/table.rb', line 165

def path

  @path
end

pointer

public pointer

Returns the actual pointer to the Tokyo Cabinet table

[View source]


446
447
448
449
# File 'lib/rufus/tokyo/cabinet/table.rb', line 446

def pointer

  @db
end

prepare_query

public prepare_query(&block)

Prepares a query instance (block is optional)

[View source]


368
369
370
371
372
373
374
# File 'lib/rufus/tokyo/cabinet/table.rb', line 368

def prepare_query (&block)

  q = TableQuery.new(self)
  block.call(q) if block

  q
end

query

public query(&block)

Prepares and runs a query, returns an array of hashes (all Ruby) (takes care of freeing the query and the result set structures)

[View source]


393
394
395
396
397
398
399
400
401
402
# File 'lib/rufus/tokyo/cabinet/table.rb', line 393

def query (&block)

  rs = do_query(&block)
  a = rs.to_a

  return a

ensure
  rs && rs.free
end

query_delete

public query_delete(&block)

Prepares a query and then runs it and deletes all the results.

[View source]


406
407
408
409
410
411
412
413
414
415
# File 'lib/rufus/tokyo/cabinet/table.rb', line 406

def query_delete (&block)

  q = prepare_query(&block)
  rs = q.delete

  return rs

ensure
  q && q.free
end

search

public search(type, *queries)

A #search a la ruby-tokyotyrant (http://github.com/actsasflinn/ruby-tokyotyrant/tree)

  r = table.search(
    :intersection,
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'es'
    },
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'li'
    }
  )

Accepts the symbols :union, :intersection, :difference or :diff as first parameter.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

Meta Tags

Raises:

[ArgumentError]
[View source]


536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
# File 'lib/rufus/tokyo/cabinet/table.rb', line 536

def search (type, *queries)

  run_query = true
  run_query = queries.pop if queries.last == false

  raise(
    ArgumentError.new("pass at least one prepared query")
  ) if queries.size < 1

  raise(
    ArgumentError.new("pass instances of Rufus::Tokyo::TableQuery only")
  ) if queries.find { |q| q.class != TableQuery }

  t = META_TYPES[type]

  raise(
    ArgumentError.new("no search type #{type.inspect}")
  ) unless t

  qs = FFI::MemoryPointer.new(:pointer, queries.size)
  qs.write_array_of_pointer(queries.collect { |q| q.pointer })

  r = lib.tab_metasearch(qs, queries.size, t)

  qs.free

  pks = Rufus::Tokyo::List.new(r).release

  run_query ? lget(pks) : pks
end

set_index

public set_index(column_name, *types)

Sets an index on a column of the table.

Types maybe be :lexical or :decimal.

Recently (TC 1.4.26 and 1.4.27) inverted indexes have been added, they are :token and :qgram. There is an :opt index as well.

Sorry couldn’t find any good doc about those inverted indexes apart from :

  http://alpha.mixi.co.jp/blog/?p=1147
  http://www.excite-webtl.jp/world/english/web/?wb_url=http%3A%2F%2Falpha.mixi.co.%2Fblog%2F%3Fp%3D1147&wb_lp=JAEN&wb_dis=2&wb_submit=<tt>96%7C</tt>%96%F3+

Use :keep to "add" and :remove (or :void) to "remove" an index.

If column_name is :pk or "", the index will be set on the primary key.

Returns true in case of success.

[View source]


220
221
222
223
224
225
226
227
# File 'lib/rufus/tokyo/cabinet/table.rb', line 220

def set_index (column_name, *types)

  column_name = column_name == :pk ? '' : column_name.to_s

  i = types.inject(0) { |i, t| i = i | INDEX_TYPES[t]; i }

  (lib.tab_setindex(@db, column_name, i) == 1)
end

size

public size

Returns the number of records in this table db

[View source]


361
362
363
364
# File 'lib/rufus/tokyo/cabinet/table.rb', line 361

def size

  lib.tab_rnum(@db)
end

tranabort

public tranabort

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction abort’.

[View source]


440
441
442
# File 'lib/rufus/tokyo/cabinet/table.rb', line 440

def tranabort
  libcall(:tctdbtranabort)
end

tranbegin

public tranbegin

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction begin’.

[View source]


422
423
424
# File 'lib/rufus/tokyo/cabinet/table.rb', line 422

def tranbegin
  libcall(:tctdbtranbegin)
end

trancommit

public trancommit

Warning : this method is low-level, you probably only need to use #transaction and a block.

Direct call for ‘transaction commit’.

[View source]


431
432
433
# File 'lib/rufus/tokyo/cabinet/table.rb', line 431

def trancommit
  libcall(:tctdbtrancommit)
end

union

public union(*queries)

Returns the union of the listed queries

  r = table.union(
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'es'
    },
    @t.prepare_query { |q|
      q.add 'lang', :includes, 'li'
    }
  )

will return a hash { primary_key => record } of the values matching the first query OR the second.

If the last element element passed to this method is the value ‘false’, the return value will the array of matching primary keys.

[View source]


468
469
470
471
# File 'lib/rufus/tokyo/cabinet/table.rb', line 468

def union (*queries)

  search(:union, *queries)
end
Generated on Friday, September 18 2009 at 10:35:39 AM by YARD 0.2.3.5 (ruby-1.8.7).