Containers¶
At the most basic level, Redis acts like an in-memory Python dictionary:
>>> db['walrus'] = 'tusk'
>>> print(db['walrus'])
tusk
>>> db['not-here']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/charles/pypath/redis/client.py", line 817, in __getitem__
raise KeyError(name)
KeyError: 'not-here'
>>> db.get('not-here') is None
True
Redis also supports several primitive data-types:
Hash: dictionaryList: linked listZSet: a sorted setHyperLogLog: probabilistic data-structure for cardinality estimation.Array: like a Python list (custom data type implemented on top ofHashusing lua scripts).BitField: a bitmap that supports random access.BloomFilter: probabilistic data-structure for testing set membership.For stream types (
Stream: andConsumerGroup) see the Streams documentation.
Let’s see how to use these types.
Hashes¶
The Hash acts like a Python dict.
>>> h = db.Hash('charlie')
>>> h.update(name='Charlie', favorite_cat='Huey')
<Hash "charlie": {'name': 'Charlie', 'favorite_cat': 'Huey'}>
We can use common Python interfaces like iteration, len, contains, etc.
>>> print(h['name'])
Charlie
>>> for key, value in h:
... print(key, '=>', value)
name => Charlie
favorite_cat => Huey
>>> del h['favorite_cat']
>>> h['age'] = 31
>>> print(h)
<Hash "charlie": {'age': '31', 'name': 'Charlie'}>
>>> 'name' in h
True
>>> len(h)
2
Lists¶
The List acts like a Python list.
>>> l = db.List('names')
>>> l.extend(['charlie', 'huey', 'mickey', 'zaizee'])
4L
>>> print(l[:2])
['charlie', 'huey']
>>> print(l[-2:])
['mickey', 'zaizee']
>>> l.pop()
'zaizee'
>>> l.prepend('scout')
4L
>>> len(l)
4
Sets¶
The Set acts like a Python set.
>>> s1 = db.Set('s1')
>>> s2 = db.Set('s2')
>>> s1.add(*range(5))
5
>>> s2.add(*range(3, 8))
5
>>> s1 | s2
{'0', '1', '2', '3', '4', '5', '6', '7'}
>>> s1 & s2
{'3', '4'}
>>> s1 - s2
{'0', '1', '2'}
>>> s1 -= s2
>>> s1.members()
{'0', '1', '2'}
>>> len(s1)
3
Sorted Sets (ZSet)¶
The ZSet acts a bit like a sorted dictionary, where the values are the scores used for sorting the keys.
>>> z1 = db.ZSet('z1')
>>> z1.add({'charlie': 31, 'huey': 3, 'mickey': 6, 'zaizee': 2.5})
4
>>> z1['huey'] = 3.5
Sorted sets provide a number of complex slicing and indexing options when retrieving values. You can slice by key or rank, and optionally include scores in the return value.
>>> z1[:'mickey'] # Who is younger than Mickey?
['zaizee', 'huey']
>>> z1[-2:] # Who are the two oldest people?
['mickey', 'charlie']
>>> z1[-2:, True] # Who are the two oldest, and what are their ages?
[('mickey', 6.0), ('charlie', 31.0)]
There are quite a few methods for working with sorted sets, so if you’re curious then check out the ZSet API documentation.
HyperLogLog¶
The HyperLogLog provides an estimation of the number of distinct elements in a collection.
>>> hl = db.HyperLogLog('hl')
>>> hl.add(*range(100))
>>> len(hl)
100
>>> hl.add(*range(1, 100, 2))
>>> hl.add(*range(1, 100, 3))
>>> len(hl)
102
Arrays¶
The Array type is implemented using lua scripts. Unlike List which is implemented as a linked-list, the Array is built on top of a Redis hash and has better run-times for certain operations (indexing, for instance). Like List, Array acts like a Python list.
>>> a = db.Array('arr')
>>> a.extend(['foo', 'bar', 'baz', 'nugget'])
>>> a[-1] = 'nize'
>>> list(a)
['foo', 'bar', 'baz', 'nize']
>>> a.pop(2)
'baz'
BitField¶
The BitField type acts as a bitmap that supports random access
read, write and increment operations. Operations use a format string (e.g. “u8”
for unsigned 8bit integer 0-255, “i4” for signed integer -8-7).
>>> bf = db.bit_field('bf')
>>> resp = (bf
... .set('u8', 8, 255)
... .get('u8', 0) # 00000000
... .get('u4', 8) # 1111
... .get('u4', 12) # 1111
... .get('u4', 13) # 111? -> 1110
... .execute())
...
[0, 0, 15, 15, 14]
>>> resp = (bf
... .set('u8', 4, 1) # 00ff -> 001f (returns old val, 0x0f).
... .get('u16', 0) # 001f (00011111)
... .set('u16', 0, 0)) # 001f -> 0000
...
>>> for item in resp: # bitfield responses are iterable!
... print(item)
...
15
31
31
>>> resp = (bf
... .incrby('u8', 8, 254) # 0000 0000 1111 1110
... .get('u16', 0)
... .incrby('u8', 8, 2, 'FAIL') # increment 254 -> 256? overflow!
... .incrby('u8', 8, 1) # increment 254 -> 255. success!
... .incrby('u8', 8, 1) # 255->256? overflow, will fail.
... .get('u16', 0))
...
>>> resp.execute()
[254, 254, None, 255, None, 255]
BitField also supports slice notation, using bit-offsets. The
return values are always unsigned integers:
>>> bf.set('u8', 0, 166).execute() # 10100110
166
>>> bf[:8] # Read first 8 bits as unsigned byte.
166
>>> bf[:4] # 1010
10
>>> bf[4:8] # 0110
6
>>> bf[2:6] # 1001
9
>>> bf[6:10] # 10?? -> 1000
8
>>> bf[8:16] # ???????? -> 00000000
0
>>> bf[:8] = 89 # 01011001
>>> bf[:8]
89
>>> bf[:8] = 255 # 1111 1111
>>> bf[:4] # 1111
15
>>> del bf[2:6] # 1111 1111 -> 1100 0011
>>> bf[:8] # 1100 0011
195
BloomFilter¶
A BloomFilter is a probabilistic data-structure used for answering
the question: “is X a member of set S?” The bloom-filter may return a false
positive, but it is impossible to receive a false negative (in other words, if
the bloom-filter contains a value, it will never erroneously report that it
does not contain such a value). The accuracy of the bloom-filter and the
likelihood of a false positive can be reduced by increasing the size of the
bloom-filter buffer. The default size is 64KB (or 524,288 bits).
>>> bf = db.bloom_filter('bf') # Create a bloom-filter, stored in key "bf".
>>> data = ('foo', 'bar', 'baz', 'nugget', 'this is a test', 'testing')
>>> for item in data:
... bf.add(item) # Add the above items to the bloom-filter.
...
>>> for item in data:
... assert item in bf # Verify that all items are present.
...
>>> for item in data:
... assert item.upper() not in bf # FOO, BAR, etc, are *not* present.
... assert item.title() not in bf # Foo, Bar, etc, are *not* present.
...
BloomFilter implements only two methods:
add()- to add an item to the bloom-filter.contains()- test whether an item exists in the filter.
Note
Items cannot be removed from a bloom-filter.
Warning
Once a BloomFilter has been created and items have been added,
you must not modify the size of the buffer.