HTTP Crate Client

Server configuration

A list of servers can be passed while creating an instance of the http client:

>>> http_client = HttpClient([crate_host])

Its also possible to pass a single server as a string:

>>> http_client = HttpClient(crate_host)

If no server` argument (or no argument at all) is passed, the default one ``127.0.0.1:9200 is used:

>>> http_client = HttpClient()
>>> http_client._active_servers
['127.0.0.1:9200']

When using a list of servers, the servers are selected by round-robin:

>>> invalid_host = "invalid_host:9999"
>>> even_more_invalid_host = "even_more_invalid_host:9999"
>>> http_client = HttpClient([crate_host, invalid_host, even_more_invalid_host])
>>> http_client._get_server()
'127.0.0.1:9295'

>>> http_client._get_server()
'invalid_host:9999'

>>> http_client._get_server()
'even_more_invalid_host:9999'

Servers with connection errors will be removed from the active server list:

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host])
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers
['127.0.0.1:9295']

Inactive servers will be re-added after a given time interval. To validate this, set the interval very short and sleep for that interval:

>>> http_client.retry_interval = 1
>>> import time; time.sleep(1)
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers
['invalid_host:9999', 'even_more_invalid_host:9999', '127.0.0.1:9295']

If no active servers are available and the retry interval is not reached, just use the oldest inactive one:

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host])
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers = []
>>> http_client._get_server()
'invalid_host:9999'

SQL Statements

Issue a select statement against our with test data pre-filled crate instance:

>>> http_client = HttpClient(crate_host)
>>> result = http_client.sql('select name from locations')
>>> pprint(result)
{u'cols': [u'name'],
 u'rows': [[u'Algol'],
           [u'Folfanga'],
           [u'Aldebaran'],
           [u'Argabuthon'],
           [u'Bartledan'],
           [u'Galactic Sector QQ7 Active J Gamma'],
           [u'Allosimanius Syneca'],
           [u'Arkintoofle Minor'],
           [u'Outer Eastern Rim'],
           [u'Altair']]}

Blobs

Check if a blob exists:

>>> http_client.blob_exists('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
False

Trying to get a non-existing blob throws an exception:

>>> http_client.blob_get('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
Traceback (most recent call last):
...
DigestNotFoundException: myfiles/040f06fd774092478d450774f5ba30c5da78acc8

Creating a new blob - this method returns True if the blob was newly created, false if it already exists:

>>> from tempfile import TemporaryFile
>>> f = TemporaryFile()
>>> _ = f.write(b'content')
>>> _ = f.seek(0)
>>> http_client.blob_put(
...     'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', f)
True

>>> _ = f.seek(0)
>>> http_client.blob_put(
...     'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', f)
False

The blob should now exist:

>>> http_client.blob_exists('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
True

Blobs are returned as generators, generating a chunk on each call:

>>> g = http_client.blob_get('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
>>> print(next(g))
content

The chunk_size can be set explicitly on get:

>>> g = http_client.blob_get(
...     'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', 5)
>>> print(next(g))
conte

>>> print(next(g))
nt

Deleting a blob - this method returns true if the blob existed:

>>> http_client.blob_del('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
True

>>> http_client.blob_del('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8')
False

Error Handling

It’s possible to define a HTTP timeout in seconds on client instantiation, so a timeout exception is raised when timeout is reached:

>>> http_client = HttpClient(crate_host, timeout=0.001)
>>> http_client.sql('select name from locations')
Traceback (most recent call last):
...
ConnectionError: No more Servers available, exception from last server: HTTPConnectionPool(host='...', port=...): Request timed out. (timeout=0.001)