Client applications¶

REST API¶

Slivka provides REST API which allows you to develop applications that interact with the servers. The interactive view of available API endpoints is available at api/ endpoint of every Slivka server.

Python¶

If you are using Python, there is an official Slivka client library that abstracts communication with the server. The most recent version of the library can be installed from our github repository:

pip install git+https://github.com/warownia1/slivka-python-client.git

Usage¶

Creating client¶

Once the library is installed, you can import SlivkaClient from slivka_client module and create its instance using a single url argument or provide scheme, auth, host, port and path as individual keyword arguments.

Warning

Remember to add trailing slash at the end of the path. example.org/slivka means slivka resource under / path, while example.org/slivka/ indicates resources under /slivka/ path.

>>> from slivka_client import SlivkaClient

>>> client = SlivkaClient('http://example.org/slivka/')
>>> # or SlivkaClient(schema='http', host='example.org', path='slivka/')
>>> client.url
Url(scheme='http', auth=None, host='example.org',
    port=None, path='/slivka/', query=None, fragment=None)

slivka_client.SlivkaClient object is used for services discovery, file uploading and retrieval and job monitoring.

Services listing and job submission¶

The list of available services can be accessed through the services property of the client object. The server will be contacted to provide the list of services on the first property access. Subsequent attempts will return the cached list immediately. Use refresh_services() method whenever you want to reload the list of services from the server. If you need to fetch one service by its name you can use item access syntax client[key] where key is the service name.

>>> client.services
[SlivkaService(clustalo), SlivkaService(mafft)]
>>> service = client['clustalo']
>>> service.name
'clustalo'

Each slivka_client.Service object represents a single service on the server side. They provide access to service information as well as the submission forms needed to send new job requests to the server.

A slivka_client.Form stores the collection of parameters needed to run a job. A new empty form is created every time slivka_client.Service.form attribute is accessed or slivka_client.Service.new_form() method is called. Forms are dictionary-like objects providing a view of the service input parameters, storing input values and mediating job submission to the server. Iterating over the form directly or accessing fields gives an iterable of fields each of them storing the information about its corresponding input parameter and its constraints. Individual field information may be retrieved using item access syntax form[key] where key is the field name.

Suppose that the ClustalO service takes the following parameters: input-file, dealign and iterations. Then, iterating over the form would produce the following output

>>> for field in form:
...     print(field)
FileField(name='input-file', label='Input file', description'...',
          required=True, default=None, multiple=False,
          media_type='application/fasta', media_type_parameters={})
BooleanField(name='dealign', label='Dealign input sequences',
             description='...', required=False, default=False,
             multiple=False)
IntegerField(name='iterations', label='Number of iterations (combined)',
             description='...', required=False, default=1, multiple=False,
             min=1, max=100)

The properties common for all field objects are:

type: type of the field (see FieldType)
name: field name/identifier
label: short human-readable label
description: longer, more detailed description
required: whether the field is required or not
default: default value or None if not provided
multiple: whether multiple values are accepted

Additionally, each field may have extra constrains according to its type. All form fields and their respective properties are listed in the API Reference section Form Fields.

In order to provide the input values for the form, you can set the value of the field using item assignment form[key] = value or calling form’s set(key, value) method, where key is the field name and value is the value to be set. This method replaces the old value with the new one. The value should be of the type matching the field type or a list of such values whenever multiple values are allowed. In most cases the following types are expected for each field:

IntegerField: int
DecimalField: float
TextField: str
BooleanField: bool
ChoiceField: str
FileField: str (file uuid) or slivka_client.File or io.IOBase

However, these values might differ depending on the service requirements and custom server-side validation. For more information refer to the service documentation on the particular server you want to access.

If the field accepts multiple values, you can either set it to the list of values using the method above or you can use Form.append(key, value) method to append a single value or Form.extend(key, iterable) to append all values from the iterable to the end of the values list. Files can be provided using special File objects described below as well as passing open streams.

Note

The client application will NOT attempt to seek to the beginning of the stream. It’s up to the user to prepare the stream in the proper state before submitting the job.

Internally, the values are stored in the defaultdict having an empty list as a default factory. That dictionary may be accessed and manipulated directly (it’s not recommended though) through the values attribute of the slivka_client.Form object. Keep in mind that methods slivka_client.Form.append() and slivka_client.Form.extend() use the respective methods of the underlying lists in the values dictionary. If the value was previously set to something not implementing MutableSequence those calls will raise AttributeError.

The inserted values can be removed using item deletion del form[key] or delete(key) method. Additionally, clear() deletes all the values from the form and returns it to its initial state.

Once the form is populated, a new job request can be submitted to the server using Form.submit() method that returns job uuid or raises an exception if the submission was not successful. Additionally, you can supply one-off input values passing a mapping as an argument or providing the values as keyword arguments to submit(). Those will be added on top of the existing field values and will not persist in the form during consecutive submits.

Example of submitting the job to ClustalO service. Two parameters input-file and iterations are stored in the form while dealign is set for this single submission only.

>>> form['input-file'] = open('/path/to/input.txt', 'rb')
>>> form['iterations'] = 100
>>> job_id = form.submit(dealign=True)
>>> job_id
'1WsqR3H2RwOxTps1XI22xQ'

Retrieving job state and result¶

Once the job is successfully submitted its state can be checked using slivka_client.SlivkaClient.get_job_state() which takes job uuid returned by slivka_client.Form.submit() and returns the slivka_client.JobState.

You can fetch the list of results even if the job is still running using slivka_client.SlivkaClient.get_job_results() providing job uuid as an argument. Keep in mind that the results may be incomplete when the job is in unfinished state. The method returns the list of file handlers (more information in the next section) that can be used to inspect files’ metadata and stream their content.

Working with File objects¶

Instead of working with files directly, Slivka assigns a uuid to each file and uses it instead. This approach helps to avoid sending the same data over the internet multiple times and allows using the output of one service as an input to the other without the need to download and re-upload the file.

On the client side, files present on the server are represented as File instances. You obtain them when uploading the file to the server or retrieving job results. Those object are subclasses of string and are equal to file’s uuid. They also provide additional meta-data it their properties such as title, label, media_type and url.

The File objects, however, does not store file content. If you want to download the content either to the file, or to the memory you can use dump() method and pass it either the path to the output file or an open writable stream. It’s highly recommended to use binary streams, but in many cases text streams should work as well.

Example of uploading and downloading the file from the server:

>>> file = client.upload_file(open('/path/to/file.txt'))
>>> file
File(uploaded, [b2uEAxmrS6mZzLIqkkbfXQ])

>>> # save to the file at the specified location
>>> file.dump('/path/to/output.txt')

>>> # open the file for writing and write to it
>>> with open('/path/to/output_file.txt', 'wb') as fp:
...    file.dump(fp)

>>> import io
>>> # create in-memory buffer and write to it
>>> stream = io.BytesIO()
>>> file.dump(stream)
>>> content = stream.getvalue()

If you want to use the same input file multiple times, it’s a good idea to upload it once and re-use the file handler to speed up the submission process, save time and bandwidth.

API Reference¶

class slivka_client.SlivkaClient(url)¶

class slivka_client.SlivkaClient(*, scheme, auth, host, port, path)

url¶

Get the URL the client will connect to.

Return type: urllib3.util.url.Url

services¶

Return the list of services.

Return type: list[slivka_client.Service]

refresh_services()¶: Force reloading the services list from the server.

upload_file(file: Union[str, io.BufferedIOBase], title: str = '') → slivka_client.file.File¶

Upload the file to the server and obtain its handler.

file can be either a stream open for reading or a path. If path is provided, the file will be opened in binary mode. Optionally, the file title can be specified.

Returns: handler to the file on the server.
Return type: slivka_client.File

get_file(id: str) → slivka_client.file.File¶: Fetch the file handler using file id.

get_job_state(uuid: str) → slivka_client.state.JobState¶

Check the state of the job.

Parameters: uuid – job uuid assigned on submission

get_job_results(uuid: str) → List[slivka_client.file.File]¶

Fetch the output files of the job

Parameters: uuid – job uuid
Returns: list of the output files
Return type: list[slivka_client.File]

class slivka_client.Service¶

A class representing one of the services on the server.

property name → str¶: Name (identifier) of the service.

property label → str¶: Human-friendly label.

property url → str¶: Url where the service is available at.

property classifiers → List[str]¶: List of tags or classifiers that the service has. They can be set arbitrarily by the server administrator to help identifying the service types.

new_form() → Form¶

property form → Form¶: Creates and returns new submission form. A new empty form is created every time the the property is accessed or the method is called.

refresh_form()¶: Forces the form data to be reloaded from the server.

class slivka_client.Form¶

Representation of a new job submission request form.

__delitem__(key)¶: Clear the field specified by key.

__getitem__(key) → slivka_client.form._BaseField¶: Return the description of the field named key.

__iter__() → Iterator[slivka_client.form._BaseField]¶: Return an iterator over all form fields.

__setitem__(key, value)¶: Set the value of the field key to value.

append(key, value)¶: Append value to the list of values of key field.

clear()¶: Erase all values from this form

copy() → slivka_client.form.Form¶: Create and return an empty copy of that form.

extend(key, iterable)¶: Append all values from the iterable to the field values.

property fields¶: View of all fields in this form.

property name¶: Name of the form.

set(key, value)¶: Set the value of the field key to value.

submit(_items=(), **kwargs) → str¶

Submit the form to the server and start a new job. You can provide an optional dictionary of input parameters that will be used in this submission only or pass them as keyword arguments.

Parameters

_items – dict of submission parameters
kwargs – submission parameters as kwargs

Returns

job uuid

property url¶: Url location of the form.

property values¶: Dictionary of entered values.

class slivka_client.FieldType¶

An enumeration.

BOOLEAN = 'boolean'¶

CHOICE = 'choice'¶

DECIMAL = 'decimal'¶

FILE = 'file'¶

FLAG = 'boolean'¶: Alias of FieldType.BOOLEAN

FLOAT = 'decimal'¶: Alias of FieldType.DECIMAL

INT = 'integer'¶: Alias of FieldType.INTEGER

INTEGER = 'integer'¶

TEXT = 'text'¶

UNDEFINED = 'undefined'¶

class slivka_client.JobState¶

An enumeration.

ACCEPTED = 'accepted'¶: Request was accepted and is scheduled for execution

COMPLETED = 'completed'¶: Job has finished its execution successfully

DELETED = 'deleted'¶: Job was deleted from the queuing system

ERROR = 'error'¶: Job failed to run due to internal error

FAILED = 'failed'¶: Job finished with non-zero exit code

INTERRUPTED = 'interrupted'¶: Job was interrupted during execution

PENDING = 'pending'¶: Request was submitted successfully and awaits further processing

QUEUED = 'queued'¶: Job was sent to the queuing system and waits for resources to become available

REJECTED = 'rejected'¶: Request was rejected due to resource limitations

RUNNING = 'running'¶: Job is currently running

UNKNOWN = 'unknown'¶: Job status can not be determined

class slivka_client.File¶

dump(fp)¶

property label¶

property media_type¶

property title¶

property url¶

property uuid¶

Form Fields¶

class slivka_client.form._BaseField¶

The base for other fields. This class is never instantiated directly but provides common attributes for deriving types.

default¶: default value

description¶: longer description

label¶: short human-readable label

multiple¶: whether multiple values are allowed

name¶: field name/identifier

required¶: whether the field is required

type¶: field type

class slivka_client.form.UndefinedField¶

Class for all custom or unrecognised fields.

attributes¶: dictionary of field parameters as provided by the server

type¶

Value: FieldType.UNDEFINED

class slivka_client.form.IntegerField¶

max¶: maximum value constraint

min¶: minimum value constraint

type¶

Value: FieldType.INTEGER

class slivka_client.form.DecimalField¶

max¶: maximum value constraint

max_exclusive¶: whether the maximum value is excluded

min¶: minimum value constraint

min_exclusive¶: whether the minimum value is excluded

type¶

Value: FieldType.DECIMAL

class slivka_client.form.TextField¶

max_length¶: maximum length of the text

min_length¶: minimum length of the text

type¶

Value: FieldType.TEXT

class slivka_client.form.BooleanField¶

type¶

Value: FieldType.BOOLEAN

class slivka_client.form.ChoiceField¶

choices¶: list of available choices

type¶

Value: FieldType.CHOICE

class slivka_client.form.FileField¶

media_type¶: media type of the file

media_type_parameters¶: additional annotations regarding file content

type¶

Value: FieldType.FILE

Client applications¶

REST API¶

Python¶

Usage¶

Creating client¶

Services listing and job submission¶

Retrieving job state and result¶

Working with File objects¶

API Reference¶

Form Fields¶

Table of Contents

Previous topic

This Page